The OVAL analyzer falsely assumes that any vulnerabilities detected on a
host only come from OVAL. However, it is possible that NVD detects
vulnerabilities on these hosts even though it excludes software from
deb_packages and rpm_packages. For example, a python package twisted
v22.20 has a vulnerability CVE-2022-39348 detected by NVD. The OVAL
analyzer would delete this vulnerability, and it would be re-inserted by
the NVD scanner on the next run. This creates a loop.
The fix is to only delete vulnerabilities that are actually detected
using OVAL. We already store this in the source column in the
software_cve table.
* WIP
* Adding DEP functionality to Fleet
* Better organize additional MDM code
* Add cmdr.py and amend API paths
* Fix lint
* Add demo file
* Fix demo.md
* go mod tidy
* Add munki setup to Fleet
* Add diagram to demo.md
* Add fixes
* Update TODOs and demo.md
* Fix cmdr.py and add TODO
* Add endpoints to demo.md
* Add more Munki PoC/demo stuff
* WIP
* Remove proposals from PoC
* Replace prepare commands with fleetctl commands
* Update demo.md with current state
* Remove config field
* Amend demo
* Remove Munki setup from MVP-Dogfood
* Update demo.md
* Add apple mdm commands (#7769)
* fleetctl enqueue mdm command
* fix deps
* Fix build
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
* Add command to upload installers
* go mod tidy
* fix subcommands help
There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.
* Add support for installing apps
* Add a way to list enrolled devices
* Add dep listing
* Rearrange endpoints
* Move DEP routine to schedule
* Define paths globally
* Add a way to list enrollments and installers
* Parse device-ids as comma-separated string
* Remove unused types
* Add simple commands and nest under enqueue-command
* Fix simple commands
* Add help to enqueue-command
* merge apple_mdm database
* Fix commands
* update nanomdm
* Split nanomdm and nanodep schemas
* Set 512 MB in memory for upload
* Remove empty file
* Amend profile
* Add sample commands
* Add delete installers and fix bug in DEP profile assigning
* Add dogfood.md deployment guide
* Update schema.sql
* Dump schema with MySQL 5
* Set default value for authenticate_at
* add tokens to enrollment profiles
When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.
- Rename enrollments to enrollment profiles. Enrollments is used by nano
to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=
* remove mdm apple server url
* update docs
* make dump-test-schema
* Update nanomdm with missing prefix table
* Add docs and simplify changes
* Add changes file
* Add method docs
* Fix compile and revert prepare.go changes
* Revert migration status check change
* Amend comments
* Add more docs
* Clarify storage of installers
* Remove TODO
* Remove unused
* update dogfood.md
* remove cmdr.py
* Add authorization tests
* Add TODO comment
* use kitlog for nano logging
* Add yaml tags
* Remove unused flag
* Remove changes file
* Only run DEP routine if MDM is enabled
* Add docs to all new exported types
* Add docs
* more nano logging changes
* Fix unintentional removal
* more nano logging changes
* Fix compile test
* Use string for configs and fix config test
* Add docs and amend changes
* revert changes to basicAuthHandler
* remove exported BasicAuthHandler
* rename rego authz type
* Add more information to dep list
* add db tag
* update deps
* Fix schema
* Remove unimplemented
Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
* Ingest installed Windows updates and store them in the windows_updates table.
* Added config option for enabling/disabling Windows update ingestion and Windows OS vuln. detection.
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts
To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
not set on the INSERT.
- OUT: Only sets the ID on the passed session and returns it. (`CreatedAt`, `AccessedAt`, are not set.)
New version:
```go
func (ds *Datastore) NewSession(ctx context.Context, userID uint, sessionKey string) (*fleet.Session, error) {
sqlStatement := `
INSERT INTO sessions (
user_id,
` + "`key`" + `
)
VALUES(?,?)
`
result, err := ds.writer.ExecContext(ctx, sqlStatement, userID, sessionKey)
if err != nil {
return nil, ctxerr.Wrap(ctx, err, "inserting session")
}
id, _ := result.LastInsertId() // cannot fail with the mysql driver
return ds.sessionByID(ctx, ds.writer, uint(id))
}
```
- IN: Define arguments that are truly used when creating a session.
- OUT: Load and return the fleet.Session struct with all values set (using the `ds.writer` to support read replicas correctly).
PS: The new `NewSession` version mimics what we already do with other entities, like policies (`Datastore.NewGlobalPolicy`).
* Add CentOS parsing and post-processing in fleet
* Add tests and amend SyncCPEDatabase
* Add test for centosPostProcessing
* Changes from PR comments
* Amend software test
* Fix sync test
* Add index to source and vendor
* Use os.MkdirTemp
* Rearrange migrations
* Regenerate test schema
* Add support for testing migrations (#4112)
* Add support for testing migrations
* Rename migration in tests
* Changes suggested in PR
* Go mod tidy
* WIP
* Amend tests
* Do not load aggregated stats for packs
* Add option to host lite
* Fix remaining TODOs
* Fix osquery_utils tests
* Fix SQL
* Fix SQL (bis)
* Restore AuthenticateHost to load once
* Code improvements and re-add deferred host save
* More fixes to the PR
* Wrap users table update on tx
* Add caching to ListPacksForHost and ListScheduledQueriesInPack
* Remove SaveHostSoftware (replaced by UpdateHostSoftware)
* Add unit tests for new functionality
* Add changes file
* Fix scheduled queries test
* Remove cleanups and fk for host ids
* Readd missing things to the schema
* Remove unused
* Add changes file and fix some error messages
* Fix test
* Use tx instead of plain writer
* Other fixes
* More not found test fixes
* Go back to getcontext
* Add webhook to app config
* Add redis failing policies set and webhook
* Add basic webhook test
* Store hostname in redis
* Global policy deletion to remove policy ID from set and config
* Also process new passing policies
* Fix unit test
* Sort hosts
* Add more tests
* Add ListSets to the failing policies interface
* Fix server URL and garbage collect on the triggering side
* Do not use Redis SCAN
* Fix Redis operation order
* Add API changes to doc
* Add comments
* Add more tests
* Fix tests
* Add tests for config update upon deletion of policies
* Run make dump-test-schema
* Ignore policies that failed to run
* Add proper unit tests to trigger logic
* Fix comments
* WIP
* Add tests to service_osquerty_test.go
* Use SSCAN for listing hosts instead of SMEMBERS
* Add failing policies to docs/01-Using-Fleet/configuration-files/README.md
* Remove skip
* Fix PR comments
* Draft for mdm and munki as direct ingest funcs
* Expose mdm/munki over API
* Add test for service and fix bug
* Update queries test
* Fix lint
* Address review comments
* Add software count API
* Fix makefile
* Fine no mock generating at this point
* Actually, one last try
* Use go install instead
* Fix go sum/mod
* Improve documentation
* Try setting node to 14
* Add fleetctl command to get missing migrations
* Fix copy paste and lint
* Detect migrations applied out of order
* Add extra bullet to changes
* Trigger creation of migration status tables
* Fix unit tests
* PR fixes
* PR comment fixes
* Further optimize host select with policies
* Add missing row close
* Improve migration
* Skip migration if column already exists
* Add test for host with/without extras
* Serialize hosts writes per instance
* Write hosts asynchronously
* Dont make the save in a goroutine
* Revert "Dont make the save in a goroutine"
This reverts commit 4a890c5271142755dec69a741582e7eca5c4c62c.
* Make all savehosts async
* Address review comments and make this approach configurable
* Address review comments
* Disable bulk seen time marking for a test
* Move host seen times to a new table
* Remove unused
* Add seen_time to list hosts
* Add some jitter to seen time flushing
* Remove unused
* Add timeout to deferred save host
* Add tests for serialSaveHost
* Update hosts in labels and policy executions in a serial way
* Address review comments and remove fk constraints in host software
* Make errCh buffered
* Add changes file
* Readd key
* wip
* wip
* wip
* Add performance stats
* Add docs and other self review fixes
* lint
* Update based on review comments
* Add quick cleanup first and then reset to 1hr
* Reduce the load in the test
* Cache app config in redis
* Add changes files
* Replace string with constant
* Revert some test refactorign and duplicate a bit of test code
* Add test for AppConfig with redis failing
* Fix lint
* Use Doer so it works better in clusters
* Skip unmarshalling if we already did
* Allow to cache hosts if configured
* Omit the setting if empty
* Remove hashing, too much CPU
* Revert caching of host auth... needs a more thought through approach
* Remove config
* Remove old config
* Remove locker interface
* Fix test and address review comments
* Add max jitter percent config
* Fix jitter calc
* Remove comment
* Reduce test jitter to make tests less flaky
* Remove jitter entirely
* Document new config
* Fix doc link
* Add team policies
* Add team policy documentation
* Add changes file
* Update titles
* Fix lint
* Rewrite TeamAuthorize for more clarify
* Explicitly use two slices for clarity
* Simplify switch
* Implement fleetctl get software and the underlying API
* Add documentation
* Simplify list software implementation
* Lint fixes
* Make team name unique
* Address review comments
* Fix lint
* Fix tests
