Load queries before starting simulated hosts -- This is a much quicker
step and allows the user to see the queries in the interface while they
wait for hosts to start.
* Fix build, add missing tests for cpe translations
Also dont fail alltogether if there's one issue translating CPEs, log it and continue
* Make it once every hour again
* Use MATCH but escape strings
* add team_id filter to fleetctl via get hosts --team flag & api via api/v1/fleet/hosts and api/v1/fleet/labels/id/hosts
* update tests & add changes file
* Add safe mkdirall and open
* Use secure as much as possible and merge gomodules for orbit to fleet
* Improve openfile and mkdirall to check for permissiveness instead of equality
* Don't shift
* Fix links
* Address review comments
* WIP
* WIP
* Make path optional and fix tests
* Add first generate
* Move to nvd package
* remove replace
* Re-add replace
* It's path, not file name
* Change how db path is set and use etag
* Fix typos
* Make db generation faster
* Remove quotes
* Doesn't like comments
* Samitize etag and save to file
* Refactor some things and improve writing of etagenv
* Compress file and truncate amount of items for faster testing
* Remove quotes
* Try to improve performance
* Ignore truncate error if not exists
* Minor cleanup and make sqlite have cpe prefix
* Simplify code and test sync
* Add VCR for sync test
* Check for nvdRelease nil
* Add test for the actual translation
* Address review comments
* Rename generate command because we'll have a cve one too
* Move to its own dir
* Add first cve db generation
* WIP but with final strategy, preparring to merge main
* Fix merge conflicts
* WIP
* wip
* Insert CVEs to the db
* Remove unused code
* Use wg instead of counting
* Call cancelFunc to avoid ctx leak
* Fix logs for better readability
* Point code to fleetdm instead of my repo
* Add basic idea
* Implement the new logging strategy everywhere
* Remove unused const
* Add tests and fix error cases
* Fix logging in osquery service
* If there are extras, log info unless force debug
* Change to info
* Fix test
* Make logging context more chainable and force info for sessions
* WIP
* WIP
* Make path optional and fix tests
* Add first generate
* Move to nvd package
* remove replace
* Re-add replace
* It's path, not file name
* Change how db path is set and use etag
* Fix typos
* Make db generation faster
* Remove quotes
* Doesn't like comments
* Samitize etag and save to file
* Refactor some things and improve writing of etagenv
* Compress file and truncate amount of items for faster testing
* Remove quotes
* Try to improve performance
* Ignore truncate error if not exists
* Minor cleanup and make sqlite have cpe prefix
* Simplify code and test sync
* Add VCR for sync test
* Check for nvdRelease nil
* Add test for the actual translation
* Address review comments
* Rename generate command because we'll have a cve one too
* Move to its own dir
* Address review comments
* fleetctl preview downloads standard query library and populates console
* add change log file, use specGroupFromBytes to parse standard query library file
* update printHost to serialize HostResponse rather than just Host
* added change log file
* Update changes/issue-1373-add-status-to-fleetctl-get-command
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* WIP
* Send usage analytics
* Improve loggin of cron tasks and fix test
* Implement appconfig method now that we are checking that as well
* Address review comments
* WIP
* Add get user_roles and apply for a user_roles spec to fleetctl
* Uncomment other tests
* Update test to check output
* Update test with the new struct
* Mock token so that it doesn't pick up the one in the local machine
* Address review comments
* Fix printJSON and printYaml
* Fix merge conflict error
* WIP
* wip
* wip
* Finish implementation
* Address review comments
* Fix flaky test
* WIP
* Add get user_roles and apply for a user_roles spec to fleetctl
* Uncomment other tests
* Update test to check output
* Update test with the new struct
* Mock token so that it doesn't pick up the one in the local machine
* Address review comments
* Fix printJSON and printYaml
* Fix merge conflict error
* If both roles are specified, fail
* Fix test
* Switch arguments around
* Update test with the new rule
* Fix other tests that fell through the cracks
* Remove username from UI code
* Remove username from tests
* Remove username from database
* Modify server endpoints for removing username
* Implement backend aspects of removing username
* Update API docs
* Add name to fleetctl
- Add enable_analytics column to database.
- Allow enable_analytics to be set via API.
- Add messaging in fleetctl setup.
Note that this defaults to off for existing installations, and defaults
on for newly set up installs.
No collection or sending of analytics yet exists, we are strictly
storing the preference at this time.
Part of #454
The previous strategy unintentionally cleared existing environment
values, preventing us from setting a specific version of the Fleet
container to run.
This should support Redis in both cluster and non-cluster modes.
Updates were made separately to github.com/throttled/throttled to support the slight changes in types.
Co-authored-by: Joseph Macaulay <joseph.macaulay@uber.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
- Add policy.rego file defining authorization policies.
- Add Go integrations to evaluate Rego policies (via OPA).
- Add middleware to ensure requests without authorization check are rejected (guard against programmer error).
- Add authorization checks to most service endpoints.
- Move team-related service methods to `ee/server/service`.
- Instantiate different service on startup based on license key.
- Refactor service errors into separate package.
- Add support for running E2E tests in both Core and Basic tiers.
- Add `team_id` field to secrets.
- Remove secret `name` and `active` fields (migration deletes inactive secrets).
- Assign hosts to Team based on secret provided.
- Add API for retrieving secrets by Team.
Some datastore and service methods would return slices of structs,
rather than slices to pointers of structs (which most methods used).
Make this more consistent.
- Add config option for license key.
- Define license details data structure.
- Include license details in app config API responses.
Currently any non-empty value for `--license_key` behaves as though the
installation is licensed for `basic`. If the license key is empty,
`core` is returned.
Still to come is the appropriate parsing for the license key.
This feature enables a new config option (redis.duplicate_results). When set to true, all Live Query results will be copied to an additional Redis pubsub channel named LQDuplicate
This is useful in a scenario that would involve shipping the Live Query results outside of Fleet, near-realtime.
- Allow agent options to be set on per-team basis.
- Move global agent options into app configs.
- Update logic for calculating agent options for hosts.
- Updates to relevant testing.
Instead of synchronously updating the seen_time column for a host on an update, batch these updates to be written together every 1 second.
This results in a ~33% reduction in MySQL CPU usage in a local test with 4,000 simulated hosts and MySQL running in Docker.
