* Add webhook to app config
* Add redis failing policies set and webhook
* Add basic webhook test
* Store hostname in redis
* Global policy deletion to remove policy ID from set and config
* Also process new passing policies
* Fix unit test
* Sort hosts
* Add more tests
* Add ListSets to the failing policies interface
* Fix server URL and garbage collect on the triggering side
* Do not use Redis SCAN
* Fix Redis operation order
* Add API changes to doc
* Add comments
* Add more tests
* Fix tests
* Add tests for config update upon deletion of policies
* Run make dump-test-schema
* Ignore policies that failed to run
* Add proper unit tests to trigger logic
* Fix comments
* WIP
* Add tests to service_osquerty_test.go
* Use SSCAN for listing hosts instead of SMEMBERS
* Add failing policies to docs/01-Using-Fleet/configuration-files/README.md
* Remove skip
* Fix PR comments
* Finish first draft of API versions
* wip
* Finalize tests
* Revert change in handler
* Remove made up version
* Update versioning with aliases
* Add changes file
* Address review comments
* Revert overupdated routes
* Expand life time of deprecated APIs
* Fix test
* Comment out problematic part of test
* Revert bad path changes
* Add extra debug logging for hosts
* Add changes file
* Ignore if appconfig is nil
* Use slice of uints instead of a string
* Debug response request for enabled hosts
* Add host-id to request/response
* Lint fixes
* Add missing AppConfigFuncs
This should support Redis in both cluster and non-cluster modes.
Updates were made separately to github.com/throttled/throttled to support the slight changes in types.
Co-authored-by: Joseph Macaulay <joseph.macaulay@uber.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
- Add policy.rego file defining authorization policies.
- Add Go integrations to evaluate Rego policies (via OPA).
- Add middleware to ensure requests without authorization check are rejected (guard against programmer error).
- Add authorization checks to most service endpoints.
- Migrate old admins to global admins
- Migrate old non-admins to global maintainers
- Remove old admin column
- Give initial user global admin privilege
- Comment out some tests (to be refactored for new permissions model later)
Prevent abuse of these endpoints with rate limiting backed by Redis. The
limits assigned should be appropriate for almost any Fleet deployment.
Closes#530
- Support both /api/v1/fleet and /api/v1/kolide routes in server.
- Add logging for use of deprecated routes.
- Rename routes in frontend JS.
- Rename routes and add notes in documentation.
This change optimizes live queries by pushing the computation of query
targets to the creation time of the query, and efficiently caching the
targets in Redis. This results in a huge performance improvement at both
steady-state, and when running live queries.
- Live queries are stored using a bitfield in Redis, and takes
advantage of bitfield operations to be extremely efficient.
- Only run Redis live query test when REDIS_TEST is set in environment
- Ensure that live queries are only sent to hosts when there is a client
listening for results. Addresses an existing issue in Fleet along with
appropriate cleanup for the refactored live query backend.
- Add the server_url_prefix flag for configuring this functionality
- Add prefix handling to the server routes
- Refactor JS to use appropriate paths from modules
- Use JS template to get URL prefix into JS environment
- Update webpack config to support prefixing
Thanks to securityonion.net for sponsoring the development of this feature.
Closes#1661
An incorrect authorization check allowed non-admin users to modify the details of other users. We now enforce the appropriate authorization so that unprivileged users can only modify their own details.
Thanks to 'Quikke' for the report.
Replaces (and appropriately refactors) a number of endpoints that were removed long ago when we decided to kill the UI with the fleetctl release. We turned out not to do this, and now need to restore these missing endpoints.
This is not a straight up replacement of the existing code because of refactoring to the DB schemas that was also done in the migration.
Most of the replaced code was removed in #1670 and #1686.
Fixes#1811, fixes#1810
- Add new Apply spec methods for queries and packs
- Remove now extraneous datastore/service methods
- Remove import service (unused, and had many dependencies that this breaks)
- Refactor tests as appropriate
Notable refactoring:
- Use stdlib "context" in place of "golang.org/x/net/context"
- Go-kit no longer wraps errors, so we remove the unwrap in transport_error.go
- Use MakeHandler when setting up endpoint tests (fixes test bug caught during
this refactoring)
Closes#1411.
* Initial scaffolding of the host summary endpoint
* inmem datastore implementation of GenerateHostStatusStatistics
* HostSummary docstring
* changing the url of the host summary endpoint
* datastore tests for GenerateHostStatusStatistics
* MySQL datastore implementation of GenerateHostStatusStatistics
* <= and >= to catch exact time edge case
* removing clock interface method
* lowercase error wraps
* removin superfluous whitespace
* use updated_at
* adding a seen_at column to the hosts table
* moving the update of seen_time to the caller
* using db.Get instead of db.Select
This PR adds the `host_ids` and `label_ids` field to the packs HTTP API so that one can operate on the hosts/labels which a pack is scheduled to be executed on. This replaces (and deletes) the `/api/v1/kolide/packs/123/labels/456` API in favor of `PATCH /api/v1/packs/123` and specifying the `label_ids` field. This also allows for bulk operations.
Consider the following API examples:
## Creating a pack with a known set of hosts and labels
The key addition is the `host_ids` and `label_ids` field in both the request and the response.
### Request
```
POST /api/v1/kolide/packs
```
```json
{
"name": "My new pack",
"description": "The newest of the packs",
"host_ids": [1, 2, 3],
"label_ids": [1, 3, 5]
}
```
### Response
```json
{
"pack": {
"id": 123,
"name": "My new pack",
"description": "The newest of the packs",
"platform": "",
"created_by": 1,
"disabled": false,
"query_count": 0,
"total_hosts_count": 5,
"host_ids": [1, 2, 3],
"label_ids": [1, 3, 5]
}
}
```
## Modifying the hosts and/or labels that a pack is scheduled to execute on
### Request
```
PATCH /api/v1/kolide/packs/123
```
```json
{
"host_ids": [1, 2, 3, 4, 5],
"label_ids": [1, 3, 5, 7]
}
```
### Response
```json
{
"pack": {
"id": 123,
"name": "My new pack",
"description": "The newest of the packs",
"platform": "",
"created_by": 1,
"disabled": false,
"query_count": 0,
"total_hosts_count": 5,
"host_ids": [1, 2, 3, 4, 5],
"label_ids": [1, 3, 5, 7]
}
}
```
close#633
* Moving query attributes from the query object to the pack-query relationship
* some additional tests
* http request parsing test
* QueryOptions in new test_util code
* initial scaffolding of new request structures
* service and datastore
* test outline
* l2 merge conflict scrub
* service tests for scheduled query service
* service and datastore tests
* most endpoints and transports
* order of values are not deterministic with inmem
* transport tests
* rename PackQuery to ScheduledQuery
* removing existing implementation of adding queries to packs
* accounting for the new argument to NewQuery
* fix alignment in sql query
* removing underscore
* add removed to the datastore
* removed differential from the schema
- New datastore method for bulk deletion
- New service method calling this datastore method
- Endpoint, transport and handler connections for service method
Closes#389
