Commit Graph

70 Commits

Author SHA1 Message Date
Tomas Touceda
876aa0e366
Add host count API (#2356)
* Add host count API

* Add rest api docs

* Add host count for labels and address review comments

* Add regex for id
2021-10-07 08:25:35 -03:00
Tomas Touceda
2df4389aa0
Ignore empty host users or software inventory (#2317)
* Ignore empty host users or software inventory

* Only store additional if it's not nil

* Update label/policy updated at when we record the executions and skip saving host

* Update changes file
2021-10-01 18:27:57 -03:00
Tomas Touceda
d81a6317a0
Return host count when modifying a label (#2221) 2021-09-24 15:56:55 -03:00
Tomas Touceda
1f324339f8
Add jitter to intervals (#2158)
* Add max jitter percent config

* Fix jitter calc

* Remove comment

* Reduce test jitter to make tests less flaky

* Remove jitter entirely

* Document new config

* Fix doc link
2021-09-21 14:21:44 -03:00
Tomas Touceda
4650484960
Remove fk label membership (#2157)
* Remove fk from label membership

* Add changes file

* Fix tests

* No need to IGNORE anymore
2021-09-21 11:48:20 -03:00
Lucas Manuel Rodriguez
c69937945a
Introduce entityName type for mysql entity table names (#2139)
* Introduce entity type to specify mysql table names for deleteEntit* functions

* Remove changes entry for issue (non-user facing changes)
2021-09-20 14:47:06 -03:00
Martin Angers
d956ca1bec
Use the context in the Datastore layer. (#2030) 2021-09-14 10:44:02 -04:00
Martin Angers
4f4185372d
Add support for context in datastore/mysql layer (#1962)
This is just to pass down the context to the datastore layer, it doesn't
use it just yet - this will be in a follow-up PR.
2021-09-14 08:11:07 -04:00
Martin Angers
a099854045
Add missing db transactions, read from primary when after a write (#1957) 2021-09-08 14:43:22 -04:00
Tomas Touceda
69e06476a7
Ignore nonexistent label query executions (#1953) 2021-09-07 15:51:49 -03:00
Martin Angers
3755a58070
Add support for MySQL read replicas (#1786)
Adds configuration options to use a read-only MySQL replica,
and uses it instead of the primary for reads.
2021-09-01 15:50:52 -04:00
Tomas Touceda
2a17f9fe56
Remove hardcoded limits to the labels (#1857)
* Remove hardcoded limits to the labels

* Add changes file

* Remove test that doens't apply anymore
2021-08-30 14:56:15 -03:00
Tomas Touceda
a8642493ad
Add global policies (#1750)
* Add global policies

* Update documentation and add extra parameter to config

* Fix failing tests

* Store historic policy records

* Address review comments

And also remove other inmem references I saw by chance

* Add documentation for get by id request

* Add parameter doc

* Move schema generation to a cmd instead of a test

Otherwise it messes up running all tests sometimes depending on how parallel it does

* Remove brain dump for another task

* Make migration tests a separate beast

* Make schema generation idempotent and move dbutils cmd to tools

* Allow all filters and add counts to Policy

* Add test for Policy
2021-08-24 17:24:52 -03:00
Martin Angers
e61cb7e0db
Configure golangci-lint for the whole repository
Add a relatively minimal set of linters that raise safe and 
mostly un-opinionated issues with the code. It runs 
automatically on CI via a github action.
2021-08-24 13:35:03 -04:00
Martin Angers
bfdc014cc1
Fix sql-related missing Close/Err calls (#1769) 2021-08-23 16:33:41 -04:00
Tomas Touceda
512f5defce
Detect centos as a host platform more accurately (#1664)
* Detect centos as a host platform more accurately

* Add test for centos host labels
2021-08-13 13:22:09 -03:00
Benjamin Edwards
8bdad712d8
add team_id filter to fleetctl & api (#1596)
* add team_id filter to fleetctl via get hosts --team flag & api via api/v1/fleet/hosts and api/v1/fleet/labels/id/hosts

* update tests & add changes file
2021-08-11 10:40:56 -04:00
Tomas Touceda
3b67366bf4
Allow filtering by status as well as label and match query when listing hosts (#1562) 2021-08-05 14:56:29 -03:00
Zach Wasserman
e0799bbc59
Include Team names when listing label hosts (#1267)
Fixes #1266
2021-06-30 09:28:45 -07:00
Zach Wasserman
c5280c0517
Add v4 suffix in go.mod (#1224) 2021-06-25 21:46:51 -07:00
Zach Wasserman
1417d01407
Make naming of host columns consistent (#1183)
Adding consistency between API and DB helps to make it easier for users
and developers working with the API to correctly order things.

Closes #317
2021-06-23 17:32:19 -07:00
Zach Wasserman
4800856bf7
Add label ID to get labels response (#1164)
Fixes #1065
2021-06-22 16:41:25 -07:00
Zach Wasserman
b859f7b747
Sort labels for membership update (#1156)
This may help with deadlocks on the `label_membership` table. It is not
clear from MySQL documentation whether the order of the records is
significant for locking within a single query. If it is, this should
help the problem. If it is not, this should have no negative impact.

May fix #1146
2021-06-21 16:10:24 -07:00
Zach Wasserman
19e8da177f
Allow Packs to be targeted to Teams (#1130)
- Add additional target type for packs.
- Refactor pack target datastore.
- Fixes for frontend target selector tier logic on packs page.
2021-06-18 09:43:16 -07:00
Zach Wasserman
2ad557e3b3 Merge branch 'main' into teams 2021-06-18 09:42:20 -07:00
Zach Wasserman
0fd8590605
Add additional logging for label save failures (#1012)
This is intended to help with debugging #744.
2021-06-08 15:29:26 -07:00
Zach Wasserman
fb32f0cf40
Remove kolide types and packages from backend (#974)
Generally renamed `kolide` -> `fleet`
2021-06-06 15:07:29 -07:00
Zach Wasserman
e4358a92bc
Filter hosts and label counts by teams (#949)
- Add TeamFilter to relevant host and label methods.
- Pass appropriate filter in service methods.

The dashboard should now show the appropriate hosts for a user's team membership.
2021-06-03 18:53:43 -07:00
Zach Wasserman
0e9eb920d4
Refactor struct slices for consistent pointer use (#864)
Some datastore and service methods would return slices of structs,
rather than slices to pointers of structs (which most methods used).
Make this more consistent.
2021-05-25 18:53:22 -07:00
Zach Wasserman
15b81824f5
Filter query page API responses based on team membership (#850)
- Include only hosts that the user has access to in search targets API.
- Add parameter to specify whether `observer` hosts should be included.
- Generate counts based on which hosts user can access.
- Update API doc.
2021-05-24 21:34:08 -07:00
Zach Wasserman
a79ad7d312
Improvements to built-in labels (#562)
- Fix issue with built-in labels showing multiple platforms when hosts
  are reinstalled with new platform.
- Add Red Hat Linux built-in label.
- Display more labels by default in target selector.

Fixes #546, #553
2021-03-30 15:37:33 -07:00
Zach Wasserman
c6c5e65a7d
Add support for filtering hosts in label (#335)
Support the same query syntax as the hosts endpoint, here also bounded
by the membership of the label.
2021-02-18 12:52:43 -08:00
Mike Arpaia
af96e52a00
Update the Go import paths to new repo name (#27) 2020-11-11 09:59:12 -08:00
Brendan Shaklovitz
c6b285c56e
Remove soft-deletion pattern (#2327)
* Perform migration to delete any entries with `deleted` set, and
subsequently drop columns `deleted` and `deleted_at`.
* Remove `deleted` and `deleted_at` references.

Closes #2146
2020-10-22 10:51:26 -07:00
Zachary Wasserman
7494513400 Clean up and comments before merge. 2020-07-21 14:05:46 -07:00
Zachary Wasserman
96fc090723 Improve server performance for host operations
- Debounce frontend to reduce number of target searches in live query.
- More efficiently calculate label counts in live query and hosts
  dashboard. Instead of using the (slow) CountHostsInTargets function,
  retrieve the host counts while looking up the labels.
- Optimize targets search query. Removing the nested query retrieves the
  same logical result set, but substantially optimizes MySQL CPU usage.
  Testing indicates about a 50% reduction in MySQL CPU usage for the
  frontend targets search API call after applying this change.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
42bea2a144 Implement manual labels
"Manual" labels can be specified by hostname, allowing users to specify
the membership of a label without having to use a dynamic query. See the
included documentation.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
608772917c Refactor label membership storage
Label membership is now stored in the label_membership table. This is
done in preparation for adding "manual" labels, as previously label
membership was associated directly with label query executions.

Label queries are now all executed at the same time, rather than on
separate intervals. This simplifies the calculation of which distributed
queries need to be run when a host checks in.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
b0595748f1 Implement pagination of hosts in the web UI
This commit takes advantage of the existing pagination APIs in the Fleet
server, and provides additional APIs to support pagination in the web
UI. Doing this dramatically reduces the response sizes for requests from
the UI, and limits the performance impact of UI clients on the Fleet and
MySQL servers.
2020-07-21 14:05:46 -07:00
billcobbler
44c40e4255
Retry sqlx transactions (#2203)
- Add exponential backoff method to MySQL datastore
- Refactor queries using transaction to use backup.

Fixes #845
2020-03-10 18:14:02 -07:00
Zachary Wasserman
6dbc3bc77a
Implement fleetctl goquery (#2186)
- Update fleet APIs to support necessary operations in goquery
- Implement support for goquery in fleetctl
2020-01-23 21:27:20 -08:00
Zachary Wasserman
c269886389
Fix target search behavior with - and + symbols (#2067)
This PR makes the target search more user-friendly by stripping symbols that
have a special interpretation in MySQL FTS.

Closes #2017
2019-07-03 10:47:43 -07:00
Zachary Wasserman
6424a029ee
Fix labels UI issues in Fleet 2.0 (#1830)
Replaces the UI endpoints for creating and modifying labels. These were removed
in #1686 because we thought we were killing the UI.

Now labels can be created and edited in the UI again.
2018-06-18 10:09:08 -07:00
Zachary Wasserman
b80e0a102d
Add fleetctl query command (#1784)
Allow queries targeted by hostname and label name.
2018-05-17 15:54:34 -07:00
Mike Arpaia
b42e4bacf2
Add name not empty validation for spec types (#1783) 2018-05-14 12:23:38 -06:00
Zachary Wasserman
87331b47e2
Return 404 errors where appropriate in spec endpoints (#1777) 2018-05-09 16:54:42 -07:00
Zachary Wasserman
d146412364 Implement Get methods for single entities of label, pack and query specs (#1766)
- Datastore methods (where necessary)
- Service methods
- Endpoints/Handlers
2018-05-07 19:54:29 -06:00
Zachary Wasserman
003ad9b5fa
Delete spec entities by name instead of ID (#1755)
With the UI, deleting by ID made sense. With fleetctl, we now want to delete
by name. Transition only the methods used for spec related entities, as others
will be removed soon.
2018-05-04 11:05:55 -07:00
Zachary Wasserman
bde7256a09
Update labels interfaces for fleetctl (#1686)
- Remove unnecessary labels code
- Remove unnecessary packs code
- Update tests as appropriate
2018-01-10 11:38:20 -08:00
Mike Arpaia
018b91ab2c Rename project to Kolide Fleet (#1529) 2017-06-22 15:50:45 -04:00