Commit Graph

372 Commits

Author SHA1 Message Date
Tomas Touceda
685245c2bd
Cache app config in redis (#2205)
* Cache app config in redis

* Add changes files

* Replace string with constant

* Revert some test refactorign and duplicate a bit of test code

* Add test for AppConfig with redis failing

* Fix lint

* Use Doer so it works better in clusters

* Skip unmarshalling if we already did

* Allow to cache hosts if configured

* Omit the setting if empty

* Remove hashing, too much CPU

* Revert caching of host auth... needs a more thought through approach

* Remove config

* Remove old config

* Remove locker interface

* Fix test and address review comments
2021-09-28 10:01:53 -03:00
Tomas Touceda
2033d8208c
Add policy updated at (#2246)
* wip

* Add policy updated at interval and update the UI to use that

* Update rest api

* Fix tests
2021-09-27 16:27:38 -03:00
Tomas Touceda
d81a6317a0
Return host count when modifying a label (#2221) 2021-09-24 15:56:55 -03:00
Tomas Touceda
6497e0ba2e
Improve performance of cascade host software migration (#2163) 2021-09-21 16:37:13 -03:00
Tomas Touceda
1f324339f8
Add jitter to intervals (#2158)
* Add max jitter percent config

* Fix jitter calc

* Remove comment

* Reduce test jitter to make tests less flaky

* Remove jitter entirely

* Document new config

* Fix doc link
2021-09-21 14:21:44 -03:00
Tomas Touceda
4650484960
Remove fk label membership (#2157)
* Remove fk from label membership

* Add changes file

* Fix tests

* No need to IGNORE anymore
2021-09-21 11:48:20 -03:00
Martin Angers
86dce785ae
Improve performance of the Go test suite (#2060)
Closes #1805
2021-09-20 14:09:38 -04:00
Lucas Manuel Rodriguez
c69937945a
Introduce entityName type for mysql entity table names (#2139)
* Introduce entity type to specify mysql table names for deleteEntit* functions

* Remove changes entry for issue (non-user facing changes)
2021-09-20 14:47:06 -03:00
Tomas Touceda
b32b441c12
Issue 1512 filter observer can run queries (#2110)
* wip

* Filter queries for observers

* Update e2e test now that we filter queries
2021-09-20 13:07:51 -03:00
Tomas Touceda
baa42d367e
Add team policies (#2103)
* Add team policies

* Add team policy documentation

* Add changes file

* Update titles

* Fix lint

* Rewrite TeamAuthorize for more clarify

* Explicitly use two slices for clarity

* Simplify switch
2021-09-20 11:00:57 -03:00
Tomas Touceda
bdae8d04a2
Skip saving host users and inventory if disabled (#2127) 2021-09-18 11:33:36 -03:00
Tomas Touceda
e6368cc57f
Refactor integration tests (#1821)
* Refactor integration tests

* Remove nopCloser and use io.NopCloser

* Address review comments
2021-09-15 16:27:53 -03:00
Martin Angers
1fa5ce16b8
Add configurable Redis connection retries and following of cluster redirections (#2045)
Closes #1969
2021-09-15 08:50:32 -04:00
Martin Angers
d956ca1bec
Use the context in the Datastore layer. (#2030) 2021-09-14 10:44:02 -04:00
Tomas Touceda
176037cad2
Implement fleetctl get software and the underlying API (#1999)
* Implement fleetctl get software and the underlying API

* Add documentation

* Simplify list software implementation

* Lint fixes

* Make team name unique

* Address review comments

* Fix lint

* Fix tests
2021-09-14 10:58:48 -03:00
Martin Angers
4f4185372d
Add support for context in datastore/mysql layer (#1962)
This is just to pass down the context to the datastore layer, it doesn't
use it just yet - this will be in a follow-up PR.
2021-09-14 08:11:07 -04:00
Tomas Touceda
db83c40d45
Make app config json migration not be skipped by patch release (#2034)
* Make app config json migration not be skipped by patch release

* Update test schema
2021-09-13 14:41:50 -03:00
Benjamin Edwards
42014c02e3
Fix migration that isn't compatible with some mysql8 settings (#1992)
Fixes #1991

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-09-13 10:20:31 -07:00
Martin Angers
d267fb5168
Fix reading policy after creation for read replicas (#1983) 2021-09-09 16:23:35 -04:00
Tomas Touceda
b2efc9f51c
Make redis conn timeout and keep alive configurable (#1968)
* Make redis conn timeout and keep alive configurable

* Document new configs

* Correct config name
2021-09-08 17:55:12 -03:00
Martin Angers
a099854045
Add missing db transactions, read from primary when after a write (#1957) 2021-09-08 14:43:22 -04:00
Tomas Touceda
69e06476a7
Ignore nonexistent label query executions (#1953) 2021-09-07 15:51:49 -03:00
Martin Angers
0ac4fc4c1a
Update redisc and replace EachRedisNode implementation (#1949) 2021-09-07 13:25:36 -04:00
Tomas Touceda
7c34956d31
Add coverage to uncovered mysql code (#1855)
* Add coverage to uncovered mysql code

* Add deleted method and update mock

* Fix test
2021-09-07 13:48:04 -03:00
Tomas Touceda
a6acb1cd1a
Allow users to be readded if they were ever removed (#1945)
* Allow users to be readded if they were ever removed

* Address review comment

* lint
2021-09-07 13:33:40 -03:00
Tomas Touceda
99e9e846f4
Remove id column from host_users to allow for bulk insert (#1931)
* Remove id column from host_users to allow for bulk insert

* Update documentation
2021-09-07 11:02:35 -03:00
Tomas Touceda
d5a5f014ef
Add test to check that two hosts can store stats concurrently (#1929) 2021-09-07 10:39:17 -03:00
Tomas Touceda
db2e04a7c3
Update new sched query stats if there's already some (#1918)
* Update new sched query stats if there's already some

* IGNORE if the sched query is not present

* Make tests less flaky
2021-09-02 17:39:08 -03:00
Tomas Touceda
681e7f18a0
Add default for agent options (#1917) 2021-09-02 15:59:48 -03:00
Martin Angers
9a0871a2f1
Address issues related to Redis Cluster support (#1885)
Closes #1847 .
2021-09-01 16:32:57 -04:00
Martin Angers
3755a58070
Add support for MySQL read replicas (#1786)
Adds configuration options to use a read-only MySQL replica,
and uses it instead of the primary for reads.
2021-09-01 15:50:52 -04:00
Tomas Touceda
79b5330a43
Do host user inserts one by one to not lock the whole database (#1884) 2021-09-01 11:39:23 -03:00
Tomas Touceda
22267d8e0a
Group concat limit (#1860)
* wiup

* Decouple software select in two

* Add changes file

* Add missing error check
2021-08-30 16:07:24 -03:00
Tomas Touceda
2a17f9fe56
Remove hardcoded limits to the labels (#1857)
* Remove hardcoded limits to the labels

* Add changes file

* Remove test that doens't apply anymore
2021-08-30 14:56:15 -03:00
Tomas Touceda
8b908f6506
Issue 1599 offline webhook (#1777)
* wip

* Add tests and finish implementation

* Add proper default for periodicity, changes file, and documentation

* Fix tests and add defaults also to new installs

* EnableHostUsers should be true if undefined as well

* In some cases, periodicity can be zero because of the migrations

* Apply defaults when migrating appconfig

* Fix lint

* lint

* Address review comments
2021-08-27 11:15:36 -03:00
Tomas Touceda
2fd725c81b
Export fields so the select will work with the struct (#1819) 2021-08-26 11:56:05 -03:00
Tomas Touceda
5fb5995b83
Remove unneeded interfaces (#1779)
* Remove unneeded interfaces

* Remove unused code
2021-08-24 18:49:56 -03:00
Tomas Touceda
a8642493ad
Add global policies (#1750)
* Add global policies

* Update documentation and add extra parameter to config

* Fix failing tests

* Store historic policy records

* Address review comments

And also remove other inmem references I saw by chance

* Add documentation for get by id request

* Add parameter doc

* Move schema generation to a cmd instead of a test

Otherwise it messes up running all tests sometimes depending on how parallel it does

* Remove brain dump for another task

* Make migration tests a separate beast

* Make schema generation idempotent and move dbutils cmd to tools

* Allow all filters and add counts to Policy

* Add test for Policy
2021-08-24 17:24:52 -03:00
Martin Angers
e61cb7e0db
Configure golangci-lint for the whole repository
Add a relatively minimal set of linters that raise safe and 
mostly un-opinionated issues with the code. It runs 
automatically on CI via a github action.
2021-08-24 13:35:03 -04:00
Martin Angers
bfdc014cc1
Fix sql-related missing Close/Err calls (#1769) 2021-08-23 16:33:41 -04:00
Tomas Touceda
a1ceebf267
Make team schedule names more user friendly and hide them from host pack stats (#1737)
* Make team schedule names more user friendly and hide them from host pack stats

* Delete test for a different bug and add migration

* Update name template
2021-08-23 14:23:47 -03:00
Tomas Touceda
26a08096fb
Use GROUP_CONCAT instead of JSON_ARRAYAGG (#1752)
* Use GROUP_CONCAT instead of JSON_ARRAYAGG

* Update server/datastore/mysql/software.go

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2021-08-23 14:12:30 -03:00
Tomas Touceda
b7a65e920a
Remove orphan host_software and software before adding fk constriants (#1760) 2021-08-23 13:42:43 -03:00
Tomas Touceda
ad18728ac3
Issue 1601 disk space (#1735)
* wip

* Add tests

* Add migration

* Fix tests and set platform for windows correctly
2021-08-20 14:57:37 -03:00
Tomas Touceda
605970c441
Add cascade deletes for host software (#1739)
* Add cascade deletes for host software

* Add changes file

* The drop doesn't work on certain mysql

* Fix error message
2021-08-20 14:29:00 -03:00
Tomas Touceda
c6c63ab12a
Refactor app config (POC, for now) (#1685) 2021-08-20 12:27:41 -03:00
Tomas Touceda
33791dbee8
Remove fk in scheduled_query_stats table and clean up orphan rows for it (#1720)
* Remove fk in scheduled_query_stats table and clean up orphan rows for it

* Improve test and fix bug with the cleanup
2021-08-18 18:30:48 -03:00
Tomas Touceda
402303bc5d
Add All Linux label (#1582)
* Add All Linux label

* Change name to Linux instead of All Linux to see if e2e likes it better

* Revert "Change name to Linux instead of All Linux to see if e2e likes it better"

This reverts commit 26b79f214e3b744e73270c544f89bb698575f6ea.

* Fix all linux label insert
2021-08-18 15:55:48 -03:00
Tomas Touceda
53e25cf59d
Use tx if within a tx (#1668) 2021-08-13 14:59:31 -03:00
Tomas Touceda
512f5defce
Detect centos as a host platform more accurately (#1664)
* Detect centos as a host platform more accurately

* Add test for centos host labels
2021-08-13 13:22:09 -03:00