Commit Graph

467 Commits

Author SHA1 Message Date
keepwn
b42d213689
Support close the websocket of LiveQueryResultsHandler (#1006)
* Support close `LiveQueryResultsHandler`

* Start adding test

* Make LiveQuery exit when the context is Done

* Fix lint and remove debug print

* Update server/service/client_live_query.go

Co-authored-by: Zach Wasserman <zach@fleetdm.com>

* Revert "Update server/service/client_live_query.go"

This reverts commit be67ca1512fe502503e821393c2b9e84f5e6e82e.

Co-authored-by: Tomas Touceda <chiiph@gmail.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-09-10 16:26:39 -03:00
Tomas Touceda
13e5e6a6db
Debug log per host (#1978)
* Add extra debug logging for hosts

* Add changes file

* Ignore if appconfig is nil

* Use slice of uints instead of a string

* Debug response request for enabled hosts

* Add host-id to request/response

* Lint fixes

* Add missing AppConfigFuncs
2021-09-10 14:48:33 -03:00
Martin Angers
a099854045
Add missing db transactions, read from primary when after a write (#1957) 2021-09-08 14:43:22 -04:00
gillespi314
4a307f1274
Update email template image links (#1938)
* Update email image links; move pngs to fleetdm.com
2021-09-03 17:00:30 -05:00
Tomas Touceda
6d2ae02efd
Rename core->free and basic->premium (#1870)
* Rename core->free and basic->premium

* Fix lint js

* Comment out portion of test that seems to timeout

* Rename tier  to premium if basic is still loaded
2021-09-03 13:05:23 -03:00
Tomas Touceda
15ce06263d
Dont fail config if agent opts is nil (#1898)
* Handle agentopts nil better

* Add changes file
2021-09-01 19:17:41 -03:00
Martin Angers
9a0871a2f1
Address issues related to Redis Cluster support (#1885)
Closes #1847 .
2021-09-01 16:32:57 -04:00
Benjamin Edwards
047dc5ac62
fix get hosts command to properly output json/yaml (#1774)
* fix get hosts command to properly output json/yaml based on command line flag

* add changes file

* added tests for get hosts when specifiying host

* added additional hosts to be returned in test cases

* go fmt
2021-08-31 11:37:03 -04:00
Tomas Touceda
8b908f6506
Issue 1599 offline webhook (#1777)
* wip

* Add tests and finish implementation

* Add proper default for periodicity, changes file, and documentation

* Fix tests and add defaults also to new installs

* EnableHostUsers should be true if undefined as well

* In some cases, periodicity can be zero because of the migrations

* Apply defaults when migrating appconfig

* Fix lint

* lint

* Address review comments
2021-08-27 11:15:36 -03:00
Tomas Touceda
d6d53acda5
Issue 1600 fleetctl license expiration (#1800)
* Show banner in fleet and fleetctl if license expired

* Ignore if license is nil or tier is core

* Address review comments
2021-08-26 10:28:53 -03:00
Zach Wasserman
5993018214
Fix live query timeout handling (#1759)
Correct cleanup of the connections was verified with the pprof goroutine
tool.

Fixes #1751
2021-08-25 15:28:04 -07:00
Tomas Touceda
1f5094d97e
Make adding routes a bit simpler (#1771)
* Make adding routes a bit simpler

* Remove unused handle

* Lint

* More lint
2021-08-25 10:08:14 -03:00
Tomas Touceda
5fb5995b83
Remove unneeded interfaces (#1779)
* Remove unneeded interfaces

* Remove unused code
2021-08-24 18:49:56 -03:00
Tomas Touceda
a8642493ad
Add global policies (#1750)
* Add global policies

* Update documentation and add extra parameter to config

* Fix failing tests

* Store historic policy records

* Address review comments

And also remove other inmem references I saw by chance

* Add documentation for get by id request

* Add parameter doc

* Move schema generation to a cmd instead of a test

Otherwise it messes up running all tests sometimes depending on how parallel it does

* Remove brain dump for another task

* Make migration tests a separate beast

* Make schema generation idempotent and move dbutils cmd to tools

* Allow all filters and add counts to Policy

* Add test for Policy
2021-08-24 17:24:52 -03:00
Martin Angers
e61cb7e0db
Configure golangci-lint for the whole repository
Add a relatively minimal set of linters that raise safe and 
mostly un-opinionated issues with the code. It runs 
automatically on CI via a github action.
2021-08-24 13:35:03 -04:00
Martin Angers
a3714d2ed9
Add fleetctl debug connection command (#1706)
Adds the `fleetctl debug connection` command to investigate
connection issues to the fleet server.

Closes #1579 .
2021-08-24 08:50:03 -04:00
Tomas Touceda
612fa05dd3
Log errors when osquery endpoints have issues (#1764) 2021-08-23 19:40:00 -03:00
Tomas Touceda
66d864cc84
Remove extra logIPs from host auth mw (#1762)
* Remove extra logIPs from host auth mw

* Fix test
2021-08-23 14:23:27 -03:00
Tomas Touceda
ad18728ac3
Issue 1601 disk space (#1735)
* wip

* Add tests

* Add migration

* Fix tests and set platform for windows correctly
2021-08-20 14:57:37 -03:00
Tomas Touceda
c6c63ab12a
Refactor app config (POC, for now) (#1685) 2021-08-20 12:27:41 -03:00
Tomas Touceda
402303bc5d
Add All Linux label (#1582)
* Add All Linux label

* Change name to Linux instead of All Linux to see if e2e likes it better

* Revert "Change name to Linux instead of All Linux to see if e2e likes it better"

This reverts commit 26b79f214e3b744e73270c544f89bb698575f6ea.

* Fix all linux label insert
2021-08-18 15:55:48 -03:00
Tomas Touceda
5b8a607a81
Add support for MariaDB 10.5.4 and later (#1658)
* Add support for MariaDB 10.5.4 and later

* Fix tests

* Add changes file

* Use require.JSONEq instead of our own version of it
2021-08-13 11:16:19 -03:00
Tomas Touceda
96c0244c04
Fix redis locking receive (#1655)
* Receive redis data with timeout to not hold the connection

* Address review comments
2021-08-13 11:16:11 -03:00
Tomas Touceda
e8def4af00
Allow to refetch hosts if you are an observer/maintainer for the team for that host (#1649) 2021-08-11 19:01:37 -03:00
Tomas Touceda
3d8a766ca1
Make receive calls to redis conn thread safe (#1641)
* Make receive calls to redis conn thread safe

Also removes REDIS_TEST env var. Redis is lightweight and fast, no need
to skip these tests.

* No need to increase the wait
2021-08-11 17:34:35 -03:00
Tomas Touceda
ae2bd137b1
Issue 1632 software inventory config (#1636)
* Add config option for software inventory

* Add documentation for the new config
2021-08-11 15:57:53 -03:00
Tomas Touceda
11dff7ca92
Issue 1588 allow disabling host users (#1611)
* Allow users to disable host_users

* Add missing files

* Address review comments

* Fix tests

* Check additional queries for nil, not the whole hostsettings

* Finally fix tests
2021-08-11 14:56:11 -03:00
Zach Wasserman
740ba7bc25
Remove duplicate registration of results handler (#1605) 2021-08-11 09:39:47 -07:00
Benjamin Edwards
8bdad712d8
add team_id filter to fleetctl & api (#1596)
* add team_id filter to fleetctl via get hosts --team flag & api via api/v1/fleet/hosts and api/v1/fleet/labels/id/hosts

* update tests & add changes file
2021-08-11 10:40:56 -04:00
Benjamin Edwards
29611d8377
modify packs api to filter non-empty pack_type from response (#1558)
* modify packs api to filter non-empty pack_type from response
* change list packs store method to allow filtering for "system-level" packs, by default the api filters these packs from being returned
* add changes file
* don't allow modifications to global or team packs via apply spec
* refactor to use PackListOptions
2021-08-10 09:43:27 -04:00
Zach Wasserman
a29844120b
Fix live query for observers (#1603)
Set observer can run when performing authz check for live query results.

Final backend fix for #1515
2021-08-09 19:30:17 -07:00
Tomas Touceda
5b5dca32e9
Add more checks to observers running queries (#1589)
* Add more checks to observers running queries

* Fix test

* Use proper authorize policy instead of doing it by hand
2021-08-09 14:38:06 -03:00
Tomas Touceda
93a10e6f94
Expose vulnerabilities data in host software data (#1555)
* Expose vulnerabilities data in host software data

* Gather cves and software in one query

* Expand the test to cover all cases

* Make test less flaky
2021-08-06 14:04:37 -03:00
Tomas Touceda
f8b7a83cc6
Process stored CPEs and store found CVEs (#1533)
* WIP

* WIP

* Make path optional and fix tests

* Add first generate

* Move to nvd package

* remove replace

* Re-add replace

* It's path, not file name

* Change how db path is set and use etag

* Fix typos

* Make db generation faster

* Remove quotes

* Doesn't like comments

* Samitize etag and save to file

* Refactor some things and improve writing of etagenv

* Compress file and truncate amount of items for faster testing

* Remove quotes

* Try to improve performance

* Ignore truncate error if not exists

* Minor cleanup and make sqlite have cpe prefix

* Simplify code and test sync

* Add VCR for sync test

* Check for nvdRelease nil

* Add test for the actual translation

* Address review comments

* Rename generate command because we'll have a cve one too

* Move to its own dir

* Add first cve db generation

* WIP but with final strategy, preparring to merge main

* Fix merge conflicts

* WIP

* wip

* Insert CVEs to the db

* Remove unused code

* Use wg instead of counting

* Call cancelFunc to avoid ctx leak

* Fix logs for better readability

* Point code to fleetdm instead of my repo
2021-08-04 18:01:39 -03:00
Tomas Touceda
8cdf61f8df
Remove inmem store (#1534)
* Remove inmem store

* Use full package name for db to avoid conflicts

* Fix lint

* Remove unneeded check/comment
2021-08-04 10:40:04 -03:00
Tomas Touceda
52ae04ecf5
Only log distributed queries ingestion errors (except for one case) (#1535)
* Don't return errors in distributed query ingestion, just log them

* Allow for multiple errors in the logging context

* Update check when loading host

* Log multiple errors and add tests for other changes

* Add missing host func

* Add another missing host func

* Add changes file
2021-08-04 10:11:51 -03:00
Tomas Touceda
f2837fd4b3
Make decoder completely generic and simplify things (#1542)
* Make decoder completely generic and simplify things

* Add commends and unexport func
2021-08-03 16:56:54 -03:00
Tomas Touceda
7730515be7
Implement team schedules (#1528)
* Implement team schedules

* Improve description
2021-08-03 10:33:27 -03:00
Tomas Touceda
5859db36bb
Move logger up to the HTTP layer and make it generic (#1439)
* Add basic idea

* Implement the new logging strategy everywhere

* Remove unused const

* Add tests and fix error cases

* Fix logging in osquery service

* If there are extras, log info unless force debug

* Change to info

* Fix test

* Make logging context more chainable and force info for sessions
2021-08-02 19:06:27 -03:00
Benjamin Edwards
c18214be37
add logging settings to config api response (#1467)
- add docker-compose file for locally testing aws dependencies
- update firehose & kinesis configs to optionally supply endpoint url override
- serialize `logging` field in appconfig api response
2021-07-30 11:45:49 -04:00
Tomas Touceda
46b0b7765b
Issue 1435 software to cpe (#1488)
* WIP

* WIP

* Make path optional and fix tests

* Add first generate

* Move to nvd package

* remove replace

* Re-add replace

* It's path, not file name

* Change how db path is set and use etag

* Fix typos

* Make db generation faster

* Remove quotes

* Doesn't like comments

* Samitize etag and save to file

* Refactor some things and improve writing of etagenv

* Compress file and truncate amount of items for faster testing

* Remove quotes

* Try to improve performance

* Ignore truncate error if not exists

* Minor cleanup and make sqlite have cpe prefix

* Simplify code and test sync

* Add VCR for sync test

* Check for nvdRelease nil

* Add test for the actual translation

* Address review comments

* Rename generate command because we'll have a cve one too

* Move to its own dir

* Address review comments
2021-07-29 13:10:34 -03:00
Tomas Touceda
866e8a2961
Fix typo, it's target_count (#1478)
* Fix typo, it's target_count

* Update server/service/service_campaigns.go

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
2021-07-26 16:12:18 -03:00
Tomas Touceda
4cd169cad7
Return gravatar whenever available (#1448) 2021-07-22 12:28:42 -03:00
Tomas Touceda
484c6153e3
Issue 1359 fleetctl team transfer (#1413)
* wip

* Add delete user command and translator

* Add host transfer command

* Add changes file

* Undo bad refactor

* Fix copypaste error

* Implement with interfaces instead of assertions

* Ad documentation and simplify implementation further

* Update docs/1-Using-Fleet/3-REST-API.md

Co-authored-by: Zach Wasserman <zach@fleetdm.com>

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-07-21 14:03:10 -03:00
Tomas Touceda
9863b0f4bb
Issue 1361 fleetctl teams (#1405)
* WIP

* Add get user_roles and apply for a user_roles spec to fleetctl

* Uncomment other tests

* Update test to check output

* Update test with the new struct

* Mock token so that it doesn't pick up the one in the local machine

* Address review comments

* Fix printJSON and printYaml

* Fix merge conflict error

* WIP

* wip

* wip

* Finish implementation

* Address review comments

* Fix flaky test
2021-07-19 16:48:49 -03:00
Tomas Touceda
29570bd860
Issue 1278 select leader (#1367)
* Add leader selection

* remove comment

* Address review comments

* Add changes file

* Simplify implementation

* Simplify further

* Whoops, removed a little too much
2021-07-19 15:08:41 -03:00
Tomas Touceda
545b3f396e
Issue 1362 fleetctl user roles (#1397)
* WIP

* Add get user_roles and apply for a user_roles spec to fleetctl

* Uncomment other tests

* Update test to check output

* Update test with the new struct

* Mock token so that it doesn't pick up the one in the local machine

* Address review comments

* Fix printJSON and printYaml

* Fix merge conflict error

* If both roles are specified, fail

* Fix test

* Switch arguments around

* Update test with the new rule

* Fix other tests that fell through the cracks
2021-07-16 15:28:13 -03:00
Tomas Touceda
2d553db2aa
Issue 1231 add global schedule (#1383)
* Add Global Schedule

* Uncomment tests

* Uncomment integration tests

* Add targets for global schedule

* Add host label

* Simplify implementation

* wip

* Fix mock

* Add missing test

* Further simplify the implementation

* Forgot to remove test name

* Update names
2021-07-16 10:15:15 -03:00
Tomas Touceda
804136127e
Remove repository sources from software inventory (#1387) 2021-07-15 10:23:27 -03:00
Tomas Touceda
82ab0a798e
Add host users (#1334)
* Add host users

* Add changes file and test removing pull_request from the on test

* Remove users and store the removal timestamp

* Improve test yml to allow for PRs from forks
2021-07-13 17:15:38 -03:00