Martin Angers
0cc57bd294
Mna 1782 migrate endpoints ( #3226 )
2021-12-14 16:34:11 -05:00
Martin Angers
4143a37056
Fix redis scan keys issue for live queries ( #3107 )
2021-12-14 16:30:26 -05:00
Zach Wasserman
33797ddfc1
Optimize requests on packs page ( #3327 )
...
Improves #3259
2021-12-13 21:50:24 -08:00
Lucas Manuel Rodriguez
25fd04ea18
Fix team packs rego policy rules ( #3356 )
2021-12-13 20:53:29 -08:00
Zach Wasserman
ac3d8ddf02
Increase default login session length to 24 hours ( #3340 )
2021-12-13 20:42:30 -08:00
Zach Wasserman
3a5037f798
Update query for users listing ( #3324 )
...
- Filter undesired entries.
- LEFT JOIN to include users missing groups.
#2488
2021-12-11 16:25:08 -08:00
Lucas Manuel Rodriguez
4213ddb141
Rename missing policy platforms to platform ( #3334 )
2021-12-10 13:55:49 -03:00
Lucas Manuel Rodriguez
8bbc52bbff
Fix policy rego bug with fleet policies ( #3331 )
...
* Fix policy rego bug with fleet policies
* Simplify rego policy fix
2021-12-10 11:47:41 -03:00
Tomas Touceda
66f4fc46c0
Only join big tables when absolutely needed ( #3308 )
...
* Only join big tables when absolutely needed
* Fix test
* Revert unintended change in test
2021-12-09 17:36:12 -03:00
Zach Wasserman
32571c0990
Add wrap for SMTP connection error ( #3206 )
2021-12-09 09:58:33 -08:00
Lucas Manuel Rodriguez
d0765cb9ee
Do not exit on serve/prepare if there are unknown migrations ( #3262 )
...
* Do not exit on serve/prepare if there are unknown migrations
* PR review changes
2021-12-08 19:50:00 -03:00
Tomas Touceda
8686c1520b
Add index software ( #3241 )
...
* Add index to software over name/id
* Update schema
* Remove double semi colon
2021-12-07 16:36:13 -03:00
Lucas Manuel Rodriguez
8cea7f480a
Add HostLinuxOSs ( #3217 )
...
* Add HostLinuxOSs
* Add test for PlatformFromHost
* Add sles, gentoo and kali on linux host OSs
2021-12-07 15:37:00 -03:00
Lucas Manuel Rodriguez
edec1abc86
Remove platform and team from policy editing ( #3233 )
2021-12-07 13:01:00 -03:00
Lucas Manuel Rodriguez
a47c605bdd
Log request errors in non-debug mode (take 2) ( #3215 )
...
* Log request errors in non-debug mode
* Amend test for debug
* Add check for error log
2021-12-07 12:52:43 -03:00
Lucas Manuel Rodriguez
6b8a2cb61a
Fix mysql8 compatibility with zero dates ( #3231 )
...
* Fix mysql8 compatibility with zero dates
* Amend date
2021-12-07 12:51:53 -03:00
Tomas Touceda
dc9686024b
Remove tx from SaveHost ( #3232 )
...
* Remove tx from SaveHost
Also change REPLACE for software for an insert.
* Remove bundle identifier from the get since it's not part of the key
* Revert unneeded change
2021-12-07 12:39:07 -03:00
gillespi314
6ee6dba4a4
Update anonymous usage statistics to record Fleet feature adoption ( #3056 )
...
* Collect additional anonymous usage statistics
* Update fleetdm.com and docs for usage stats
* Fix lint-go errors
* Fix failing test
* update webhook inputs
* Update receive-usage-analytics.js
* fix failing test
* Update statistics_test.go
Co-authored-by: eashaw <eashaw@sailsjs.com>
2021-12-06 14:39:00 -06:00
Lucas Manuel Rodriguez
c5375263b6
Change policy platforms name to platform singular (for consistency) ( #3221 )
...
* Change policy platforms name to platform singular
* Fix fleetctl tests
2021-12-06 13:56:28 -03:00
Martin Angers
b84ef79bf4
Add the fleetctl debug {errors,db-locks} sub-commands ( #3168 )
2021-12-06 09:26:01 -05:00
Lucas Manuel Rodriguez
b9a408704e
Add platforms field to policies ( #3181 )
...
* Add platforms field to policies
* Fix fleetctl tests
* PR review changes
* Add missing tests
* Add changes for ListPoliciesForHost
2021-12-03 15:33:33 -03:00
Tomas Touceda
142006cbdd
Issue 2099 enable vulnerability processing new installs ( #3104 )
...
* Enable vulnerability processing by default
* Add changes file
* Remove wrong env var
* Fix import
* Fix tests
* Update log line
* Remove unneeded import
2021-12-03 14:44:57 -03:00
Tomas Touceda
17bd4d5110
Remove policy membership history and only store the latest value ( #3178 )
...
* Remove policy membership history and only store the latest value
* Address review comments
* Correct error msg wrap
2021-12-03 13:10:11 -03:00
Tomas Touceda
5cff07f332
Issue 3184 drop event host expiry ( #3185 )
...
* Drop event host_expiry if it exists
* Add changes file
2021-12-03 12:34:58 -03:00
Tomas Touceda
a0404b6736
Add software count API ( #3105 )
...
* Add software count API
* Fix makefile
* Fine no mock generating at this point
* Actually, one last try
* Use go install instead
* Fix go sum/mod
* Improve documentation
* Try setting node to 14
2021-12-03 10:54:17 -03:00
Martin Angers
7464e72ba8
Move carves endpoints to new endpoint pattern ( #3148 )
2021-12-01 15:45:29 -05:00
Tomas Touceda
3a031e946d
Sort hostIDs before inserting seen time ( #3162 )
...
* Sort hostIDs before inserting seen time
* Fix status test
* Add comment
2021-12-01 16:20:54 -03:00
Lucas Manuel Rodriguez
e64a88d8b1
Add COALESCEs on all host_seen_times JOINs ( #3147 )
...
* Add COALESCEs on all host_seen_times JOINs
* Use tx instead of d.writer
* Add unit tests
* Fix compile test
2021-12-01 09:05:23 -03:00
Tomas Touceda
4ef9cfdf63
Increase grace period of status for hosts ( #3151 )
...
* Increase grace period of status for hosts
* Update tests
* Update more tests
2021-11-30 18:37:27 -03:00
Tomas Touceda
925d4df8f7
Add new disable_failing_policies query parameter to list hosts ( #3123 )
...
* Update documentation
* Update based on review comments
* Explain why the switch exists
2021-11-29 18:04:33 -03:00
Tomas Touceda
c82a8e8428
Sanitize OrderKey ( #3128 )
2021-11-29 18:03:19 -03:00
Tomas Touceda
036093874d
Add cursor pagination to list hosts ( #3120 )
...
* Add cursor pagination to list hosts
* Update docs
* Update based on review comments
2021-11-29 15:06:00 -03:00
Tomas Touceda
a36e029d95
App config caching per instance ( #3071 )
...
* Do caching of app config per instance instead of across all of them in redis
* Add changes file
* Simplify code based on review comment
* Use go-cache instead of creating our own
* Dont export consts
* Copy app config before returning it
* Fix lint
* Update go sum
* Update go sum
2021-11-29 12:51:57 -03:00
Martin Angers
5c113bd468
Refactor ListActivities to new endpoint pattern ( #3115 )
2021-11-29 08:12:22 -05:00
Lucas Manuel Rodriguez
c6fc91f2e4
Fix ListPoliciesForHost, missing filter by teams ( #3106 )
...
* Fix ListPoliciesForHost, missing filter by teams
* Amend existing tests
2021-11-24 18:17:44 -03:00
Martin Angers
c997f853e5
Make creation of http.Client uniform across the codebase ( #3097 )
2021-11-24 15:56:54 -05:00
Lucas Manuel Rodriguez
964f85b174
Amend policy creation and spec (for proprietary query), and add update APIs ( #2890 )
...
* Amend policy creation (proprietary query), add update APIs
* Fix Datastore.SavePolicy bug (and add tests)
* Add integration tests for new policy APIs
* Add author email
* Add activities
* Push breaking changes for return policy fields
* WIP
* Add integration test for host policies
* Make more improvements to policy representation
* Improve upgrade code (from PR review comments)
* PR changes
* Revert activities for policies
* Use *uint instead of uint for queryID, use fleet.PolicyPayload
* Filter out other schemas
* New policy flow (#2922 )
* created new policy flow -- no API connection
* added api props
* fixed prop name
* lint fixes
* removed unused modal; fixed style
* name, desc icons; created global components
* lint fixes
* ignoring certain files and lines for prettier
* Update frontend/pages/policies/PolicyPage/PolicyPage.tsx
* Make policy names unique across deployment
* Amend upgrade script
* Fix migration for unique names
* Do not deduplicate but instead rename policies
Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>
2021-11-24 14:16:42 -03:00
Tomas Touceda
c65f8507fe
Remove host counts from software ( #3082 )
...
* Remove host counts from software
* Actually remove the host count from the struct
* Fix get test
2021-11-23 15:50:51 -03:00
Martin Angers
67293aca95
Add tests to SSO/SAML implementation ( #2997 )
2021-11-23 08:25:43 -05:00
Tomas Touceda
4cd9dab577
Update shell in host users when they exist ( #3072 )
2021-11-23 10:23:12 -03:00
Lucas Manuel Rodriguez
028c2643f7
Fleet serve to exit if migrations are missing (default) ( #2803 )
...
* Add option to fleet to exit if migrations are missing
* Reverse serve exit migrations logic
* Fix typo
2021-11-22 14:47:24 -03:00
Lucas Manuel Rodriguez
0642bfdb1d
Add fleetctl debug migrations command to get missing migrations ( #2967 )
...
* Add fleetctl command to get missing migrations
* Fix copy paste and lint
* Detect migrations applied out of order
* Add extra bullet to changes
* Trigger creation of migration status tables
* Fix unit tests
* PR fixes
* PR comment fixes
2021-11-22 14:06:12 -03:00
gillespi314
886b30fc4a
Remove five-row limit for teams search ( #3021 )
2021-11-22 09:30:55 -06:00
Martin Angers
69a4985cac
Use new error handling approach in other packages ( #2954 )
2021-11-22 09:13:26 -05:00
Zach Wasserman
e04f210f51
Disable storage of errors ( #3066 )
...
This is a temporary mitigation for the issue described in #3065 .
The intent is to merge this, cut a 4.6.1 release, and then come up with
a more comprehensive solution for 4.7.0.
2021-11-21 21:02:20 -08:00
Lucas Manuel Rodriguez
59e01fbe08
Make fleetctl apply -f fail with unknown kind: config fields ( #3026 )
...
* Make fleetctl apply fail with unknown fields
* Add unit test
2021-11-19 12:37:36 -03:00
Tomas Touceda
887e42650b
Further optimize host select with policies ( #3013 )
...
* Further optimize host select with policies
* Add missing row close
* Improve migration
* Skip migration if column already exists
* Add test for host with/without extras
2021-11-18 14:36:35 -03:00
Lucas Manuel Rodriguez
cb54d9a8dc
Fix duplicate schedules and platform matching on scheduled queries ( #2977 )
...
* Fix duplicate schedules and platform matching on scheduled queries
* scheduled_queries.platform can be NULL
* Add unit tests
* Add rhel host and check zero stats
2021-11-17 19:03:30 -03:00
Tomas Touceda
ddbfb7f621
Allow sort by more than one key ( #2987 )
...
* Allow sort by more than one key
* Add test for multi sort
* Expand documentation
2021-11-17 16:37:26 -03:00
gillespi314
bfdedd65e8
Allow API user to delete all team secrets ( #2941 )
2021-11-17 10:26:24 -06:00
Tomas Touceda
77a91a7ab7
Add new index to policy_memership_history ( #2973 )
...
* Add new index to policy_memership_history
* Fix typos
2021-11-17 11:42:24 -03:00
Zach Wasserman
e279dc1682
Implement fleetctl updates rotate ( #2831 )
...
Add support for updating keys used in the TUF update system.
2021-11-15 10:01:48 -08:00
Martin Angers
a4b0d9d4a1
Add team_id to host summary when filtered for a specific team ( #2869 )
2021-11-15 09:56:13 -05:00
Martin Angers
f8d118af34
Add tests for hosts dynamic where clause ( #2882 )
2021-11-15 09:55:27 -05:00
Martin Angers
45168bed1d
Remove deprecated app_configs table ( #2884 )
2021-11-15 09:34:51 -05:00
Martin Angers
3ae57a0242
Create errors with ctxerr, add the call to store them in redis ( #2786 )
2021-11-15 09:11:38 -05:00
Lucas Manuel Rodriguez
1f560c56a2
Return all scheduled queries and their stats on the host fetch API ( #2855 )
...
* Return all scheduled queries and their stats on the host fetch API (pack_stats)
* Use 'pack' instead of null and wrap errs with ctxerr
2021-11-12 08:18:25 -03:00
Tomas Touceda
a0730ca643
Add update invite API ( #2837 )
2021-11-11 17:33:06 -03:00
gillespi314
3e2dad8caa
Add login shell information to host users table ( #2854 )
2021-11-11 11:26:03 -06:00
gillespi314
229b91b530
Add endpoint for management of team enroll secrets ( #2849 )
2021-11-11 10:45:39 -06:00
Tomas Touceda
b802af6f44
Add host count to software API ( #2879 )
...
* Add host count to software API
* Update docs
* Update fleetctl tests to account for host counts
* Update docs to mention host_count special case
* Update func comment
2021-11-11 08:49:17 -03:00
Tomas Touceda
88b32d8c7f
List policies for hosts even if they don't have stats ( #2865 )
2021-11-11 08:40:32 -03:00
Tomas Touceda
b95bb52c89
Issue 2868 ignore admin reset password sso ( #2877 )
...
* Ignore admin forced password reset if sso is enabled
* Add tests for authenticatedUser
* If SSOEnabled is true, then set AdminForcedPasswordReset to false
* Update tests to cover new password reset and sso flow
* Add changes file
2021-11-10 19:30:15 -03:00
Martin Angers
b7ed3589a8
Fix where clause building with right hosts alias ( #2875 )
2021-11-10 12:24:19 -05:00
Tomas Touceda
058ee84419
Change quote so it doesn't display as weird chars in webhooks ( #2861 )
2021-11-09 18:58:22 -03:00
Martin Angers
b57b64ccb2
Add total and per platform counts to host summary endpoint ( #2845 )
2021-11-09 09:35:36 -05:00
Tomas Touceda
cc7a405f93
Enable software inventory for new installations ( #2852 )
2021-11-09 10:28:14 -03:00
Tomas Touceda
d0777ccfd3
Change message for text so webhook works on slack ( #2838 )
...
* Change message for text so webhook works on slack
* Update preview text for the webhook in the FE
2021-11-08 15:13:02 -03:00
Lucas Manuel Rodriguez
bb187a7885
Fix flaky TestSoftware/filters_by_team_and_paginates ( #2835 )
2021-11-08 13:07:42 -03:00
Tomas Touceda
7db6de7278
Serialize hosts writes per instance ( #2753 )
...
* Serialize hosts writes per instance
* Write hosts asynchronously
* Dont make the save in a goroutine
* Revert "Dont make the save in a goroutine"
This reverts commit 4a890c5271142755dec69a741582e7eca5c4c62c.
* Make all savehosts async
* Address review comments and make this approach configurable
* Address review comments
* Disable bulk seen time marking for a test
* Move host seen times to a new table
* Remove unused
* Add seen_time to list hosts
* Add some jitter to seen time flushing
* Remove unused
* Add timeout to deferred save host
* Add tests for serialSaveHost
* Update hosts in labels and policy executions in a serial way
* Address review comments and remove fk constraints in host software
* Make errCh buffered
* Add changes file
* Readd key
2021-11-08 11:42:37 -03:00
Martin Angers
1b5ec19cf5
Add author_email to payload of queries (Get/List) ( #2798 )
2021-11-08 09:06:56 -05:00
Tomas Touceda
586c2f9ead
Properly parse order direction ( #2812 )
2021-11-05 13:36:05 -03:00
Lucas Manuel Rodriguez
837bfed167
Try goqu as sql builder on listSoftwareDB ( #2773 )
...
* Try goqu as sql builder on listSoftwareDB
* Fix case insensitive search and add test
2021-11-04 15:21:39 -03:00
Lucas Manuel Rodriguez
21908f31cf
GET packs to always return disabled field ( #2799 )
2021-11-04 14:25:08 -03:00
Tomas Touceda
2902da76ca
Issue 2027 better error visibility ( #2069 )
2021-11-02 13:35:57 -04:00
Lucas Manuel Rodriguez
43d1a8c9a0
Fix races in go tests and run with -race on CI ( #2712 )
...
* Fix races in go tests and run with -race on CI
* Fix race in pubsub
* Increase timeout to 15m for go tests
* CI takes forever, try disabling race
* Remove timeout from go tests
2021-11-01 15:31:01 -03:00
Lucas Manuel Rodriguez
8642bb785e
Add MVP support for software inventory to osquery-perf for load testing ( #2751 )
...
* Add MVP support for software inventory to osquery-perf for load testing
* Fix test compile
2021-11-01 15:23:31 -03:00
Martin Angers
a8735d55bb
Implement async processing of hosts for label queries ( #2288 )
2021-11-01 14:13:16 -04:00
Tomas Touceda
fcb5d5b392
Make vulnerability processing less RAM hungry ( #2739 )
...
* Make preview work when run from scratch (no orbit running)
* Make vulnerability processing less RAM hungry
* Add changes file
* Only get the cpe list once
* Remove cache
* Try killing osquery as well as orbit and clear their db dir
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2021-10-29 11:27:12 -03:00
Tomas Touceda
1aea7484d4
Forward policy resolution ( #2733 )
...
* Pass through the resoution for the policy
* Update tests
2021-10-28 10:10:03 -03:00
dsbaha
51e35e1ba0
Implementation of a Kafka REST Proxy logging plugin ( #2534 )
...
This PR implements the status/result logger functions necessary interface with a Kafka REST Proxy service.
Specifically, this is compatible with the [Confluent KAFKA Rest Proxy Service ](https://docs.confluent.io/1.0/kafka-rest/docs/intro.html ).
2021-10-27 21:51:17 -07:00
Zach Wasserman
fe23fcd3ee
Add documentation to package launcher ( #2669 )
2021-10-26 21:12:44 -07:00
Tomas Touceda
623a38aa9d
Live queries post merge fixes ( #2684 )
...
* Refactor based on PR review comments
* Use deadline instead of duration
* Address review comments
* Only set the timeout once
2021-10-26 16:12:07 -03:00
Tomas Touceda
79f2c7c391
Issue 1798 rest api for live queries ( #2675 )
...
* wip
* wip
* Make tests not flaky
* Add changes file
* Make logging and authz thread safe
* Move cancel defer below err check
2021-10-26 11:33:31 -03:00
Lucas Manuel Rodriguez
3136cc105e
Replace into software to update empty bundle_identifier entries ( #2676 )
2021-10-26 09:48:25 -03:00
Martin Angers
c0e0e461e6
Allow empty TLS CA for Redis TLS connections ( #2668 )
2021-10-25 14:47:53 -04:00
Lucas Manuel Rodriguez
fb1e78d039
Make /refetch run labels and policies for the host ( #2651 )
...
* Make /refetch run labels and policies for the host
* PR rework changes
2021-10-25 15:46:49 -03:00
Tomas Touceda
3369436741
Remove policies yaml for now and apply policies in preview ( #2644 )
2021-10-25 09:17:34 -03:00
RachelElysia
36babcc510
UI feature: Frontend of performance impact bubbles ( #2589 )
...
Includes backend fixes and test
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2021-10-22 16:05:49 -04:00
Luke Heath
99673eb148
Fix broken links to docs ( #2561 )
2021-10-22 12:39:20 -05:00
Lucas Manuel Rodriguez
c84cbb1679
Fix SearchHosts to match for one-char and two-chars queries ( #2590 )
...
* Fix SearchHosts to match for one-char and two-chars queries
* Add issue number for future reference
2021-10-21 17:46:21 -03:00
Tomas Touceda
bcf6697741
Handle null resolution in host details ( #2622 )
...
* Handle null resolution in host details
* Add test for null resolution
2021-10-21 15:53:23 -03:00
Lucas Manuel Rodriguez
91ae5ba870
By default, force password reset on CreateUser ( #2570 )
...
* By default, force password reset on CreateUser
* Use ptr.Bool and add test
* Add changes file
* Simple fix for e2e tests
* Add API changes
2021-10-21 15:00:38 -03:00
Zach Wasserman
35523017c9
Make migrations compatible with GTID replication ( #2615 )
...
* Make migrations compatible with GTID replication
Fixes an issue some deployments encountered when migrations used a
statement that is unsupported in GTID replication mode (#2462 ).
Local dev MySQL now enforces this consistency, so it should be easier to
maintain compatibility going forward.
* Update docker-compose formatting
* if exists
2021-10-21 07:46:02 -03:00
Tomas Touceda
923d094471
Allow to skip loading CVEs in some cases ( #2611 )
2021-10-20 19:26:25 -03:00
Tomas Touceda
c2696c1261
Issue 1892 query perf everywhere ( #2580 )
...
* wip
* wip
* wip
* Add performance stats
* Add docs and other self review fixes
* lint
* Update based on review comments
* Add quick cleanup first and then reset to 1hr
* Reduce the load in the test
2021-10-20 18:35:38 -03:00
Tomas Touceda
cbe931c0e9
Add vulnerable filter for software and also wire up the query search ( #2604 )
...
* Add vulnerable filter for software and also wire up the query search
* Add documentation
* Update to use software list options
2021-10-20 18:01:20 -03:00
Tomas Touceda
86b1926d7c
Add missing join condition ( #2586 )
2021-10-20 13:24:51 -03:00
Tomas Touceda
c3f7577bd8
Add query description and resolution to the list of policies in hosts ( #2584 )
...
* Add query description and resolution to the list of policies in hosts
* Add docs
* Fix get host tests
2021-10-20 12:07:16 -03:00
Martin Angers
07cf136d03
Add TLS support to Redis connections ( #2568 )
2021-10-20 10:09:18 -04:00
