1055 Commits

Author	SHA1	Message	Date
Tomas Touceda	dc9686024b	Remove tx from SaveHost (#3232) ... * Remove tx from SaveHost Also change REPLACE for software for an insert. * Remove bundle identifier from the get since it's not part of the key * Revert unneeded change	2021-12-07 12:39:07 -03:00
gillespi314	6ee6dba4a4	Update anonymous usage statistics to record Fleet feature adoption (#3056) ... * Collect additional anonymous usage statistics * Update fleetdm.com and docs for usage stats * Fix lint-go errors * Fix failing test * update webhook inputs * Update receive-usage-analytics.js * fix failing test * Update statistics_test.go Co-authored-by: eashaw <eashaw@sailsjs.com>	2021-12-06 14:39:00 -06:00
Lucas Manuel Rodriguez	c5375263b6	Change policy platforms name to platform singular (for consistency) (#3221) ... * Change policy platforms name to platform singular * Fix fleetctl tests	2021-12-06 13:56:28 -03:00
Martin Angers	b84ef79bf4	Add the fleetctl debug {errors,db-locks} sub-commands (#3168)	2021-12-06 09:26:01 -05:00
Lucas Manuel Rodriguez	b9a408704e	Add platforms field to policies (#3181) ... * Add platforms field to policies * Fix fleetctl tests * PR review changes * Add missing tests * Add changes for ListPoliciesForHost	2021-12-03 15:33:33 -03:00
Tomas Touceda	142006cbdd	Issue 2099 enable vulnerability processing new installs (#3104) ... * Enable vulnerability processing by default * Add changes file * Remove wrong env var * Fix import * Fix tests * Update log line * Remove unneeded import	2021-12-03 14:44:57 -03:00
Tomas Touceda	17bd4d5110	Remove policy membership history and only store the latest value (#3178) ... * Remove policy membership history and only store the latest value * Address review comments * Correct error msg wrap	2021-12-03 13:10:11 -03:00
Tomas Touceda	5cff07f332	Issue 3184 drop event host expiry (#3185) ... * Drop event host_expiry if it exists * Add changes file	2021-12-03 12:34:58 -03:00
Tomas Touceda	a0404b6736	Add software count API (#3105) ... * Add software count API * Fix makefile * Fine no mock generating at this point * Actually, one last try * Use go install instead * Fix go sum/mod * Improve documentation * Try setting node to 14	2021-12-03 10:54:17 -03:00
Martin Angers	7464e72ba8	Move carves endpoints to new endpoint pattern (#3148)	2021-12-01 15:45:29 -05:00
Tomas Touceda	3a031e946d	Sort hostIDs before inserting seen time (#3162) ... * Sort hostIDs before inserting seen time * Fix status test * Add comment	2021-12-01 16:20:54 -03:00
Lucas Manuel Rodriguez	e64a88d8b1	Add COALESCEs on all host_seen_times JOINs (#3147) ... * Add COALESCEs on all host_seen_times JOINs * Use tx instead of d.writer * Add unit tests * Fix compile test	2021-12-01 09:05:23 -03:00
Tomas Touceda	4ef9cfdf63	Increase grace period of status for hosts (#3151) ... * Increase grace period of status for hosts * Update tests * Update more tests	2021-11-30 18:37:27 -03:00
Tomas Touceda	925d4df8f7	Add new disable_failing_policies query parameter to list hosts (#3123) ... * Update documentation * Update based on review comments * Explain why the switch exists	2021-11-29 18:04:33 -03:00
Tomas Touceda	c82a8e8428	Sanitize OrderKey (#3128)	2021-11-29 18:03:19 -03:00
Tomas Touceda	036093874d	Add cursor pagination to list hosts (#3120) ... * Add cursor pagination to list hosts * Update docs * Update based on review comments	2021-11-29 15:06:00 -03:00
Tomas Touceda	a36e029d95	App config caching per instance (#3071) ... * Do caching of app config per instance instead of across all of them in redis * Add changes file * Simplify code based on review comment * Use go-cache instead of creating our own * Dont export consts * Copy app config before returning it * Fix lint * Update go sum * Update go sum	2021-11-29 12:51:57 -03:00
Martin Angers	5c113bd468	Refactor ListActivities to new endpoint pattern (#3115)	2021-11-29 08:12:22 -05:00
Lucas Manuel Rodriguez	c6fc91f2e4	Fix ListPoliciesForHost, missing filter by teams (#3106) ... * Fix ListPoliciesForHost, missing filter by teams * Amend existing tests	2021-11-24 18:17:44 -03:00
Martin Angers	c997f853e5	Make creation of http.Client uniform across the codebase (#3097)	2021-11-24 15:56:54 -05:00
Lucas Manuel Rodriguez	964f85b174	Amend policy creation and spec (for proprietary query), and add update APIs (#2890) ... * Amend policy creation (proprietary query), add update APIs * Fix Datastore.SavePolicy bug (and add tests) * Add integration tests for new policy APIs * Add author email * Add activities * Push breaking changes for return policy fields * WIP * Add integration test for host policies * Make more improvements to policy representation * Improve upgrade code (from PR review comments) * PR changes * Revert activities for policies * Use *uint instead of uint for queryID, use fleet.PolicyPayload * Filter out other schemas * New policy flow (#2922) * created new policy flow -- no API connection * added api props * fixed prop name * lint fixes * removed unused modal; fixed style * name, desc icons; created global components * lint fixes * ignoring certain files and lines for prettier * Update frontend/pages/policies/PolicyPage/PolicyPage.tsx * Make policy names unique across deployment * Amend upgrade script * Fix migration for unique names * Do not deduplicate but instead rename policies Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>	2021-11-24 14:16:42 -03:00
Tomas Touceda	c65f8507fe	Remove host counts from software (#3082) ... * Remove host counts from software * Actually remove the host count from the struct * Fix get test	2021-11-23 15:50:51 -03:00
Martin Angers	67293aca95	Add tests to SSO/SAML implementation (#2997)	2021-11-23 08:25:43 -05:00
Tomas Touceda	4cd9dab577	Update shell in host users when they exist (#3072)	2021-11-23 10:23:12 -03:00
Lucas Manuel Rodriguez	028c2643f7	Fleet serve to exit if migrations are missing (default) (#2803) ... * Add option to fleet to exit if migrations are missing * Reverse serve exit migrations logic * Fix typo	2021-11-22 14:47:24 -03:00
Lucas Manuel Rodriguez	0642bfdb1d	Add fleetctl debug migrations command to get missing migrations (#2967) ... * Add fleetctl command to get missing migrations * Fix copy paste and lint * Detect migrations applied out of order * Add extra bullet to changes * Trigger creation of migration status tables * Fix unit tests * PR fixes * PR comment fixes	2021-11-22 14:06:12 -03:00
gillespi314	886b30fc4a	Remove five-row limit for teams search (#3021)	2021-11-22 09:30:55 -06:00
Martin Angers	69a4985cac	Use new error handling approach in other packages (#2954)	2021-11-22 09:13:26 -05:00
Zach Wasserman	e04f210f51	Disable storage of errors (#3066) ... This is a temporary mitigation for the issue described in #3065. The intent is to merge this, cut a 4.6.1 release, and then come up with a more comprehensive solution for 4.7.0.	2021-11-21 21:02:20 -08:00
Lucas Manuel Rodriguez	59e01fbe08	Make fleetctl apply -f fail with unknown kind: config fields (#3026) ... * Make fleetctl apply fail with unknown fields * Add unit test	2021-11-19 12:37:36 -03:00
Tomas Touceda	887e42650b	Further optimize host select with policies (#3013) ... * Further optimize host select with policies * Add missing row close * Improve migration * Skip migration if column already exists * Add test for host with/without extras	2021-11-18 14:36:35 -03:00
Lucas Manuel Rodriguez	cb54d9a8dc	Fix duplicate schedules and platform matching on scheduled queries (#2977) ... * Fix duplicate schedules and platform matching on scheduled queries * scheduled_queries.platform can be NULL * Add unit tests * Add rhel host and check zero stats	2021-11-17 19:03:30 -03:00
Tomas Touceda	ddbfb7f621	Allow sort by more than one key (#2987) ... * Allow sort by more than one key * Add test for multi sort * Expand documentation	2021-11-17 16:37:26 -03:00
gillespi314	bfdedd65e8	Allow API user to delete all team secrets (#2941)	2021-11-17 10:26:24 -06:00
Tomas Touceda	77a91a7ab7	Add new index to policy_memership_history (#2973) ... * Add new index to policy_memership_history * Fix typos	2021-11-17 11:42:24 -03:00
Zach Wasserman	e279dc1682	Implement fleetctl updates rotate (#2831) ... Add support for updating keys used in the TUF update system.	2021-11-15 10:01:48 -08:00
Martin Angers	a4b0d9d4a1	Add team_id to host summary when filtered for a specific team (#2869)	2021-11-15 09:56:13 -05:00
Martin Angers	f8d118af34	Add tests for hosts dynamic where clause (#2882)	2021-11-15 09:55:27 -05:00
Martin Angers	45168bed1d	Remove deprecated app_configs table (#2884)	2021-11-15 09:34:51 -05:00
Martin Angers	3ae57a0242	Create errors with ctxerr, add the call to store them in redis (#2786)	2021-11-15 09:11:38 -05:00
Lucas Manuel Rodriguez	1f560c56a2	Return all scheduled queries and their stats on the host fetch API (#2855) ... * Return all scheduled queries and their stats on the host fetch API (pack_stats) * Use 'pack' instead of null and wrap errs with ctxerr	2021-11-12 08:18:25 -03:00
Tomas Touceda	a0730ca643	Add update invite API (#2837)	2021-11-11 17:33:06 -03:00
gillespi314	3e2dad8caa	Add login shell information to host users table (#2854)	2021-11-11 11:26:03 -06:00
gillespi314	229b91b530	Add endpoint for management of team enroll secrets (#2849)	2021-11-11 10:45:39 -06:00
Tomas Touceda	b802af6f44	Add host count to software API (#2879) ... * Add host count to software API * Update docs * Update fleetctl tests to account for host counts * Update docs to mention host_count special case * Update func comment	2021-11-11 08:49:17 -03:00
Tomas Touceda	88b32d8c7f	List policies for hosts even if they don't have stats (#2865)	2021-11-11 08:40:32 -03:00
Tomas Touceda	b95bb52c89	Issue 2868 ignore admin reset password sso (#2877) ... * Ignore admin forced password reset if sso is enabled * Add tests for authenticatedUser * If SSOEnabled is true, then set AdminForcedPasswordReset to false * Update tests to cover new password reset and sso flow * Add changes file	2021-11-10 19:30:15 -03:00
Martin Angers	b7ed3589a8	Fix where clause building with right hosts alias (#2875)	2021-11-10 12:24:19 -05:00
Tomas Touceda	058ee84419	Change quote so it doesn't display as weird chars in webhooks (#2861)	2021-11-09 18:58:22 -03:00
Martin Angers	b57b64ccb2	Add total and per platform counts to host summary endpoint (#2845)	2021-11-09 09:35:36 -05:00
Tomas Touceda	cc7a405f93	Enable software inventory for new installations (#2852)	2021-11-09 10:28:14 -03:00
Tomas Touceda	d0777ccfd3	Change message for text so webhook works on slack (#2838) ... * Change message for text so webhook works on slack * Update preview text for the webhook in the FE	2021-11-08 15:13:02 -03:00
Lucas Manuel Rodriguez	bb187a7885	Fix flaky TestSoftware/filters_by_team_and_paginates (#2835)	2021-11-08 13:07:42 -03:00
Tomas Touceda	7db6de7278	Serialize hosts writes per instance (#2753) ... * Serialize hosts writes per instance * Write hosts asynchronously * Dont make the save in a goroutine * Revert "Dont make the save in a goroutine" This reverts commit 4a890c5271142755dec69a741582e7eca5c4c62c. * Make all savehosts async * Address review comments and make this approach configurable * Address review comments * Disable bulk seen time marking for a test * Move host seen times to a new table * Remove unused * Add seen_time to list hosts * Add some jitter to seen time flushing * Remove unused * Add timeout to deferred save host * Add tests for serialSaveHost * Update hosts in labels and policy executions in a serial way * Address review comments and remove fk constraints in host software * Make errCh buffered * Add changes file * Readd key	2021-11-08 11:42:37 -03:00
Martin Angers	1b5ec19cf5	Add author_email to payload of queries (Get/List) (#2798)	2021-11-08 09:06:56 -05:00
Tomas Touceda	586c2f9ead	Properly parse order direction (#2812)	2021-11-05 13:36:05 -03:00
Lucas Manuel Rodriguez	837bfed167	Try goqu as sql builder on listSoftwareDB (#2773) ... * Try goqu as sql builder on listSoftwareDB * Fix case insensitive search and add test	2021-11-04 15:21:39 -03:00
Lucas Manuel Rodriguez	21908f31cf	GET packs to always return disabled field (#2799)	2021-11-04 14:25:08 -03:00
Tomas Touceda	2902da76ca	Issue 2027 better error visibility (#2069)	2021-11-02 13:35:57 -04:00
Lucas Manuel Rodriguez	43d1a8c9a0	Fix races in go tests and run with -race on CI (#2712) ... * Fix races in go tests and run with -race on CI * Fix race in pubsub * Increase timeout to 15m for go tests * CI takes forever, try disabling race * Remove timeout from go tests	2021-11-01 15:31:01 -03:00
Lucas Manuel Rodriguez	8642bb785e	Add MVP support for software inventory to osquery-perf for load testing (#2751) ... * Add MVP support for software inventory to osquery-perf for load testing * Fix test compile	2021-11-01 15:23:31 -03:00
Martin Angers	a8735d55bb	Implement async processing of hosts for label queries (#2288)	2021-11-01 14:13:16 -04:00
Tomas Touceda	fcb5d5b392	Make vulnerability processing less RAM hungry (#2739) ... * Make preview work when run from scratch (no orbit running) * Make vulnerability processing less RAM hungry * Add changes file * Only get the cpe list once * Remove cache * Try killing osquery as well as orbit and clear their db dir Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>	2021-10-29 11:27:12 -03:00
Tomas Touceda	1aea7484d4	Forward policy resolution (#2733) ... * Pass through the resoution for the policy * Update tests	2021-10-28 10:10:03 -03:00
dsbaha	51e35e1ba0	Implementation of a Kafka REST Proxy logging plugin (#2534) ... This PR implements the status/result logger functions necessary interface with a Kafka REST Proxy service. Specifically, this is compatible with the [Confluent KAFKA Rest Proxy Service ](https://docs.confluent.io/1.0/kafka-rest/docs/intro.html).	2021-10-27 21:51:17 -07:00
Zach Wasserman	fe23fcd3ee	Add documentation to package launcher (#2669)	2021-10-26 21:12:44 -07:00
Tomas Touceda	623a38aa9d	Live queries post merge fixes (#2684) ... * Refactor based on PR review comments * Use deadline instead of duration * Address review comments * Only set the timeout once	2021-10-26 16:12:07 -03:00
Tomas Touceda	79f2c7c391	Issue 1798 rest api for live queries (#2675) ... * wip * wip * Make tests not flaky * Add changes file * Make logging and authz thread safe * Move cancel defer below err check	2021-10-26 11:33:31 -03:00
Lucas Manuel Rodriguez	3136cc105e	Replace into software to update empty bundle_identifier entries (#2676)	2021-10-26 09:48:25 -03:00
Martin Angers	c0e0e461e6	Allow empty TLS CA for Redis TLS connections (#2668)	2021-10-25 14:47:53 -04:00
Lucas Manuel Rodriguez	fb1e78d039	Make /refetch run labels and policies for the host (#2651) ... * Make /refetch run labels and policies for the host * PR rework changes	2021-10-25 15:46:49 -03:00
Tomas Touceda	3369436741	Remove policies yaml for now and apply policies in preview (#2644)	2021-10-25 09:17:34 -03:00
RachelElysia	36babcc510	UI feature: Frontend of performance impact bubbles (#2589) ... Includes backend fixes and test Co-authored-by: Tomas Touceda <chiiph@gmail.com>	2021-10-22 16:05:49 -04:00
Luke Heath	99673eb148	Fix broken links to docs (#2561)	2021-10-22 12:39:20 -05:00
Lucas Manuel Rodriguez	c84cbb1679	Fix SearchHosts to match for one-char and two-chars queries (#2590) ... * Fix SearchHosts to match for one-char and two-chars queries * Add issue number for future reference	2021-10-21 17:46:21 -03:00
Tomas Touceda	bcf6697741	Handle null resolution in host details (#2622) ... * Handle null resolution in host details * Add test for null resolution	2021-10-21 15:53:23 -03:00
Lucas Manuel Rodriguez	91ae5ba870	By default, force password reset on CreateUser (#2570) ... * By default, force password reset on CreateUser * Use ptr.Bool and add test * Add changes file * Simple fix for e2e tests * Add API changes	2021-10-21 15:00:38 -03:00
Zach Wasserman	35523017c9	Make migrations compatible with GTID replication (#2615) ... * Make migrations compatible with GTID replication Fixes an issue some deployments encountered when migrations used a statement that is unsupported in GTID replication mode (#2462). Local dev MySQL now enforces this consistency, so it should be easier to maintain compatibility going forward. * Update docker-compose formatting * if exists	2021-10-21 07:46:02 -03:00
Tomas Touceda	923d094471	Allow to skip loading CVEs in some cases (#2611)	2021-10-20 19:26:25 -03:00
Tomas Touceda	c2696c1261	Issue 1892 query perf everywhere (#2580) ... * wip * wip * wip * Add performance stats * Add docs and other self review fixes * lint * Update based on review comments * Add quick cleanup first and then reset to 1hr * Reduce the load in the test	2021-10-20 18:35:38 -03:00
Tomas Touceda	cbe931c0e9	Add vulnerable filter for software and also wire up the query search (#2604) ... * Add vulnerable filter for software and also wire up the query search * Add documentation * Update to use software list options	2021-10-20 18:01:20 -03:00
Tomas Touceda	86b1926d7c	Add missing join condition (#2586)	2021-10-20 13:24:51 -03:00
Tomas Touceda	c3f7577bd8	Add query description and resolution to the list of policies in hosts (#2584) ... * Add query description and resolution to the list of policies in hosts * Add docs * Fix get host tests	2021-10-20 12:07:16 -03:00
Martin Angers	07cf136d03	Add TLS support to Redis connections (#2568)	2021-10-20 10:09:18 -04:00
Tomas Touceda	fe5660e006	Reimplement host expiration to not need mysql events (#2552) ... * Reimplement host expiration to not need mysql events * Update mocks	2021-10-19 17:47:37 -03:00
Martin Angers	a7420140ce	Use a redis cluster-friendly store for rate limit (#2577)	2021-10-19 16:24:13 -04:00
Martin Angers	b1bd3aee5b	Try to fix flaky redis standalone test on ci (#2553)	2021-10-18 11:08:36 -04:00
Tomas Touceda	b783b5721f	Add RoleAdmin to ValidTeamRoles for verification (#2551)	2021-10-18 11:38:47 -03:00
Martin Angers	057d4e8b2e	Add configuration and support for Redis to read from replicas (#2509)	2021-10-18 09:32:17 -04:00
Tomas Touceda	74960d74be	Update calls to new signature (#2536)	2021-10-15 10:48:44 -03:00
Tomas Touceda	89c6d18ed2	Issue 2293 reveal failing policies list host (#2528) ... * Add host issues * Update docs * Remove comment * Update tests	2021-10-15 07:34:30 -03:00
Tomas Touceda	d3a0d62902	Issue 2456 policies yaml (#2512) ... * wip * Add policy specs support * Add documentation * Make policy apply idempotent * Fold in code * Improve tests and simplify auth checks * Lint and fix test	2021-10-15 07:34:11 -03:00
Tomas Touceda	bb1dc401a3	Load vulnerabilities alongisde the software (#2518) ... * Load vulnerabilities alongisde the software * Update docs	2021-10-14 13:51:41 -03:00
Zach Wasserman	4d6956b6cb	Remove usage of temporary table in migration (#2515) ... Temporary tables were causing migration issues under some MySQL configurations as discussed in #2462.	2021-10-13 16:25:08 -07:00
Tomas Touceda	0510075398	Issue 2134 add team admin role (#2499) ... * wip * Add team admin role and tests * Revert change in invites * Update permission doc * Fix lint	2021-10-13 12:34:59 -03:00
Benjamin Edwards	16c5823692	add support for minio backend file carving (#2448) ... * add support for minio backend file carving * add changes file Co-authored-by: Zach Wasserman <zach@fleetdm.com>	2021-10-12 15:32:06 -04:00
Tomas Touceda	28be73c410	Add missing software in list hosts response (#2492)	2021-10-12 15:59:01 -03:00
Tomas Touceda	782f276e0b	Issue 2050 add software filter list hosts (#2486) ... * Add software_id filter to list hosts * Add integration test for API endpoint * Add doc * Simplify sql query	2021-10-12 11:38:12 -03:00
RachelElysia	981028705e	Sentence Case: Throughout UI, tests, validators, docs (#2455)	2021-10-11 10:58:27 -04:00
Martin Angers	5e1f872ccb	Refactor ListHosts to new endpoint pattern (#2396)	2021-10-11 10:37:48 -04:00
Martin Angers	fce3e42abb	Refactor GetPack to new endpoint pattern (#2409)	2021-10-11 10:17:21 -04:00
Tomas Touceda	0fd569ebaf	Move migrations around and add missing column only if not available (#2461) ... * Move migrations around and add missing column only if not available * Add changes file * Remove debug print * Remove wrong import	2021-10-08 15:41:52 -03:00
Tomas Touceda	e08fcb949b	Fix logging of errors in kitlog (#2427) ... * Fix logging of errors in kitlog * Update error messages	2021-10-07 18:15:40 -03:00
Tomas Touceda	9b19e3ace6	Count can happen with only list permissions (#2441)	2021-10-07 14:25:04 -03:00
Martin Angers	3aef96e15c	Add test for fleetctl preview (#2388) ... * Start a fleetctl preview test * Add tests for fleetctl preview * Fix setting of fleetctl auth token in test * Add fleet instance vulnerabilities config to response of GetAppConfig * Add checks that fleetctl preview enables vulnerability detection * Adjust doc for get config API response * Add the include-server-config flag to fleetctl get config * Update test now that some of the PRs have been merged Co-authored-by: Tomas Touceda <chiiph@gmail.com>	2021-10-07 10:19:10 -03:00
Tomas Touceda	876aa0e366	Add host count API (#2356) ... * Add host count API * Add rest api docs * Add host count for labels and address review comments * Add regex for id	2021-10-07 08:25:35 -03:00
Tomas Touceda	9b809a2d35	Issue 1890 host details policy (#2410) ... * wip * Add policies to hosts	2021-10-07 08:11:10 -03:00
Tomas Touceda	f44c809d85	Check env var along with app config for software inv (#2406)	2021-10-06 14:17:33 -03:00
Tomas Touceda	1b611012df	Allow team maintainers to bulk delete hosts for teams they belong (#2399)	2021-10-06 12:58:01 -03:00
Tomas Touceda	1af3803cdb	Load all host values from the db to prevent overwriting columns with empty vals (#2394)	2021-10-06 09:27:53 -03:00
Tomas Touceda	02b2c64b89	Make software list not flaky, hopefully (#2387)	2021-10-05 18:26:02 -03:00
Tomas Touceda	70cf7aa0a0	Delete policies for hosts in teams before transferring them (#2383) ... * Delete policies for hosts in teams before transferring them * Add missing error check	2021-10-05 15:48:26 -03:00
Tomas Touceda	ddc6b300d4	Allow team maintainers to delete hosts from their teams (#2373)	2021-10-05 15:15:05 -03:00
Zach Wasserman	a2e9b539a1	Optimize policy_updated_at migration (#2362) ... - Use `TRUNCATE TABLE` rather than `DELETE FROM` for improved performance. - Move DDL statement after truncate to avoid issues with retries (due to column already being created). #2360	2021-10-05 09:24:03 -07:00
Tomas Touceda	2df4389aa0	Ignore empty host users or software inventory (#2317) ... * Ignore empty host users or software inventory * Only store additional if it's not nil * Update label/policy updated at when we record the executions and skip saving host * Update changes file	2021-10-01 18:27:57 -03:00
Martavis Parker	b6659cfaaa	Refactored Manage Hosts (#2233) ... * Add new feature: team policies * Continue work on team policies * Continue work on team policies * Continue team policies * Revert accidental deletion * Rename variables * code refactored; working on runtime errors * updated front end docs * Update URLs from team to teams, add tests for policy auth * Fix test * Continue work on team policies * Add permission checks * mange hosts functional and cleaned up; typing * improved label logic * added try catch to awaits * lint fixes * frontend unit tests don't work for functional components * test fix * revert * Address errors related to refetch on window focus * Add loading error check * Fix typos in loading error checks * Guard against invariant condition in useEffect * Update links and routes for team policies * lint fixes * Update frontend/pages/hosts/ManageHostsPage/helpers.ts Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com> * Change inherited policies button, tooltip * lint fixes Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Tomas Touceda <chiiph@gmail.com>	2021-09-30 12:32:06 -07:00
Tomas Touceda	36b4c0df5d	Allow team maintainers to read global policies and schedule (#2282) ... * Allow team maintainers to read global policies and schedules * Update docs	2021-09-29 14:07:10 -03:00
Tomas Touceda	d9f34b2c19	Issue 1840 bulk delete hosts (#2268) ... * wp * Add test by ids * Add changes file * Update docs and remove unneeded return values * Address review comments * Improve integration tests * Use TearDownTest	2021-09-29 13:13:23 -03:00
Tomas Touceda	435178e93c	Add bundle identifier to software when available (#2220) ... * Add bundle identifier to software when available * Update docs * Delete unneeded test	2021-09-28 18:13:34 -03:00
Tomas Touceda	e2caf46d6d	Issue 2133 team maintainer can edit delete queries (#2256) ... * wip * Team maintainers can edit and delete queries they authored * Update documentation * Fix test	2021-09-28 14:53:05 -03:00
Tomas Touceda	5695d2a9ae	Issue 2058 team maintainer add queries (#2255) ... * Team maintainers can read and write team schedules * Allow team maintainers to read and write team schedules	2021-09-28 13:34:43 -03:00
Tomas Touceda	685245c2bd	Cache app config in redis (#2205) ... * Cache app config in redis * Add changes files * Replace string with constant * Revert some test refactorign and duplicate a bit of test code * Add test for AppConfig with redis failing * Fix lint * Use Doer so it works better in clusters * Skip unmarshalling if we already did * Allow to cache hosts if configured * Omit the setting if empty * Remove hashing, too much CPU * Revert caching of host auth... needs a more thought through approach * Remove config * Remove old config * Remove locker interface * Fix test and address review comments	2021-09-28 10:01:53 -03:00
Tomas Touceda	2033d8208c	Add policy updated at (#2246) ... * wip * Add policy updated at interval and update the UI to use that * Update rest api * Fix tests	2021-09-27 16:27:38 -03:00
Tomas Touceda	5653f1e868	Update URLs from team to teams, add tests for policy auth (#2228) ... * Update URLs from team to teams, add tests for policy auth * Fix test * Address review comments	2021-09-27 14:02:11 -03:00
Tomas Touceda	d81a6317a0	Return host count when modifying a label (#2221)	2021-09-24 15:56:55 -03:00
Tomas Touceda	8931163882	Don't check authViewer if there's no bearer token (#2200)	2021-09-23 12:44:04 -03:00
Tomas Touceda	6497e0ba2e	Improve performance of cascade host software migration (#2163)	2021-09-21 16:37:13 -03:00
Tomas Touceda	1f324339f8	Add jitter to intervals (#2158) ... * Add max jitter percent config * Fix jitter calc * Remove comment * Reduce test jitter to make tests less flaky * Remove jitter entirely * Document new config * Fix doc link	2021-09-21 14:21:44 -03:00
Tomas Touceda	4650484960	Remove fk label membership (#2157) ... * Remove fk from label membership * Add changes file * Fix tests * No need to IGNORE anymore	2021-09-21 11:48:20 -03:00
Martin Angers	86dce785ae	Improve performance of the Go test suite (#2060) ... Closes #1805	2021-09-20 14:09:38 -04:00
Lucas Manuel Rodriguez	c69937945a	Introduce entityName type for mysql entity table names (#2139) ... * Introduce entity type to specify mysql table names for deleteEntit* functions * Remove changes entry for issue (non-user facing changes)	2021-09-20 14:47:06 -03:00
Lucas Manuel Rodriguez	eecef148eb	Fail early if process does not have permissions to write to log file (#2138) ... * Fail early if process does not have permissions to write to log file * Open file once on NewFilesystemLogWriter	2021-09-20 14:46:51 -03:00
Tomas Touceda	b32b441c12	Issue 1512 filter observer can run queries (#2110) ... * wip * Filter queries for observers * Update e2e test now that we filter queries	2021-09-20 13:07:51 -03:00
Tomas Touceda	e286ee387e	Allow team maintainers to run new queries in the team hosts (#2076) ... * Allow team maintainers to run new queries in the team hosts * Add policies for other roles	2021-09-20 11:09:51 -03:00
Tomas Touceda	baa42d367e	Add team policies (#2103) ... * Add team policies * Add team policy documentation * Add changes file * Update titles * Fix lint * Rewrite TeamAuthorize for more clarify * Explicitly use two slices for clarity * Simplify switch	2021-09-20 11:00:57 -03:00
Tomas Touceda	bdae8d04a2	Skip saving host users and inventory if disabled (#2127)	2021-09-18 11:33:36 -03:00
Tomas Touceda	e6368cc57f	Refactor integration tests (#1821) ... * Refactor integration tests * Remove nopCloser and use io.NopCloser * Address review comments	2021-09-15 16:27:53 -03:00
Martin Angers	1fa5ce16b8	Add configurable Redis connection retries and following of cluster redirections (#2045) ... Closes #1969	2021-09-15 08:50:32 -04:00
Tomas Touceda	15cf91dcc0	Add limit 1 to disk space queries (#2063)	2021-09-14 18:17:52 -03:00
Martin Angers	d956ca1bec	Use the context in the Datastore layer. (#2030)	2021-09-14 10:44:02 -04:00
Tomas Touceda	176037cad2	Implement fleetctl get software and the underlying API (#1999) ... * Implement fleetctl get software and the underlying API * Add documentation * Simplify list software implementation * Lint fixes * Make team name unique * Address review comments * Fix lint * Fix tests	2021-09-14 10:58:48 -03:00
Tomas Touceda	429af72b69	Issue 1963 vulnerabilities no sync (#1976) ... * wip * Add tests for skip sync * Add changes file * Fix lint	2021-09-14 10:58:35 -03:00
Martin Angers	4f4185372d	Add support for context in datastore/mysql layer (#1962) ... This is just to pass down the context to the datastore layer, it doesn't use it just yet - this will be in a follow-up PR.	2021-09-14 08:11:07 -04:00
Tomas Touceda	db83c40d45	Make app config json migration not be skipped by patch release (#2034) ... * Make app config json migration not be skipped by patch release * Update test schema	2021-09-13 14:41:50 -03:00
Benjamin Edwards	42014c02e3	Fix migration that isn't compatible with some mysql8 settings (#1992) ... Fixes #1991 Co-authored-by: Zach Wasserman <zach@fleetdm.com>	2021-09-13 10:20:31 -07:00
keepwn	b42d213689	Support close the websocket of LiveQueryResultsHandler (#1006) ... * Support close `LiveQueryResultsHandler` * Start adding test * Make LiveQuery exit when the context is Done * Fix lint and remove debug print * Update server/service/client_live_query.go Co-authored-by: Zach Wasserman <zach@fleetdm.com> * Revert "Update server/service/client_live_query.go" This reverts commit be67ca1512fe502503e821393c2b9e84f5e6e82e. Co-authored-by: Tomas Touceda <chiiph@gmail.com> Co-authored-by: Zach Wasserman <zach@fleetdm.com>	2021-09-10 16:26:39 -03:00
Tomas Touceda	13e5e6a6db	Debug log per host (#1978) ... * Add extra debug logging for hosts * Add changes file * Ignore if appconfig is nil * Use slice of uints instead of a string * Debug response request for enabled hosts * Add host-id to request/response * Lint fixes * Add missing AppConfigFuncs	2021-09-10 14:48:33 -03:00
Martin Angers	d267fb5168	Fix reading policy after creation for read replicas (#1983)	2021-09-09 16:23:35 -04:00
Tomas Touceda	b2efc9f51c	Make redis conn timeout and keep alive configurable (#1968) ... * Make redis conn timeout and keep alive configurable * Document new configs * Correct config name	2021-09-08 17:55:12 -03:00
Martin Angers	a099854045	Add missing db transactions, read from primary when after a write (#1957)	2021-09-08 14:43:22 -04:00
Tomas Touceda	69e06476a7	Ignore nonexistent label query executions (#1953)	2021-09-07 15:51:49 -03:00
Martin Angers	0ac4fc4c1a	Update redisc and replace EachRedisNode implementation (#1949)	2021-09-07 13:25:36 -04:00
Tomas Touceda	7c34956d31	Add coverage to uncovered mysql code (#1855) ... * Add coverage to uncovered mysql code * Add deleted method and update mock * Fix test	2021-09-07 13:48:04 -03:00
Tomas Touceda	a6acb1cd1a	Allow users to be readded if they were ever removed (#1945) ... * Allow users to be readded if they were ever removed * Address review comment * lint	2021-09-07 13:33:40 -03:00
Tomas Touceda	99e9e846f4	Remove id column from host_users to allow for bulk insert (#1931) ... * Remove id column from host_users to allow for bulk insert * Update documentation	2021-09-07 11:02:35 -03:00
Tomas Touceda	d5a5f014ef	Add test to check that two hosts can store stats concurrently (#1929)	2021-09-07 10:39:17 -03:00
Tomas Touceda	cb687f6e2d	Increase timeout to check for smpt availability (#1942)	2021-09-06 14:29:52 -03:00
gillespi314	4a307f1274	Update email template image links (#1938) ... * Update email image links; move pngs to fleetdm.com	2021-09-03 17:00:30 -05:00
Tomas Touceda	6d2ae02efd	Rename core->free and basic->premium (#1870) ... * Rename core->free and basic->premium * Fix lint js * Comment out portion of test that seems to timeout * Rename tier to premium if basic is still loaded	2021-09-03 13:05:23 -03:00
Tomas Touceda	db2e04a7c3	Update new sched query stats if there's already some (#1918) ... * Update new sched query stats if there's already some * IGNORE if the sched query is not present * Make tests less flaky	2021-09-02 17:39:08 -03:00
Tomas Touceda	681e7f18a0	Add default for agent options (#1917)	2021-09-02 15:59:48 -03:00
Tomas Touceda	15ce06263d	Dont fail config if agent opts is nil (#1898) ... * Handle agentopts nil better * Add changes file	2021-09-01 19:17:41 -03:00
Martin Angers	9a0871a2f1	Address issues related to Redis Cluster support (#1885) ... Closes #1847 .	2021-09-01 16:32:57 -04:00
Martin Angers	3755a58070	Add support for MySQL read replicas (#1786) ... Adds configuration options to use a read-only MySQL replica, and uses it instead of the primary for reads.	2021-09-01 15:50:52 -04:00
Tomas Touceda	79b5330a43	Do host user inserts one by one to not lock the whole database (#1884)	2021-09-01 11:39:23 -03:00
Benjamin Edwards	047dc5ac62	fix get hosts command to properly output json/yaml (#1774) ... * fix get hosts command to properly output json/yaml based on command line flag * add changes file * added tests for get hosts when specifiying host * added additional hosts to be returned in test cases * go fmt	2021-08-31 11:37:03 -04:00
Tomas Touceda	22267d8e0a	Group concat limit (#1860) ... * wiup * Decouple software select in two * Add changes file * Add missing error check	2021-08-30 16:07:24 -03:00
Tomas Touceda	2a17f9fe56	Remove hardcoded limits to the labels (#1857) ... * Remove hardcoded limits to the labels * Add changes file * Remove test that doens't apply anymore	2021-08-30 14:56:15 -03:00
Tomas Touceda	8b908f6506	Issue 1599 offline webhook (#1777) ... * wip * Add tests and finish implementation * Add proper default for periodicity, changes file, and documentation * Fix tests and add defaults also to new installs * EnableHostUsers should be true if undefined as well * In some cases, periodicity can be zero because of the migrations * Apply defaults when migrating appconfig * Fix lint * lint * Address review comments	2021-08-27 11:15:36 -03:00
Tomas Touceda	2fd725c81b	Export fields so the select will work with the struct (#1819)	2021-08-26 11:56:05 -03:00
Tomas Touceda	d6d53acda5	Issue 1600 fleetctl license expiration (#1800) ... * Show banner in fleet and fleetctl if license expired * Ignore if license is nil or tier is core * Address review comments	2021-08-26 10:28:53 -03:00
Zach Wasserman	5993018214	Fix live query timeout handling (#1759) ... Correct cleanup of the connections was verified with the pprof goroutine tool. Fixes #1751	2021-08-25 15:28:04 -07:00
Tomas Touceda	1f5094d97e	Make adding routes a bit simpler (#1771) ... * Make adding routes a bit simpler * Remove unused handle * Lint * More lint	2021-08-25 10:08:14 -03:00
Tomas Touceda	5fb5995b83	Remove unneeded interfaces (#1779) ... * Remove unneeded interfaces * Remove unused code	2021-08-24 18:49:56 -03:00
Tomas Touceda	a8642493ad	Add global policies (#1750) ... * Add global policies * Update documentation and add extra parameter to config * Fix failing tests * Store historic policy records * Address review comments And also remove other inmem references I saw by chance * Add documentation for get by id request * Add parameter doc * Move schema generation to a cmd instead of a test Otherwise it messes up running all tests sometimes depending on how parallel it does * Remove brain dump for another task * Make migration tests a separate beast * Make schema generation idempotent and move dbutils cmd to tools * Allow all filters and add counts to Policy * Add test for Policy	2021-08-24 17:24:52 -03:00
Martin Angers	e61cb7e0db	Configure golangci-lint for the whole repository ... Add a relatively minimal set of linters that raise safe and mostly un-opinionated issues with the code. It runs automatically on CI via a github action.	2021-08-24 13:35:03 -04:00
Martin Angers	a3714d2ed9	Add fleetctl debug connection command (#1706) ... Adds the `fleetctl debug connection` command to investigate connection issues to the fleet server. Closes #1579 .	2021-08-24 08:50:03 -04:00
Tomas Touceda	612fa05dd3	Log errors when osquery endpoints have issues (#1764)	2021-08-23 19:40:00 -03:00
Martin Angers	bfdc014cc1	Fix sql-related missing Close/Err calls (#1769)	2021-08-23 16:33:41 -04:00
Tomas Touceda	a1ceebf267	Make team schedule names more user friendly and hide them from host pack stats (#1737) ... * Make team schedule names more user friendly and hide them from host pack stats * Delete test for a different bug and add migration * Update name template	2021-08-23 14:23:47 -03:00
Tomas Touceda	66d864cc84	Remove extra logIPs from host auth mw (#1762) ... * Remove extra logIPs from host auth mw * Fix test	2021-08-23 14:23:27 -03:00
Tomas Touceda	26a08096fb	Use GROUP_CONCAT instead of JSON_ARRAYAGG (#1752) ... * Use GROUP_CONCAT instead of JSON_ARRAYAGG * Update server/datastore/mysql/software.go Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Martin Angers <martin.n.angers@gmail.com>	2021-08-23 14:12:30 -03:00
Tomas Touceda	b7a65e920a	Remove orphan host_software and software before adding fk constriants (#1760)	2021-08-23 13:42:43 -03:00
Tomas Touceda	ad18728ac3	Issue 1601 disk space (#1735) ... * wip * Add tests * Add migration * Fix tests and set platform for windows correctly	2021-08-20 14:57:37 -03:00
Tomas Touceda	605970c441	Add cascade deletes for host software (#1739) ... * Add cascade deletes for host software * Add changes file * The drop doesn't work on certain mysql * Fix error message	2021-08-20 14:29:00 -03:00
Tomas Touceda	c6c63ab12a	Refactor app config (POC, for now) (#1685)	2021-08-20 12:27:41 -03:00
Tomas Touceda	33791dbee8	Remove fk in scheduled_query_stats table and clean up orphan rows for it (#1720) ... * Remove fk in scheduled_query_stats table and clean up orphan rows for it * Improve test and fix bug with the cleanup	2021-08-18 18:30:48 -03:00
Tomas Touceda	70f728fe84	Make vulnerability processing more configurable (#1718) ... * Make vulnerability processing more configurable * Simplify leader selection	2021-08-18 17:25:14 -03:00
Tomas Touceda	402303bc5d	Add All Linux label (#1582) ... * Add All Linux label * Change name to Linux instead of All Linux to see if e2e likes it better * Revert "Change name to Linux instead of All Linux to see if e2e likes it better" This reverts commit 26b79f214e3b744e73270c544f89bb698575f6ea. * Fix all linux label insert	2021-08-18 15:55:48 -03:00
Tomas Touceda	53e25cf59d	Use tx if within a tx (#1668)	2021-08-13 14:59:31 -03:00
Tomas Touceda	512f5defce	Detect centos as a host platform more accurately (#1664) ... * Detect centos as a host platform more accurately * Add test for centos host labels	2021-08-13 13:22:09 -03:00
Tomas Touceda	5b8a607a81	Add support for MariaDB 10.5.4 and later (#1658) ... * Add support for MariaDB 10.5.4 and later * Fix tests * Add changes file * Use require.JSONEq instead of our own version of it	2021-08-13 11:16:19 -03:00
Tomas Touceda	96c0244c04	Fix redis locking receive (#1655) ... * Receive redis data with timeout to not hold the connection * Address review comments	2021-08-13 11:16:11 -03:00
Tomas Touceda	e8def4af00	Allow to refetch hosts if you are an observer/maintainer for the team for that host (#1649)	2021-08-11 19:01:37 -03:00
Zach Wasserman	544652e833	Clarify label responses and API docs (#1633) ... - Include platform and description even when empty in API responses. - Update docs to indicate immutability.	2021-08-11 14:57:05 -07:00
Tomas Touceda	3d8a766ca1	Make receive calls to redis conn thread safe (#1641) ... * Make receive calls to redis conn thread safe Also removes REDIS_TEST env var. Redis is lightweight and fast, no need to skip these tests. * No need to increase the wait	2021-08-11 17:34:35 -03:00
Tomas Touceda	ae2bd137b1	Issue 1632 software inventory config (#1636) ... * Add config option for software inventory * Add documentation for the new config	2021-08-11 15:57:53 -03:00
Tomas Touceda	11dff7ca92	Issue 1588 allow disabling host users (#1611) ... * Allow users to disable host_users * Add missing files * Address review comments * Fix tests * Check additional queries for nil, not the whole hostsettings * Finally fix tests	2021-08-11 14:56:11 -03:00
Tomas Touceda	c781058617	Fix build, add missing tests for cpe translations (#1631) ... * Fix build, add missing tests for cpe translations Also dont fail alltogether if there's one issue translating CPEs, log it and continue * Make it once every hour again * Use MATCH but escape strings	2021-08-11 14:52:09 -03:00
Zach Wasserman	740ba7bc25	Remove duplicate registration of results handler (#1605)	2021-08-11 09:39:47 -07:00