Commit Graph

79 Commits

Author SHA1 Message Date
Roberto Dip
eeefe2fab9
enable controlled rollout of features by teams (#7408) 2022-08-30 08:13:09 -03:00
Martin Angers
f602ea3446
Provide Munki issues for MacOS hosts (#7280) 2022-08-29 14:40:16 -04:00
Juan Fernandez
3048a07fd1
Feature 7076: Ingest installed windows updates (#7138)
* Ingest installed Windows updates and store them in the windows_updates table. 
* Added config option for enabling/disabling Windows update ingestion and Windows OS vuln. detection.
2022-08-26 14:55:03 -04:00
Juan Fernandez
0fab2fe0eb
Feature 6538: Remove cpe_id from the software_cve table (#6587)
* Don't insert dummy values for non-match CPEs, exclude OVAL platforms when getting software iterator.
* Removed CPEID from SoftwareVulnerability type and software_cve.cpe_id column.
* Removed GeneratedCPEID property
2022-08-24 13:10:58 -04:00
gillespi314
7fb109e6b3
Handle errors for duplicate os records (#7294) 2022-08-22 14:34:00 -05:00
gillespi314
90f57f4849
Enhance API endpoints with host operating systems info (#7154) 2022-08-12 14:23:25 -05:00
Martin Angers
c8cdddf0ea
Update /macadmins endpoints to include MDM name and aggregate count (#7137) 2022-08-10 15:15:01 -04:00
Martin Angers
9755eb2e27
Support async saving of scheduled query statistics (#7012) 2022-08-10 10:01:05 -04:00
gillespi314
3a88afaf98
Add new database tables and queries for host operating systems (#6920) 2022-08-09 13:34:41 -05:00
Juan Fernandez
966bfbf85e
Feature 6487: Deprecate cpe_id from software_cve table (#6562)
Part 2/3 of the removal of the cpe_id column from the software_cve table in favor of using the newly added software_id coumn.
2022-08-04 09:24:44 -04:00
gillespi314
cc34585a47
Add statistics for number of hosts that have not submitted results for distributed queries (#6495) 2022-07-20 20:54:10 -05:00
Zach Wasserman
db22f68c88
Separate health checks for MySQL and Redis (#6468)
This required a bit of refactoring of some mocking due to how the code
generation does not handle having the same function in different types.
2022-07-01 08:08:03 -03:00
Martin Angers
539be8ee09
Add battery info in host details response (#6394) 2022-06-28 14:11:49 -04:00
Juan Fernandez
9d01ba33c6
Feature 6096: Scan RHEL/CentOS hosts using OVAL definitions (#6241)
Extended the OVAL parser/analyzer so that we can scan RHEL based systems.
2022-06-23 16:44:45 -04:00
Roberto Dip
2948e112f6
ensure software_host_counts is cleaned when software is deleted (#6270)
Related to #5982, this ensures we clean up software_host_counts rows referencing software that is not longer present in the software table.
2022-06-22 17:35:53 -03:00
Martin Angers
81f0e0ccfa
Track active hosts count and enforce limit (#6099) 2022-06-13 16:29:32 -04:00
Martin Angers
7f9bb6431e
Update team integrations to reference global integrations (part of failing policies automation support) (#6156) 2022-06-13 10:04:47 -04:00
gillespi314
44f902218c
Improve live query UX (#5749) 2022-06-10 13:29:45 -05:00
Juan Fernandez
ef73039559
Improve vulnerability detection for Ubuntu (#6102)
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts

To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
2022-06-07 21:09:47 -04:00
Michal Nicpon
30a864c8d2
Sync CVE scores periodically (#5838) 2022-06-01 10:06:57 -06:00
gillespi314
b4ffec10db
Remove skipLoadingExtras and includeCVEScores from GetHost and HostByIdentifier methods (#5874) 2022-05-25 11:30:03 -05:00
Michal Nicpon
983f9e59b3
Include CVE scores when listing software (#5673) 2022-05-20 10:58:40 -06:00
gillespi314
d172128183
Add label information to host summary response (#5573) 2022-05-10 10:32:55 -05:00
Lucas Manuel Rodriguez
407d8b527e
Remove unused Datastore.SaveHost and unused Modified fields (#5245) 2022-04-26 18:33:29 -03:00
Martin Angers
7187f1adac
Queue jobs for Jira integration when enabled and new vulnerabilities are found. (#4975) 2022-04-11 16:42:16 -04:00
Martin Angers
f28dc10a51
Merge jira-integration branch to main (#4949) 2022-04-05 12:56:15 -04:00
Lucas Manuel Rodriguez
57816592ba
Add read replica testing helpers and fix non-sso login bug (#4908)
not set on the INSERT.
- OUT: Only sets the ID on the passed session and returns it. (`CreatedAt`, `AccessedAt`, are not set.)

New version:

```go
func (ds *Datastore) NewSession(ctx context.Context, userID uint, sessionKey string) (*fleet.Session, error) {
	sqlStatement := `
		INSERT INTO sessions (
			user_id,
			` + "`key`" + `
		)
		VALUES(?,?)
	`
	result, err := ds.writer.ExecContext(ctx, sqlStatement, userID, sessionKey)
	if err != nil {
		return nil, ctxerr.Wrap(ctx, err, "inserting session")
	}

	id, _ := result.LastInsertId() // cannot fail with the mysql driver
	return ds.sessionByID(ctx, ds.writer, uint(id))
}
```

- IN: Define arguments that are truly used when creating a session.
- OUT: Load and return the fleet.Session struct with all values set (using the `ds.writer` to support read replicas correctly).

PS: The new `NewSession` version mimics what we already do with other entities, like policies (`Datastore.NewGlobalPolicy`).
2022-04-04 16:52:05 -07:00
Michal Nicpon
a6902cc083
Add os versions endpoint (#4749) 2022-03-28 09:15:45 -06:00
Lucas Manuel Rodriguez
d2ba34c8fc
Add distributed discovery query support for detail queries, add orbit… (#4597)
* Add distributed discovery query support for detail queries, add orbit_info ingestion

* Amend changes file
2022-03-15 16:51:00 -03:00
Martin Angers
a1c67547b3
Add new endpoints to retrieve device information by orbit identifier (#4531) 2022-03-09 16:13:56 -05:00
Martin Angers
c8bc026d6f
Migrate special-case endpoints to new pattern (#4511) 2022-03-08 11:27:38 -05:00
Martin Angers
18e7c8e236
Implement cron cleanup job of policy membership when policy platform is updated (#4331) 2022-02-23 16:10:37 -05:00
gillespi314
089f49cc5d
Add new activity types for policy-related activities (#4043)
* Add new activities for policy create, delete, and modify, and apply policy spec
* Add PoliciesByID ds method; refactor delete svcs
2022-02-16 09:33:56 -06:00
Lucas Manuel Rodriguez
be72dc356c
Add CentOS parsing+post-processing to reduce false positives in vulnerability processing (#4037)
* Add CentOS parsing and post-processing in fleet

* Add tests and amend SyncCPEDatabase

* Add test for centosPostProcessing

* Changes from PR comments

* Amend software test

* Fix sync test

* Add index to source and vendor

* Use os.MkdirTemp

* Rearrange migrations

* Regenerate test schema

* Add support for testing migrations (#4112)

* Add support for testing migrations

* Rename migration in tests

* Changes suggested in PR

* Go mod tidy
2022-02-14 15:13:44 -03:00
Tomas Touceda
35eac78aed
Add CountsUpdatedAt for munki/mdm status (#4045)
* Add CountsUpdatedAt for munki/mdm status

* Update doc
2022-02-07 14:53:33 -03:00
Tomas Touceda
cf529e70cf
Issue 3173 debug status processlist (#4009)
* Add innodb status and process list

* Make json output a bit prettier

* Add changes file

* fix lint issues
2022-02-03 14:56:22 -03:00
Martin Angers
6e2ba62744
Trigger webhooks for recently published vulnerabilities (#3941) 2022-02-02 16:34:37 -05:00
Tomas Touceda
ffabf803a3
Aggregate munki and mdm data (#3886)
* Aggregate munki and mdm data

* Update doc

* Use reader to read

* Reader to read

* Address review comments
2022-01-26 17:55:07 -03:00
Martin Angers
9a0f749641
Add hosts_count field to "list software" endpoint (#3873) 2022-01-26 09:47:56 -05:00
Tomas Touceda
f02bef6f2c
Add platform filter to host_summary (#3845)
* Add platform filter to host_summary

* Add documentation

* Actually forward the platform param down the chain

* Update mock

* Update mock
2022-01-24 14:49:21 -03:00
Martin Angers
f19e676e62
Refactor async host processing to avoid redis SCAN keys (for policies) (#3657) 2022-01-18 09:56:43 -05:00
Lucas Manuel Rodriguez
371c533bfc
Improved Datastore usage of osquery hosts requests (#3601)
* WIP

* Amend tests

* Do not load aggregated stats for packs

* Add option to host lite

* Fix remaining TODOs

* Fix osquery_utils tests

* Fix SQL

* Fix SQL (bis)

* Restore AuthenticateHost to load once

* Code improvements and re-add deferred host save

* More fixes to the PR

* Wrap users table update on tx

* Add caching to ListPacksForHost and ListScheduledQueriesInPack

* Remove SaveHostSoftware (replaced by UpdateHostSoftware)

* Add unit tests for new functionality

* Add changes file

* Fix scheduled queries test
2022-01-17 22:52:09 -03:00
gillespi314
dc8eacc95c
Add AvailableTeams to loginResponse and getUserResponse (#3585) 2022-01-13 13:57:44 -06:00
Tomas Touceda
4bed4757fb
Remove cleanups and fk for host ids (#3607)
* Remove cleanups and fk for host ids

* Readd missing things to the schema

* Remove unused

* Add changes file and fix some error messages

* Fix test

* Use tx instead of plain writer

* Other fixes

* More not found test fixes

* Go back to getcontext
2022-01-12 14:07:51 -03:00
Lucas Manuel Rodriguez
787944482b
Global policies automation webhooks (#3378)
* Add webhook to app config

* Add redis failing policies set and webhook

* Add basic webhook test

* Store hostname in redis

* Global policy deletion to remove policy ID from set and config

* Also process new passing policies

* Fix unit test

* Sort hosts

* Add more tests

* Add ListSets to the failing policies interface

* Fix server URL and garbage collect on the triggering side

* Do not use Redis SCAN

* Fix Redis operation order

* Add API changes to doc

* Add comments

* Add more tests

* Fix tests

* Add tests for config update upon deletion of policies

* Run make dump-test-schema

* Ignore policies that failed to run

* Add proper unit tests to trigger logic

* Fix comments

* WIP

* Add tests to service_osquerty_test.go

* Use SSCAN for listing hosts instead of SMEMBERS

* Add failing policies to docs/01-Using-Fleet/configuration-files/README.md

* Remove skip

* Fix PR comments
2021-12-23 18:26:55 -03:00
Martin Angers
1e2059585a
Add support for google chrome profiles (#3423) 2021-12-21 15:36:19 -05:00
Tomas Touceda
39e71c6d77
Add mdm and munki (#3406)
* Draft for mdm and munki as direct ingest funcs

* Expose mdm/munki over API

* Add test for service and fix bug

* Update queries test

* Fix lint

* Address review comments
2021-12-21 09:37:58 -03:00
gillespi314
6ee6dba4a4
Update anonymous usage statistics to record Fleet feature adoption (#3056)
* Collect additional anonymous usage statistics

* Update fleetdm.com and docs for usage stats

* Fix lint-go errors

* Fix failing test

* update webhook inputs

* Update receive-usage-analytics.js

* fix failing test

* Update statistics_test.go

Co-authored-by: eashaw <eashaw@sailsjs.com>
2021-12-06 14:39:00 -06:00
Martin Angers
b84ef79bf4
Add the fleetctl debug {errors,db-locks} sub-commands (#3168) 2021-12-06 09:26:01 -05:00
Lucas Manuel Rodriguez
b9a408704e
Add platforms field to policies (#3181)
* Add platforms field to policies

* Fix fleetctl tests

* PR review changes

* Add missing tests

* Add changes for ListPoliciesForHost
2021-12-03 15:33:33 -03:00