Commit Graph

394 Commits

Author SHA1 Message Date
gillespi314
6a3d9959fc
Refactor vulnerabilities cron to scheduler package (#7650) 2022-09-16 10:08:51 -05:00
Juan Fernandez
765031d962
Bug 6717: Duplicate calls on vulnerability third party integrations. (#7698)
If the same vulnerability was present in more than one software, then duplicated third party integration calls were being issued.
2022-09-13 10:41:52 -04:00
Juan Fernandez
b60d535d4a
Feature 7084: Add new EE endpoint for Fleet Desktop (#7530)
Added new EE endpoint, that is meant to be used by Fleet Desktop only. The new endpoint will return the number of failed policies.
2022-09-12 15:37:38 -04:00
Juan Fernandez
98e4d6e0bc
Feature 7394: Use MSRC parser to generate security bulletin artifacts (#7491)
Generate security artifacts using the MSRC parser.
2022-09-12 15:17:13 -04:00
Juan Fernandez
b6d084deec
Feature 7559: Include 3rd party severity scores in the vulnerability web-hook payload (#7581)
Premium users should receive vulnerability scores as part of the web-hook payload.
2022-09-06 15:12:17 -04:00
Michal Nicpon
0709d1bc5c
improve vuln cpe matching on macos (#6985)
* add cpe translations
* fix matching on target_sw
2022-09-01 10:02:07 -06:00
gillespi314
8157359f84
Ingest Windows host display version for hosts table (#7403) 2022-08-30 10:16:48 -05:00
Roberto Dip
eeefe2fab9
enable controlled rollout of features by teams (#7408) 2022-08-30 08:13:09 -03:00
Martin Angers
f602ea3446
Provide Munki issues for MacOS hosts (#7280) 2022-08-29 14:40:16 -04:00
Roberto Dip
f9bb0b3dc0
adjust osquery_perf to respect config settings (#7418)
users, software_macos and software_linux queries can be optionally disabled in the fleet server via the enable_host_users and enable_software_inventory config settings, however osquery_perf is always running them.

this adjusts the code accordingly to only execute them if the server asks.
2022-08-29 13:34:40 -03:00
Juan Fernandez
0572c621d5
Fix osquery-perf Ubuntu software path issue (#7417)
Fixed issue with software fixture path used in osquery-perf
2022-08-26 15:43:06 -04:00
Juan Fernandez
3048a07fd1
Feature 7076: Ingest installed windows updates (#7138)
* Ingest installed Windows updates and store them in the windows_updates table. 
* Added config option for enabling/disabling Windows update ingestion and Windows OS vuln. detection.
2022-08-26 14:55:03 -04:00
Roberto Dip
9a1d2066bb
deprecate host_settings in favor of features (#7358)
Related to #7312, this makes use of the changes introduced in #7353 to rename host_settings to features while keeping backwards compatibility.
2022-08-25 13:41:50 -03:00
Martin Angers
1ed6512c5b
Implement retries when building MSI on Apple M1 architecture (#7367) 2022-08-24 14:52:32 -04:00
Martin Angers
dfd7d3f244
Add activity types for applied team spec and edited agent options (#7355) 2022-08-24 08:32:45 -04:00
Juan Fernandez
7e1e69c9bc
Fixed flaky vulnerability CRON tests (#7311)
* Fixed flaky vuln cron tests

* Fixed typo

* Remove sub-test
2022-08-23 08:22:08 -03:00
gillespi314
7fb109e6b3
Handle errors for duplicate os records (#7294) 2022-08-22 14:34:00 -05:00
Zachary Winnerman
2103de275a
improve error handling in vulnerabilities cron (#7276)
* improve error handling in vulnerabilities cron

* fix tests

* Use errHandler and go mod tidy

* Add dep that got removed by mod tidy

* add dsl to tools

* Add changes file

Co-authored-by: Michal Nicpon <michal@fleetdm.com>
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-08-18 16:53:58 -03:00
gillespi314
90f57f4849
Enhance API endpoints with host operating systems info (#7154) 2022-08-12 14:23:25 -05:00
Zach Wasserman
acf60189ff
Add message for fleetctl when prepare commands are used (#7152) 2022-08-11 12:26:52 -07:00
Martin Angers
c8cdddf0ea
Update /macadmins endpoints to include MDM name and aggregate count (#7137) 2022-08-10 15:15:01 -04:00
Roberto Dip
fc8c15c0d1
add application config setting to enable JIT provisioning (#7140)
As part of #7053, this adds a config setting to enable JIT provisioning.
2022-08-10 15:15:35 -03:00
gillespi314
e2194be61c
Add schedule package and refactor cron jobs for cleanups, aggregations, and usage statistics (#6618) 2022-08-10 11:00:56 -05:00
gillespi314
3a88afaf98
Add new database tables and queries for host operating systems (#6920) 2022-08-09 13:34:41 -05:00
Lucas Manuel Rodriguez
6dcff28be0
Move specs parsing functionality to a new pkg/spec package (#7050) 2022-08-05 19:07:32 -03:00
Martin Angers
012869f971
Fleet Premium: de-anonymize usage stats (#7013) 2022-08-03 14:44:34 -04:00
Martin Angers
c1d38598e2
Prevent removing team enroll secrets when applying team specs without new secrets (#6890) 2022-08-02 09:51:03 -04:00
Gabriel Hernandez
f61a17bd9b
add google analytics to sandbox instances (#6941)
* add google analytics to sandbox instances

* Add serverType variable to frontend handler

* update version of html-webpack-plugin

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2022-08-01 11:27:12 +01:00
Roberto Dip
780db937de
pass a pointer to config instead of the whole config to goroutines (#6823) 2022-07-26 12:27:30 -03:00
Roberto Dip
d63f56f8c0
add support for notarization in fleetdm/fleetctl images (#6818)
#6674
2022-07-25 20:06:10 -03:00
gillespi314
cc34585a47
Add statistics for number of hosts that have not submitted results for distributed queries (#6495) 2022-07-20 20:54:10 -05:00
Michal Nicpon
d4be5ad2a1
add upgrade tests (#6596)
* add upgrade tests

* fix lint issues

go.mod

* remove req.cnf

* revert unrelated changes

* make version configurable in test

* fix golangci-lint ruleguard issue

Related to https://github.com/go-critic/go-critic/issues/1152
Need to have github.com/quasilyte/go-ruleguard/dsl

* fix lint issues

* fix

* clean up docker-compose.yml

* fix http request

* add readme

* fix lint issues

* address feedback

* fix

* add platform

* address feedback

* run go fmt
2022-07-19 15:11:51 -06:00
Eng Zer Jun
1ab171faf3
test: use T.Setenv to set env vars in tests (#6714)
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-18 14:22:28 -03:00
Roberto Dip
69f8f2a73b
add API endpoints to retrieve pre-built installers (#6672)
Rel: #6365, this adds a new endpoint to check and download pre-built installers.
2022-07-18 13:44:30 -03:00
Roberto Dip
86366b2ed8
add a new S3 datastore to retrieve pre-built packages (#6631)
Related to #6365, this extends the datastore/s3 package to retrieve installers from S3 according to the conventions listed in the parent issue. This also includes:

- A minor refactor to decouple Carves-related functionality from the core S3 functionality
- Set-up to run tests using minio (only enabled via the FILE_STORAGE_TEST env flag)
2022-07-14 14:14:24 -03:00
Roberto Dip
08cd206de7
only send vuln configs to goroutine (#6650)
In https://github.com/fleetdm/fleet/pull/6630 I added a new config for packaging, but when I started the server I got:

```
~/fleet $ ./build/fleet serve --dev --dev_license
level=info ts=2022-07-13T21:36:06.055998Z component=redis mode=standalone
fatal error: newproc: function arguments too large for new goroutine

runtime stack:
runtime.throw({0x103d85259, 0x37})
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/panic.go:1198 +0x54
runtime.newproc1(0x104569a30, 0x14000ffda28, 0x7f8, 0x140000001a0, 0x103bc85a4)
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/proc.go:4299 +0x4d0
runtime.newproc.func1()
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/proc.go:4255 +0x4c
runtime.systemstack()
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/asm_arm64.s:230 +0x6c

goroutine 1 [running]:
runtime.systemstack_switch()
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/asm_arm64.s:187 +0x8 fp=0x14000ffd9c0 sp=0x14000ffd9b0 pc=0x102b60958
runtime.newproc(0x7f8, 0x104569a30)
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/proc.go:4254 +0x54 fp=0x14000ffda10 sp=0x14000ffd9c0 pc=0x102b38034
main.runCrons({0x1045cd500, 0x14000262990}, {0x104611a38, 0x14000262930}, 0x14000298190, {0x1045a0480, 0x140009c9f20}, {{{0x103d1adc4, 0x3}, {0x103d2b188, ...}, ...}, ...}, ...)
	/Users/roperzh/fleet/cmd/fleet/serve.go:694 +0x2c4 fp=0x14000ffe260 sp=0x14000ffda10 pc=0x103bc85a4
main.createServeCmd.func1(0x1400027ca00, {0x140000bcb40, 0x0, 0x2})
```

With my local changes, `serve.go:694` is this line:

7559988000/cmd/fleet/serve.go (L685-L686)

After passing only a subset of the config the issue was solved.
2022-07-14 08:04:29 -03:00
Roberto Dip
6faee84f57
allow to configure a default global enroll secret (#6609)
Related to #6365 this adds a new config to set a global enroll token that will be used by the server.
2022-07-12 19:12:10 -03:00
Roberto Dip
f7dd8c86cd
implement a docker image to package orbit natively in Linux (#6504)
Related to #6364 and #6363, this:

- Adds a new Docker image, `fleetdm/fleetctl` equipped with all necessary dependencies to build Fleet-osquery binaries for all platforms
- Modifies the package generation logic to special case this scenario via an environment variable `FLEETCTL_NATIVE_TOOLING`
- Adds a new GitHub workflow to test this

There are more details in the README, but part of the special-casing logic is in place to output the binaries to a folder named `build` when they are run with `FLEETCTL_NATIVE_TOOLING`, this is so we can persist the binary generated by the docker container via a bind mount:

```bash
docker run -v "$(pwd):/build" fleetdm/fleetctl package --type=msi
```

To test this changeset, I have generated packages for all platforms, both via the new Docker image and via the classic `fleetctl package`.
2022-07-11 09:49:13 -03:00
Lucas Manuel Rodriguez
a336ed61e5
Add gotestfmt to improve test output and fix flaky tests (#6528) 2022-07-11 08:12:33 -03:00
Luke Heath
7408dc0aae
Update dev license (#6476) 2022-07-01 10:17:19 -07:00
Zach Wasserman
db22f68c88
Separate health checks for MySQL and Redis (#6468)
This required a bit of refactoring of some mocking due to how the code
generation does not handle having the same function in different types.
2022-07-01 08:08:03 -03:00
Zach Wasserman
974c8e008d
Improve logging for fleetctl with permissions errors (#6460) 2022-06-30 13:24:19 -07:00
Roberto Dip
b2d07e56b5
report error analytics (#6341)
Related to #5898, this reports an anonymized summary of errors stored in Redis into the analytics payload.

For each error stored, this includes:

- A `count` attribute with the number of occurrences of the error
- A `loc` attribute with the 3 topmost lines in the stack trace. Note that stack traces only contain package name + line number (example: github.com/fleetdm/fleet/server.go:12

This also includes a minor refactor around error types.
2022-06-28 16:31:14 -03:00
Martin Angers
539be8ee09
Add battery info in host details response (#6394) 2022-06-28 14:11:49 -04:00
Juan Fernandez
79bf51b03c
Feature 6242: Use oval to detect vulnerabilities on Fedora hosts (#6330)
* Feature 6242: Scan Fedora hosts using OVAL definitions
2022-06-24 11:02:51 -04:00
Juan Fernandez
9d01ba33c6
Feature 6096: Scan RHEL/CentOS hosts using OVAL definitions (#6241)
Extended the OVAL parser/analyzer so that we can scan RHEL based systems.
2022-06-23 16:44:45 -04:00
Martin Angers
8f66a50f2b
Add hosts enrolled by operating system to anonymous usage stats (#6329) 2022-06-22 16:38:11 -04:00
Roberto Dip
2948e112f6
ensure software_host_counts is cleaned when software is deleted (#6270)
Related to #5982, this ensures we clean up software_host_counts rows referencing software that is not longer present in the software table.
2022-06-22 17:35:53 -03:00
Martin Angers
dc9e158153
Add weekly active users count in anonymous usage stats (#6317) 2022-06-22 15:20:57 -04:00
Ahmed Elshaer
a9f48ff561
Create Bulk Users from CSV (#3372)
* Create Bulk Users

* WIP: Adding a test for bulk user import

* adding a user bulk create test

* Fixing description, removing password required, and adding more test cases

* Fixing description, removing password required, and adding more test cases

* Fixed all comments and added Random Password Generator

* returning an error in generateRandomPassword

* Using 2 loops to create user list and then create the actual users

* Adding a bulk user delete

* fixing a mistake in temp csv

* fixed lints and removed yamlFlag
2022-06-22 13:34:58 -03:00