Commit Graph

105 Commits

Author SHA1 Message Date
Benjamin Edwards
f3b0e4da72
add configuration parameters for filesystem logging file rotation (#10048) 2023-02-24 07:44:56 -05:00
Benjamin Edwards
0db432f7e3
vuln processing distinct command (#9813)
closes https://github.com/fleetdm/fleet/issues/3723

Add new vuln processing command, configs, and documentation on how to utilize.
2023-02-17 10:00:57 -05:00
Roberto Dip
046401d190
Ingest file vault recovery keys in macOS (#9712)
Related + details at https://github.com/fleetdm/fleet/issues/8708
2023-02-08 11:49:42 -03:00
Roberto Dip
2447a371b0
use nanomdm's multi service + integration tests and fixes (#9269)
**Use nano's multi service**

This allows us to integrate more seamlessly with nano and to run our
custom MDM logic _after_ the request was handled by nano, which gives us
more flexibility (for example: now we can issue commands after a
TokenUpdate message)

From nano's code:

> MultiService executes multiple services for the same service calls.
> The first service returns values or errors to the caller. We give the
> first service a chance to alter any 'core' request data (say, the
> Enrollment ID) by waiting for it to finish then we run the remaining
> services' calls in parallel.

**Integration tests + fixes**

- Move some of the service logic from `cmd/` to `server/service`
- Add integration tests for the MDM enrollment flow, including SCEP
authentication.
- Fixed a bug that set `host_mdm.mdm_id = 0`  during MDM enrollment due
  to how MySQL reports the last insert id when `ON DUPLICATE KEY` is
  used.
- Completely remove the host row from `host_mdm` when a device is
  unenrolled from MDM to match the behavior of how we ingest MDM data
  from osquery

Related to https://github.com/fleetdm/fleet/issues/8708,
https://github.com/fleetdm/fleet/issues/9034
2023-01-16 17:06:30 -03:00
Martin Angers
e360013dc3
Implement API endpoint for MDM manual enrollment profile download (#9232) 2023-01-16 10:22:12 -05:00
Lucas Manuel Rodriguez
e1bbcfcfda
Generate audit logs for activities (#9001)
* Generate audit logs for activities

* Fix config tests

* Fix TestGetConfig/IncludeServerConfig

* Fix use of AddAttributes in results only

* Stream activities asynchronously

* Fix index and add logging

* Revert change

* Documentation fixes
2022-12-23 19:04:13 -03:00
Stephan Miehe
b6a409fa3b
Support redis username (#9018) 2022-12-14 08:53:38 -05:00
Martin Angers
d4a3730610
Add new configuration options for Apple Business Manager (#8886) 2022-12-06 15:50:56 -05:00
gillespi314
6fb3a87ae9
Enable errcheck linter for golangci-lint (#8899) 2022-12-05 16:50:49 -06:00
Martin Angers
f5fc316ad3
Add options for configuring APNS and SCEP keys/certs in Fleet server (#8730) 2022-12-05 10:22:56 -05:00
Roberto Dip
d0cde1aaaa
Use the same HTTP server config as the production server in tests (#8254)
* Use the same HTTP server config as the production server in tests

This abstracts the default config we use to run the server into a
function so it can be used in tests to run an HTTP server using the same
configuration.

Additionally, this fixes a data race in tests, as an HTTP server
configuration can't be changed once you call `server.Start()`[1]

[1]:
https://cs.opensource.google/go/go/+/refs/tags/go1.19.2:src/net/http/httptest/server.go;l=40;drc=19309779ac5e2f5a2fd3cbb34421dafb2855ac21
2022-10-19 07:42:21 -03:00
Lucas Manuel Rodriguez
9191f4ce66
Add Apple MDM functionality (#7940)
* WIP

* Adding DEP functionality to Fleet

* Better organize additional MDM code

* Add cmdr.py and amend API paths

* Fix lint

* Add demo file

* Fix demo.md

* go mod tidy

* Add munki setup to Fleet

* Add diagram to demo.md

* Add fixes

* Update TODOs and demo.md

* Fix cmdr.py and add TODO

* Add endpoints to demo.md

* Add more Munki PoC/demo stuff

* WIP

* Remove proposals from PoC

* Replace prepare commands with fleetctl commands

* Update demo.md with current state

* Remove config field

* Amend demo

* Remove Munki setup from MVP-Dogfood

* Update demo.md

* Add apple mdm commands (#7769)

* fleetctl enqueue mdm command

* fix deps

* Fix build

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>

* Add command to upload installers

* go mod tidy

* fix subcommands help

There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.

* Add support for installing apps

* Add a way to list enrolled devices

* Add dep listing

* Rearrange endpoints

* Move DEP routine to schedule

* Define paths globally

* Add a way to list enrollments and installers

* Parse device-ids as comma-separated string

* Remove unused types

* Add simple commands and nest under enqueue-command

* Fix simple commands

* Add help to enqueue-command

* merge apple_mdm database

* Fix commands

* update nanomdm

* Split nanomdm and nanodep schemas

* Set 512 MB in memory for upload

* Remove empty file

* Amend profile

* Add sample commands

* Add delete installers and fix bug in DEP profile assigning

* Add dogfood.md deployment guide

* Update schema.sql

* Dump schema with MySQL 5

* Set default value for authenticate_at

* add tokens to enrollment profiles

When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.

- Rename enrollments to enrollment profiles. Enrollments is used by nano
  to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=

* remove mdm apple server url

* update docs

* make dump-test-schema

* Update nanomdm with missing prefix table

* Add docs and simplify changes

* Add changes file

* Add method docs

* Fix compile and revert prepare.go changes

* Revert migration status check change

* Amend comments

* Add more docs

* Clarify storage of installers

* Remove TODO

* Remove unused

* update dogfood.md

* remove cmdr.py

* Add authorization tests

* Add TODO comment

* use kitlog for nano logging

* Add yaml tags

* Remove unused flag

* Remove changes file

* Only run DEP routine if MDM is enabled

* Add docs to all new exported types

* Add docs

* more nano logging changes

* Fix unintentional removal

* more nano logging changes

* Fix compile test

* Use string for configs and fix config test

* Add docs and amend changes

* revert changes to basicAuthHandler

* remove exported BasicAuthHandler

* rename rego authz type

* Add more information to dep list

* add db tag

* update deps

* Fix schema

* Remove unimplemented

Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 19:53:54 -03:00
Michal Nicpon
0709d1bc5c
improve vuln cpe matching on macos (#6985)
* add cpe translations
* fix matching on target_sw
2022-09-01 10:02:07 -06:00
Juan Fernandez
3048a07fd1
Feature 7076: Ingest installed windows updates (#7138)
* Ingest installed Windows updates and store them in the windows_updates table. 
* Added config option for enabling/disabling Windows update ingestion and Windows OS vuln. detection.
2022-08-26 14:55:03 -04:00
Martin Angers
9755eb2e27
Support async saving of scheduled query statistics (#7012) 2022-08-10 10:01:05 -04:00
Roberto Dip
90b723e45a
consolidate sandbox env flags (#6917)
Related to #6894, this entirely replaces FLEET_DEMO with the server config added in #6597

As part of this, I also implemented a small refactor to the integration test suite to allow setting a custom config when the server is initialized.
2022-07-27 16:47:39 -03:00
Michal Nicpon
e3e17f3389
Fix double quote usage in sql query (#6707) 2022-07-20 10:10:03 -06:00
Eng Zer Jun
1ab171faf3
test: use T.Setenv to set env vars in tests (#6714)
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-18 14:22:28 -03:00
Roberto Dip
69f8f2a73b
add API endpoints to retrieve pre-built installers (#6672)
Rel: #6365, this adds a new endpoint to check and download pre-built installers.
2022-07-18 13:44:30 -03:00
Roberto Dip
7e68ee2253
add configuration to retrieve installers from S3 (#6630)
Related to #6365, this adds relevant configuration to the packaging key to retrieve installers from S3.
2022-07-13 09:34:25 -03:00
Roberto Dip
6faee84f57
allow to configure a default global enroll secret (#6609)
Related to #6365 this adds a new config to set a global enroll token that will be used by the server.
2022-07-12 19:12:10 -03:00
Tomas Touceda
7f8099db57
Add sandbox fleet serve config (#6619)
* Add sandbox fleet serve config

* Update docs

* Maybe fix lint issues

* Add lint dep
2022-07-12 18:21:15 -03:00
Martin Angers
81f0e0ccfa
Track active hosts count and enforce limit (#6099) 2022-06-13 16:29:32 -04:00
Desmi-Dizney
f648949ebe
Editor pass - Support per-task configuration for async host processing configuration (#5810)
* Editor pass - Support per-task configuration for async host processing configuration #

Editor pass for:
-  https://github.com/fleetdm/fleet/pull/5700

* Update Configuration.md

* Update config.go
2022-05-20 12:07:32 -05:00
Martin Angers
e6b90ca8b9
Support per-task configuration for async host processing configuration (#5700) 2022-05-16 09:44:50 -04:00
Martin Angers
0d0c17e0c6
Add osquery.min_software_last_opened_at_diff configuration option (#5394) 2022-04-27 09:47:09 -04:00
Martin Angers
36702ede8d
Make recent vulnerabilities max age configurable. (#5081) 2022-04-12 14:48:15 -04:00
Lucas Manuel Rodriguez
60b7425bef
Add http basic auth to /metrics (#4974)
* Add http basic auth to /metrics

* Fixes after testing applying of a --config sample.yml

* Add unit test
2022-04-07 09:40:53 -03:00
Michal Nicpon
7b671ac2a3
Add team failing policies webhook (#4633)
* add config to teams
* update api docs
* update tests
2022-03-21 13:16:47 -06:00
Benjamin Edwards
74bb559645
Add public ip to hosts & derive geolocation when rendering host (#4652)
* geoip wip
* return nil if ip is empty string or if ParseIP returns nil
* add ui component to render geolocation if available, address PR feedback
* render public ip if available
* add changes file, document geoip in deployment guide
* update rest-api docs
2022-03-21 12:29:52 -04:00
Benjamin Edwards
f8cf6ea91c
make context type value header configurable (#4441)
* make context type value header configurable
* populate config
2022-03-09 17:22:29 -05:00
Michal Nicpon
1aa7b96837
change session duration to 5 days (#4480) 2022-03-07 15:37:54 -07:00
Zachary Winnerman
c5c72ed713
Add apm for testing apm (#4053)
* Add apm for testing apm

* Testing opentracing

* testing

* Testing

* go fmt

* Add config switch for tracing.

* fixup

* Update cmd/fleet/serve.go

Co-authored-by: Tomas Touceda <chiiph@gmail.com>

* Add support for both elasticapm and opentelemetry

* Fix driver stuff and config options

* Fixup

* fixup

* Add changes file

* Add config for sql driver

* fixup

* Add doc to exported field

* testing

* fixup

* fixup

* Testing again

* fixup

* testing

* Undo

Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-02-15 12:42:22 -05:00
Tomas Touceda
11887f87f7
Add enable scheduled query stats to fleet config (#4066)
* Add enable scheduled query stats to fleet config as well

* Add documentation

* Revert "Allow disabling scheduled query stats via app config (#4049)"

This reverts commit f98fd4d331.

* Add changes file

* Update ref

* Add missing docs
2022-02-09 08:20:29 -03:00
Martin Angers
a6f3f02a85
Cleanup unused software after calculating the count of hosts (#3887) 2022-01-26 11:32:42 -05:00
Tomas Touceda
9d572309ae
Add sentry (#3669)
* Add sentry

* Fix gosum

* More gosum fixes

* Add missing def for config

* Enrich sentry scope a bit

* Add changes file

* Add goroutine safe scope to errors

* Encapsulate sentry logic

* Add documentation for new flag

* Add sentry capturing to crons and other background tasks

* Only send to sentry when enabled
2022-01-20 16:41:02 -03:00
Martin Angers
f14f97156c
Add read and write timeout options for redis (#3624) 2022-01-11 17:08:39 -05:00
Martin Angers
4143a37056
Fix redis scan keys issue for live queries (#3107) 2021-12-14 16:30:26 -05:00
Zach Wasserman
ac3d8ddf02
Increase default login session length to 24 hours (#3340) 2021-12-13 20:42:30 -08:00
Tomas Touceda
142006cbdd
Issue 2099 enable vulnerability processing new installs (#3104)
* Enable vulnerability processing by default

* Add changes file

* Remove wrong env var

* Fix import

* Fix tests

* Update log line

* Remove unneeded import
2021-12-03 14:44:57 -03:00
Lucas Manuel Rodriguez
028c2643f7
Fleet serve to exit if migrations are missing (default) (#2803)
* Add option to fleet to exit if migrations are missing

* Reverse serve exit migrations logic

* Fix typo
2021-11-22 14:47:24 -03:00
Martin Angers
69a4985cac
Use new error handling approach in other packages (#2954) 2021-11-22 09:13:26 -05:00
Tomas Touceda
2902da76ca
Issue 2027 better error visibility (#2069) 2021-11-02 13:35:57 -04:00
Martin Angers
a8735d55bb
Implement async processing of hosts for label queries (#2288) 2021-11-01 14:13:16 -04:00
dsbaha
51e35e1ba0
Implementation of a Kafka REST Proxy logging plugin (#2534)
This PR implements the status/result logger functions necessary interface with a Kafka REST Proxy service.  

Specifically, this is compatible with the [Confluent KAFKA Rest Proxy Service ](https://docs.confluent.io/1.0/kafka-rest/docs/intro.html).
2021-10-27 21:51:17 -07:00
Martin Angers
c0e0e461e6
Allow empty TLS CA for Redis TLS connections (#2668) 2021-10-25 14:47:53 -04:00
Martin Angers
07cf136d03
Add TLS support to Redis connections (#2568) 2021-10-20 10:09:18 -04:00
Martin Angers
057d4e8b2e
Add configuration and support for Redis to read from replicas (#2509) 2021-10-18 09:32:17 -04:00
Benjamin Edwards
16c5823692
add support for minio backend file carving (#2448)
* add support for minio backend file carving
* add changes file
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-10-12 15:32:06 -04:00
Tomas Touceda
2033d8208c
Add policy updated at (#2246)
* wip

* Add policy updated at interval and update the UI to use that

* Update rest api

* Fix tests
2021-09-27 16:27:38 -03:00