Commit Graph

74 Commits

Author SHA1 Message Date
Michal Nicpon
1aa7b96837
change session duration to 5 days (#4480) 2022-03-07 15:37:54 -07:00
Zachary Winnerman
c5c72ed713
Add apm for testing apm (#4053)
* Add apm for testing apm

* Testing opentracing

* testing

* Testing

* go fmt

* Add config switch for tracing.

* fixup

* Update cmd/fleet/serve.go

Co-authored-by: Tomas Touceda <chiiph@gmail.com>

* Add support for both elasticapm and opentelemetry

* Fix driver stuff and config options

* Fixup

* fixup

* Add changes file

* Add config for sql driver

* fixup

* Add doc to exported field

* testing

* fixup

* fixup

* Testing again

* fixup

* testing

* Undo

Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-02-15 12:42:22 -05:00
Tomas Touceda
11887f87f7
Add enable scheduled query stats to fleet config (#4066)
* Add enable scheduled query stats to fleet config as well

* Add documentation

* Revert "Allow disabling scheduled query stats via app config (#4049)"

This reverts commit f98fd4d331.

* Add changes file

* Update ref

* Add missing docs
2022-02-09 08:20:29 -03:00
Martin Angers
a6f3f02a85
Cleanup unused software after calculating the count of hosts (#3887) 2022-01-26 11:32:42 -05:00
Tomas Touceda
9d572309ae
Add sentry (#3669)
* Add sentry

* Fix gosum

* More gosum fixes

* Add missing def for config

* Enrich sentry scope a bit

* Add changes file

* Add goroutine safe scope to errors

* Encapsulate sentry logic

* Add documentation for new flag

* Add sentry capturing to crons and other background tasks

* Only send to sentry when enabled
2022-01-20 16:41:02 -03:00
Martin Angers
f14f97156c
Add read and write timeout options for redis (#3624) 2022-01-11 17:08:39 -05:00
Martin Angers
4143a37056
Fix redis scan keys issue for live queries (#3107) 2021-12-14 16:30:26 -05:00
Zach Wasserman
ac3d8ddf02
Increase default login session length to 24 hours (#3340) 2021-12-13 20:42:30 -08:00
Tomas Touceda
142006cbdd
Issue 2099 enable vulnerability processing new installs (#3104)
* Enable vulnerability processing by default

* Add changes file

* Remove wrong env var

* Fix import

* Fix tests

* Update log line

* Remove unneeded import
2021-12-03 14:44:57 -03:00
Lucas Manuel Rodriguez
028c2643f7
Fleet serve to exit if migrations are missing (default) (#2803)
* Add option to fleet to exit if migrations are missing

* Reverse serve exit migrations logic

* Fix typo
2021-11-22 14:47:24 -03:00
Martin Angers
69a4985cac
Use new error handling approach in other packages (#2954) 2021-11-22 09:13:26 -05:00
Tomas Touceda
2902da76ca
Issue 2027 better error visibility (#2069) 2021-11-02 13:35:57 -04:00
Martin Angers
a8735d55bb
Implement async processing of hosts for label queries (#2288) 2021-11-01 14:13:16 -04:00
dsbaha
51e35e1ba0
Implementation of a Kafka REST Proxy logging plugin (#2534)
This PR implements the status/result logger functions necessary interface with a Kafka REST Proxy service.  

Specifically, this is compatible with the [Confluent KAFKA Rest Proxy Service ](https://docs.confluent.io/1.0/kafka-rest/docs/intro.html).
2021-10-27 21:51:17 -07:00
Martin Angers
c0e0e461e6
Allow empty TLS CA for Redis TLS connections (#2668) 2021-10-25 14:47:53 -04:00
Martin Angers
07cf136d03
Add TLS support to Redis connections (#2568) 2021-10-20 10:09:18 -04:00
Martin Angers
057d4e8b2e
Add configuration and support for Redis to read from replicas (#2509) 2021-10-18 09:32:17 -04:00
Benjamin Edwards
16c5823692
add support for minio backend file carving (#2448)
* add support for minio backend file carving
* add changes file
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-10-12 15:32:06 -04:00
Tomas Touceda
2033d8208c
Add policy updated at (#2246)
* wip

* Add policy updated at interval and update the UI to use that

* Update rest api

* Fix tests
2021-09-27 16:27:38 -03:00
Tomas Touceda
1f324339f8
Add jitter to intervals (#2158)
* Add max jitter percent config

* Fix jitter calc

* Remove comment

* Reduce test jitter to make tests less flaky

* Remove jitter entirely

* Document new config

* Fix doc link
2021-09-21 14:21:44 -03:00
Martin Angers
1fa5ce16b8
Add configurable Redis connection retries and following of cluster redirections (#2045)
Closes #1969
2021-09-15 08:50:32 -04:00
Tomas Touceda
429af72b69
Issue 1963 vulnerabilities no sync (#1976)
* wip

* Add tests for skip sync

* Add changes file

* Fix lint
2021-09-14 10:58:35 -03:00
Tomas Touceda
b2efc9f51c
Make redis conn timeout and keep alive configurable (#1968)
* Make redis conn timeout and keep alive configurable

* Document new configs

* Correct config name
2021-09-08 17:55:12 -03:00
Tomas Touceda
6d2ae02efd
Rename core->free and basic->premium (#1870)
* Rename core->free and basic->premium

* Fix lint js

* Comment out portion of test that seems to timeout

* Rename tier  to premium if basic is still loaded
2021-09-03 13:05:23 -03:00
Martin Angers
3755a58070
Add support for MySQL read replicas (#1786)
Adds configuration options to use a read-only MySQL replica,
and uses it instead of the primary for reads.
2021-09-01 15:50:52 -04:00
Tomas Touceda
70f728fe84
Make vulnerability processing more configurable (#1718)
* Make vulnerability processing more configurable

* Simplify leader selection
2021-08-18 17:25:14 -03:00
Benjamin Edwards
c18214be37
add logging settings to config api response (#1467)
- add docker-compose file for locally testing aws dependencies
- update firehose & kinesis configs to optionally supply endpoint url override
- serialize `logging` field in appconfig api response
2021-07-30 11:45:49 -04:00
Benjamin Edwards
799243ffb5
Windows friendly changes after walking through getting started guide (#1441)
* update .gitattributes to be explicit about line endings with regards to the test certs
* update building-fleet guide to include python2 dependency on windows
* update configuration to default to OS specific temporary directories
2021-07-21 20:49:44 -04:00
Zach Wasserman
6013194c1d
Reduce default session duration to 4 hours (#981) 2021-06-06 18:28:47 -07:00
Zach Wasserman
0356115b03
Remove JWT in Fleet session management (#979)
See #978 for motivations for this change.

Closes #978.
2021-06-06 18:10:58 -07:00
Zach Wasserman
fb32f0cf40
Remove kolide types and packages from backend (#974)
Generally renamed `kolide` -> `fleet`
2021-06-06 15:07:29 -07:00
Zach Wasserman
08d57b40d1
Remove deprecated kolide names from API routes and configuration (#957)
Closes #260
2021-06-04 16:51:18 -07:00
Zach Wasserman
16a8e13f85
Fix tls_compatibility key in yaml config (#955)
Closes #271
2021-06-04 08:44:36 -07:00
Zach Wasserman
83b7f79699
Stub out licensing API (#810)
- Add config option for license key.
- Define license details data structure.
- Include license details in app config API responses.

Currently any non-empty value for `--license_key` behaves as though the
installation is licensed for `basic`. If the license key is empty,
`core` is returned.

Still to come is the appropriate parsing for the license key.
2021-05-19 17:29:38 -07:00
Josh Brower
86745ba2dc
Add ability to duplicate live query results in Redis (#762)
This feature enables a new config option (redis.duplicate_results). When set to true, all Live Query results will be copied to an additional Redis pubsub channel named LQDuplicate

This is useful in a scenario that would involve shipping the Live Query results outside of Fleet, near-realtime.
2021-05-13 16:01:31 -07:00
Michael Samuel
fb45806088
Copy log fields into GCP PubSub attributes (#712)
Add a config setting to allow copying message fields and decorations into Google Pub/Sub attributes, making it possible to use these values for subscription filters.
2021-05-08 12:29:52 -07:00
dsbaha
1cb514c460
Add flag to disable HTTP keepalives (#741)
In some environments, disabling keepalives helps prevent buildup of TCP sockets.
2021-05-07 17:29:54 -07:00
Zach Wasserman
f90da6a090
Make enrollment cooldown configurable (#418)
The enrollment cooldown period was sometimes causing problems when
osquery (probably unintentionally, see
https://github.com/osquery/osquery/issues/6993) tried to enroll more
than once from the same osqueryd process.

We now set this to default to off and make it configurable. With #417
this feature may be unnecessary for most deployments.
2021-03-08 21:26:09 -08:00
Zach Wasserman
cfba095cda
Make host identifier configurable within Fleet (#417)
Osquery now exposes more information during host enrollment than Fleet
previously handled. We can use this to provide more options to users in
problematic enrollment scenarios.

Users can configure --osquery_host_identifier in Fleet to set which
identifier is used to determine uniqueness of hosts. The
default (provided) replicates existing behavior in Fleet. For many
users, setting this to instance will provide better enrollment
stability.

Closes #373
2021-03-08 18:35:17 -08:00
Zach Wasserman
de0b3324b1
Add AWS Lambda as logging plugin (#347)
This plugin invokes the provided function with each log line as the
payload.

Closes #342
2021-02-24 10:02:26 -08:00
Zach Wasserman
d624e099fb
Deprecate environment variable prefix (#301)
- Support both `FLEET_` and `KOLIDE_` prefixes.
- Add logging about deprecated `KOLIDE_` prefix.
- Update documentation and sample configs.
2021-02-11 15:36:58 -08:00
Zach Wasserman
ddb05cce94
Change default TLS compatibility to intermediate (#270)
In #212 these settings were updated and caused connectivity issues for
users in common environment configurations. The new changes are
aggressive (modern enforces TLS 1.3) and Mozilla indicates that
intermediate is an appropriate default. This will ensure better
compatibility for common deployments while still allowing the option to
use the strictest settings.

Document unintentional mismatched yaml key.

Fixes #269
2021-02-03 11:48:48 -08:00
CptOfEvilMinions
626429c38e
Added support to read jwt and mysql password from a file (#141)
The current implementation of FleetDM doesn't support Docker secrets for supplying the MySQL password and JWT key. This PR provides the ability for a file path to read in secrets. The goal of this PR is to avoid storing secrets in a static config or in an environment variable. 

Example config for Docker:
```yaml
mysql:
  address: mysql:3306
  database: fleet
  username: fleet
  password_path: /run/secrets/mysql-fleetdm-password
redis:
  address: redis:6379
server:
  address: 0.0.0.0:8080
  cert: /run/secrets/fleetdm-tls-cert
  key: /run/secrets/fleetdm-tls-key
auth:
  jwt_key_path: /run/secrets/fleetdm-jwt-key
filesystem:
  status_log_file: /var/log/osquery/status.log
  result_log_file: /var/log/osquery/result.log
  enable_log_rotation: true
logging:
  json: true
```
2021-01-04 07:58:43 -08:00
Matteo Piano
c89cd370d5
Add AWS S3 as file carving backend (#126)
This adds the option to set up an S3 bucket as the storage backend for file carving (partially solving #111).

It works by using the multipart upload capabilities of S3 to maintain compatibility with the "upload in blocks" protocol that osquery uses. It does this basically replacing the carve_blocks table while still maintaining the metadata in the original place (it would probably be possible to rely completely on S3 by using object tagging at the cost of listing performance). To make this pluggable, I created a new field in the service struct dedicated to the CarveStore which, if no configuration for S3 is set up will be just a reference to the standard datastore, otherwise it will point to the S3 one (effectively this separation will allow in the future to add more backends).
2020-12-16 09:16:55 -08:00
Kilian
c61ba759dd
Add redis use_tls cfg (#2311)
Adding config parameter 'redis.use_tls' to enable tls communications with redis e.g. AWS ElastiCache

Closes #2247
2020-10-01 16:25:48 -07:00
Lars Lehtonen
d193ea1717
Remove Support for Deprecated TLSProfileOld (#2142)
Co-authored-by: Zachary Wasserman <zach@dactiv.llc>
2020-09-10 09:31:01 -07:00
James Alseth
3a63dac4a3
Add compression option for filesystem logs when they're rotated (#2292) 2020-09-09 13:33:32 -07:00
billcobbler
20328b0f87
Add stdout and kinesis logger plugins and sts assume role to Firehose (#2282)
Co-authored-by: Brendan Shaklovitz <nyanshak@users.noreply.github.com>
2020-08-19 14:56:44 -07:00
Stephan Miehe
2ad5205a4b
Add support for conn_max_lifetime (#2270)
This adds support to configure MySQL conn_max_lifetime.
2020-07-30 09:00:42 -07:00
Stephan Miehe
cf4d8ecfee
Add redis database number support (#2269)
Fixes #2268
2020-07-30 08:57:25 -07:00