Commit Graph

54 Commits

Author SHA1 Message Date
Martin Angers
1fa5ce16b8
Add configurable Redis connection retries and following of cluster redirections (#2045)
Closes #1969
2021-09-15 08:50:32 -04:00
Tomas Touceda
429af72b69
Issue 1963 vulnerabilities no sync (#1976)
* wip

* Add tests for skip sync

* Add changes file

* Fix lint
2021-09-14 10:58:35 -03:00
Tomas Touceda
b2efc9f51c
Make redis conn timeout and keep alive configurable (#1968)
* Make redis conn timeout and keep alive configurable

* Document new configs

* Correct config name
2021-09-08 17:55:12 -03:00
Tomas Touceda
6d2ae02efd
Rename core->free and basic->premium (#1870)
* Rename core->free and basic->premium

* Fix lint js

* Comment out portion of test that seems to timeout

* Rename tier  to premium if basic is still loaded
2021-09-03 13:05:23 -03:00
Martin Angers
3755a58070
Add support for MySQL read replicas (#1786)
Adds configuration options to use a read-only MySQL replica,
and uses it instead of the primary for reads.
2021-09-01 15:50:52 -04:00
Tomas Touceda
70f728fe84
Make vulnerability processing more configurable (#1718)
* Make vulnerability processing more configurable

* Simplify leader selection
2021-08-18 17:25:14 -03:00
Benjamin Edwards
c18214be37
add logging settings to config api response (#1467)
- add docker-compose file for locally testing aws dependencies
- update firehose & kinesis configs to optionally supply endpoint url override
- serialize `logging` field in appconfig api response
2021-07-30 11:45:49 -04:00
Benjamin Edwards
799243ffb5
Windows friendly changes after walking through getting started guide (#1441)
* update .gitattributes to be explicit about line endings with regards to the test certs
* update building-fleet guide to include python2 dependency on windows
* update configuration to default to OS specific temporary directories
2021-07-21 20:49:44 -04:00
Zach Wasserman
6013194c1d
Reduce default session duration to 4 hours (#981) 2021-06-06 18:28:47 -07:00
Zach Wasserman
0356115b03
Remove JWT in Fleet session management (#979)
See #978 for motivations for this change.

Closes #978.
2021-06-06 18:10:58 -07:00
Zach Wasserman
fb32f0cf40
Remove kolide types and packages from backend (#974)
Generally renamed `kolide` -> `fleet`
2021-06-06 15:07:29 -07:00
Zach Wasserman
08d57b40d1
Remove deprecated kolide names from API routes and configuration (#957)
Closes #260
2021-06-04 16:51:18 -07:00
Zach Wasserman
16a8e13f85
Fix tls_compatibility key in yaml config (#955)
Closes #271
2021-06-04 08:44:36 -07:00
Zach Wasserman
83b7f79699
Stub out licensing API (#810)
- Add config option for license key.
- Define license details data structure.
- Include license details in app config API responses.

Currently any non-empty value for `--license_key` behaves as though the
installation is licensed for `basic`. If the license key is empty,
`core` is returned.

Still to come is the appropriate parsing for the license key.
2021-05-19 17:29:38 -07:00
Josh Brower
86745ba2dc
Add ability to duplicate live query results in Redis (#762)
This feature enables a new config option (redis.duplicate_results). When set to true, all Live Query results will be copied to an additional Redis pubsub channel named LQDuplicate

This is useful in a scenario that would involve shipping the Live Query results outside of Fleet, near-realtime.
2021-05-13 16:01:31 -07:00
Michael Samuel
fb45806088
Copy log fields into GCP PubSub attributes (#712)
Add a config setting to allow copying message fields and decorations into Google Pub/Sub attributes, making it possible to use these values for subscription filters.
2021-05-08 12:29:52 -07:00
dsbaha
1cb514c460
Add flag to disable HTTP keepalives (#741)
In some environments, disabling keepalives helps prevent buildup of TCP sockets.
2021-05-07 17:29:54 -07:00
Zach Wasserman
f90da6a090
Make enrollment cooldown configurable (#418)
The enrollment cooldown period was sometimes causing problems when
osquery (probably unintentionally, see
https://github.com/osquery/osquery/issues/6993) tried to enroll more
than once from the same osqueryd process.

We now set this to default to off and make it configurable. With #417
this feature may be unnecessary for most deployments.
2021-03-08 21:26:09 -08:00
Zach Wasserman
cfba095cda
Make host identifier configurable within Fleet (#417)
Osquery now exposes more information during host enrollment than Fleet
previously handled. We can use this to provide more options to users in
problematic enrollment scenarios.

Users can configure --osquery_host_identifier in Fleet to set which
identifier is used to determine uniqueness of hosts. The
default (provided) replicates existing behavior in Fleet. For many
users, setting this to instance will provide better enrollment
stability.

Closes #373
2021-03-08 18:35:17 -08:00
Zach Wasserman
de0b3324b1
Add AWS Lambda as logging plugin (#347)
This plugin invokes the provided function with each log line as the
payload.

Closes #342
2021-02-24 10:02:26 -08:00
Zach Wasserman
d624e099fb
Deprecate environment variable prefix (#301)
- Support both `FLEET_` and `KOLIDE_` prefixes.
- Add logging about deprecated `KOLIDE_` prefix.
- Update documentation and sample configs.
2021-02-11 15:36:58 -08:00
Zach Wasserman
ddb05cce94
Change default TLS compatibility to intermediate (#270)
In #212 these settings were updated and caused connectivity issues for
users in common environment configurations. The new changes are
aggressive (modern enforces TLS 1.3) and Mozilla indicates that
intermediate is an appropriate default. This will ensure better
compatibility for common deployments while still allowing the option to
use the strictest settings.

Document unintentional mismatched yaml key.

Fixes #269
2021-02-03 11:48:48 -08:00
CptOfEvilMinions
626429c38e
Added support to read jwt and mysql password from a file (#141)
The current implementation of FleetDM doesn't support Docker secrets for supplying the MySQL password and JWT key. This PR provides the ability for a file path to read in secrets. The goal of this PR is to avoid storing secrets in a static config or in an environment variable. 

Example config for Docker:
```yaml
mysql:
  address: mysql:3306
  database: fleet
  username: fleet
  password_path: /run/secrets/mysql-fleetdm-password
redis:
  address: redis:6379
server:
  address: 0.0.0.0:8080
  cert: /run/secrets/fleetdm-tls-cert
  key: /run/secrets/fleetdm-tls-key
auth:
  jwt_key_path: /run/secrets/fleetdm-jwt-key
filesystem:
  status_log_file: /var/log/osquery/status.log
  result_log_file: /var/log/osquery/result.log
  enable_log_rotation: true
logging:
  json: true
```
2021-01-04 07:58:43 -08:00
Matteo Piano
c89cd370d5
Add AWS S3 as file carving backend (#126)
This adds the option to set up an S3 bucket as the storage backend for file carving (partially solving #111).

It works by using the multipart upload capabilities of S3 to maintain compatibility with the "upload in blocks" protocol that osquery uses. It does this basically replacing the carve_blocks table while still maintaining the metadata in the original place (it would probably be possible to rely completely on S3 by using object tagging at the cost of listing performance). To make this pluggable, I created a new field in the service struct dedicated to the CarveStore which, if no configuration for S3 is set up will be just a reference to the standard datastore, otherwise it will point to the S3 one (effectively this separation will allow in the future to add more backends).
2020-12-16 09:16:55 -08:00
Kilian
c61ba759dd
Add redis use_tls cfg (#2311)
Adding config parameter 'redis.use_tls' to enable tls communications with redis e.g. AWS ElastiCache

Closes #2247
2020-10-01 16:25:48 -07:00
Lars Lehtonen
d193ea1717
Remove Support for Deprecated TLSProfileOld (#2142)
Co-authored-by: Zachary Wasserman <zach@dactiv.llc>
2020-09-10 09:31:01 -07:00
James Alseth
3a63dac4a3
Add compression option for filesystem logs when they're rotated (#2292) 2020-09-09 13:33:32 -07:00
billcobbler
20328b0f87
Add stdout and kinesis logger plugins and sts assume role to Firehose (#2282)
Co-authored-by: Brendan Shaklovitz <nyanshak@users.noreply.github.com>
2020-08-19 14:56:44 -07:00
Stephan Miehe
2ad5205a4b
Add support for conn_max_lifetime (#2270)
This adds support to configure MySQL conn_max_lifetime.
2020-07-30 09:00:42 -07:00
Stephan Miehe
cf4d8ecfee
Add redis database number support (#2269)
Fixes #2268
2020-07-30 08:57:25 -07:00
Zachary Wasserman
f6223ca0e4
Add ability to modify host detail update interval (#2200)
This may be desirable for some deployments to reduce server load.
2020-03-02 11:08:08 -08:00
Zachary Wasserman
ee0a6e9064
Add deprecation warning for "old" TLS compatibility (#2183)
Warn users in advance of removing this in #2142.
2020-01-14 09:36:07 -08:00
Max Bigras
b524d813ca Add MysqlConfig protocol option (#2161)
Enables configuring MySQL to run with unix domain sockets.
Closes #2160
2019-12-04 09:48:24 -08:00
Zachary Wasserman
adf87140a7
Add ability to prefix Fleet URLs (#2112)
- Add the server_url_prefix flag for configuring this functionality
- Add prefix handling to the server routes
- Refactor JS to use appropriate paths from modules
- Use JS template to get URL prefix into JS environment
- Update webpack config to support prefixing

Thanks to securityonion.net for sponsoring the development of this feature.

Closes #1661
2019-10-16 16:40:45 -07:00
Michael Samuel
969d5f25af Add Google Cloud PubSub logging (#2049)
Adds Google Cloud PubSub logging for status and results.

This also changes the Write interface for logging modules to add a context.Context (only used by pubsub currently).
2019-07-16 15:41:50 -07:00
Zachary Wasserman
75868a7005
Do not panic after error reading config file (#2033)
Fixes #1445
2019-04-23 15:59:02 -07:00
Zachary Wasserman
e59714242e
Add Firehose logging capabilities for result and status logs (#2022)
- Refactor configuration for logging to use separate plugins
- Move existing filesystem logging to filesystem plugin
- Create new AWS firehose plugin
- Update documentation around logging
2019-04-08 11:47:15 -07:00
Zachary Wasserman
c8229cc0d6
Replace uses of the term "Kolide" with "Fleet" (#1999)
Almost two years ago, we began referring to the project as Fleet, but there are
many occurences of the term "Kolide" throughout the UI and documentation. This
PR attempts to clear up those uses where it is easily achievable.

The term "Kolide" is used throughout the code as well, but modifying this would
be more likely to introduce bugs.
2019-01-24 09:39:32 -08:00
Scott J. Roberts
9c52bed855 Add flags for configuring MySQL connection pooling limits (#1672) 2017-12-19 13:52:52 -08:00
Zachary Wasserman
629a740b45 Require JWT Key to be specified for server startup (#1480)
If server is started without a JWT key, a message like the following is printed:
```
################################################################################
# ERROR:
#   A value must be supplied for --auth_jwt_key. This value is used to create
#   session tokens for users.
#
#   Consider using the following randomly generated key:
#   om3w95gMA2drT5xAdLd2Q5oE8fLw+Miz
################################################################################
```

Closes #1480.
2017-04-12 15:05:56 -07:00
Zachary Wasserman
d3eb3b7272 Remove unused --app_token_key flag (#1479)
Closes #1469
2017-04-11 17:13:38 -07:00
John Murphy
c90368c4af Changed default osquery logging behavior
Made log rotation for osquery results and status logs optional.  This required writing the logwriter package which is a drop in replacement for lumberjack.  We still use lumberjack if the log rotation flag --osquery_enable_log_rotation flag is set. Note that the performance of the default is quite a bit better than lumberjack.


BenchmarkLogger-8       	 2000000	       747 ns/op
BenchmarkLumberjack-8   	 1000000	      1965 ns/op
PASS
BenchmarkLogger-8       	 2000000	       731 ns/op
BenchmarkLumberjack-8   	 1000000	      2040 ns/op
PASS
BenchmarkLogger-8       	 2000000	       741 ns/op
BenchmarkLumberjack-8   	 1000000	      1970 ns/op
PASS
BenchmarkLogger-8       	 2000000	       737 ns/op
BenchmarkLumberjack-8   	 1000000	      1930 ns/op
PASS
2017-04-03 16:48:50 -05:00
John Murphy
039e9e1a98 Add TLS profiles to command line (#1444)
* Add TLS profiles to command line

* Code review changes per @groob

* fixed busted test
2017-03-27 23:21:48 -05:00
Zachary Wasserman
5be9d69165 Ensure config values roundtrip properly through config_dump (#1266)
- Set the appropriate yaml tags for dumping
- Add test to verify roundtrip

Fixes #1261
2017-02-22 07:22:19 -08:00
Victor Vrantchan
1ee94d4f75 add mysql client certificate support (#1240) 2017-02-16 17:14:00 -07:00
Zachary Wasserman
7212d4c333 Add inline flag documentation in --help (#1135)
- Cleanup unused app.web_address flag
2017-01-31 10:06:30 -08:00
Victor Vrantchan
54408ff9e4 move osquery enroll secret to appconfig (#1004)
For #995
2017-01-20 14:48:54 -05:00
Victor Vrantchan
c13b8cc0cf support JSON output for kolide logs (#1026) 2017-01-19 09:41:28 -05:00
Victor Vrantchan
de3794b17b remove SMTP CLI flags. SMTP is now handles as app config in db (#949) 2017-01-13 08:54:16 -05:00
Zachary Wasserman
c8e6405220 Use redis for distributed query results when not in dev mode (#653)
- Add appropriate configs for Redis
- Use the Redis pubsub store by default, inmem in dev mode
2016-12-15 16:13:23 -08:00