Add wiki docs for AWS logger plugins (#2052)

This commit is contained in:
Zachary Wasserman 2016-04-25 16:20:01 -07:00 committed by Mike Arpaia
parent 262833c86a
commit 10781e896e
2 changed files with 56 additions and 0 deletions

View File

@ -0,0 +1,55 @@
As of version 1.7.4, osquery can log results directly to Amazon AWS [Kinesis Streams](https://aws.amazon.com/kinesis/streams/) and [Kinesis Firehose](https://aws.amazon.com/kinesis/firehose/). For users of these services, `osqueryd` can eliminate the need for a separate log forwarding daemon running in your deployments.
## Configuration
The Kinesis Streams and Kinesis Firehose logger plugins are named `aws_kinesis` and `aws_firehose` respectively. They can be enabled as with other logger plugins using the config flag `logger_plugin`.
Some configuration is shared between the two plugins:
```
--aws_access_key_id VALUE AWS access key ID override
--aws_profile_name VALUE AWS config profile to use for auth and region config
--aws_region VALUE AWS region override
--aws_secret_access_key VALUE AWS secret access key override
```
When working with AWS, osquery will look for credentials and region configuration in the following order:
1. Configuration flags
2. Profile from the [AWS config files](http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-config-files) (only if `--aws_profile_name` is specified)
3. Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)
4. `default` profile in the AWS config files
5. Profile from the EC2 Instance Metadata Service
### Kinesis Streams
When logging to Kinesis Streams, the stream name must be specified with `aws_kinesis_stream`, and the log flushing period can be configured with `aws_kinesis_period`.
### Kinesis Firehose
Similarly for Kinesis Firehose delivery streams, the stream name must be specified with `aws_firehose_stream`, and the period can be configued with `aws_firehose_period`.
### Sample Config File
```
{
"options": {
"host_identifier": "hostname",
"schedule_splay_percent": 10,
"logger_plugin": "kinesis,firehose",
"aws_kinesis_stream": "foo_stream",
"aws_firehose_stream": "bar_delivery_stream",
"aws_access_key_id": "ACCESS_KEY",
"aws_secret_access_key": "SECRET_KEY",
"aws_region": "us-east-1"
},
"schedule": {
"time": {
"query": "SELECT * FROM time;",
"interval": 2,
"removed": false
}
}
}
```
**Note**: Kinesis services have a maximum 1MB record size. Result logs bigger than this will not be forwarded by `osqueryd` as they will be rejected by the Kinesis services.

View File

@ -19,6 +19,7 @@ pages:
- Configuration: deployment/configuration.md
- Logging: deployment/logging.md
- Aggregating Logs: deployment/log-aggregation.md
- AWS Logging: deployment/aws-logging.md
- Performance Safety: deployment/performance-safety.md
- Anomaly Detection: deployment/anomaly-detection.md
- Extensions: deployment/extensions.md