valitydev/osquery-1
14
0
Fork 0
mirror of https://github.com/valitydev/osquery-1.git synced 2024-11-07 01:55:20 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

docs: Updating process auditing documentation (#4477)

Browse Source
This commit is contained in:
Chris Long 2018-06-18 22:51:55 -07:00 committed by Nick Anderson
parent 4d580d0fa9
commit 5ee495504f
1 changed files with 85 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

101
docs/wiki/deployment/process-auditing.md
View File

@ -1,4 +1,8 @@
Process auditing is available in osquery, but requires additional configuration. It uses the same event-based architecture as the [File Integrity Monitoring (FIM)](../deployment/file-integrity-monitoring.md). To read more about how event-based tables are created and designed, check out the osquery [Table Pubsub Framework](../development/pubsub-framework.md). On all supported platforms, process events are abstracted into [`process_events`](https://osquery.io/schema/current/#process_events). This table abstracts the basic details about process creation. Process state changes and destruction is not yet represented, nor planned.
# Process and socket auditing with osquery
Osquery has the ability to leverage the Linux Audit System to record process executions and network connections in near real-time on Linux and MacOS systems. Although these auditing features are extremely powerful for recording the activity from a host, they may introduce additional computational overhead and greatly increase the number of log events generated by osquery.
Enabling these auditing features requires additional configuration to osquery. The `process_events` and `socket_events` tables use the same event-based architecture as the [File Integrity Monitoring (FIM)](../deployment/file-integrity-monitoring.md). To read more about how event-based tables are created and designed, check out the osquery [Table Pubsub Framework](../development/pubsub-framework.md). On all supported platforms, process events are abstracted into the [`process_events`](https://osquery.io/schema/2.11.2#process_events) table. Similarly, socket events are abstracted into the [`socket_events`](https://osquery.io/schema/2.11.2#process_events) table.
To collect process events add a query like:
```sql
@ -8,28 +12,93 @@ to your query schedule, or to a query pack.
## Linux process auditing
osquery can use Linux Audit on the supported Linux distributions. This does NOT require any audit configuration or `auditd`; actually, auditd should not be running if using osquery's process auditing.
osquery uses the Linux Audit System to collect and process audit events from the kernel. It accomplishes this by monitoring the `execve()` syscall. Auditd should not be running when using osquery's process auditing, as it will conflict with osqueryd over access to the audit netlink socket. You should also ensure auditd is not configured to start at boot.
This creates a bit of confusion since audit, auditd, and libaudit are ambiguous-- osquery only uses the audit features in the kernel. Most distributions do not install libaudit or auditd, this is perfectly fine. If you are configuring audit, using a control binary, or `/etc/audit.conf`, your osquery *may* override your settings.
The only prerequisite for using osquery's auditing functionality on Linux is that you must use a kernel version that contains the audit functionality. Most kernels over version 2.6 have this capability.
How does this work? Let's walk through 3 configuration options. These can be set at the [command line](../installation/cli-flags.md), or via the configuration's options.
There is no requirement to install auditd or libaudit. Osquery only uses the audit features that exist in the kernel.
1. `--disable_audit=false` by default this is set to `true` and prevents osquery from opening the kernel audit's netlink socket.
2. `--audit_allow_config=true` by default this is set to `false` and prevents osquery from making audit configuration changes. These changes include adding/removing rules, setting the global enable flags, and adjusting performance and rate parameters.
3. `--audit_persist=true` but default this is `true` and instructs osquery to 'regain' the audit netlink socket if another process also accesses it.
A sample log entry from process_events may look something like this:
```
{
"action": "added",
"columns": {
"uid": "0",
"time": "1527895541",
"pid": "30219",
"path": "/usr/bin/curl",
"auid": "1000",
"cmdline": "curl google.com",
"ctime": "1503452096",
"cwd": "",
"egid": "0",
"euid": "0",
"gid": "0",
"parent": ""
},
"unixTime": 1527895550,
"hostIdentifier": "vagrant",
"name": "process_events"
}
```
On Linux a companion table `user_events` is included that provides several authentication-based events. If you are enabling process auditing it should be trivial to also include this table.
To better understand how this works, let's walk through 4 configuration options. These flags can be set at the [command line](../installation/cli-flags.md) or placed into the osquery.flags file.
If you would like to debug the audit logging use the hidden flag `--audit_debug`. This will print all of the RAW audit lines to osquery's stdout.
1. `--disable_audit=false` by default this is set to `true` and prevents osquery from opening the kernel audit's netlink socket. By setting it to `false`, we are telling osquery that we want to enable auditing functionality.
2. `--audit_allow_config=true` by default this is set to `false` and prevents osquery from making changes to the audit configuration settings. These changes include adding/removing rules, setting the global enable flags, and adjusting performance and rate parameters. Unless you plan to set all of those things manually, you should leave this as true. If you are configuring audit, using a control binary, or `/etc/audit.conf`, your osquery *may* override your settings.
3. `--audit_persist=true` but default this is `true` and instructs osquery to 'regain' the audit netlink socket if another process also accesses it. However, you should do your best to ensure there will be no other program running which is attempting to access the audit netlink socket.
4. `--audit_allow_process_events=true` this flag indicates that you would like to record process events
#### Linux socket auditing
## Linux socket auditing
Another audit-based table is provided on Linux: `socket_events`. This table reports events for the syscalls `bind` and `connect`. This table is not enabled with process events by default because it introduces considerable added load on the system.
Osquery can also be used to record network connections by enabling `socket_events`. This table uses the syscalls `bind()` and `connect()` to gather information about network connections. This table is not automatically enabled when process_events are enabled because it can introduce considerable load on the system.
Use `--audit_allow_sockets` to enable the associated event subscriber.
To enable socket events, use the `--audit_allow_sockets` flag.
A sample socket_event log entry looks like this:
```
{
"action": "added",
"columns": {
"time": "1527895541",
"success": "1",
"remote_port": "80",
"action": "connect",
"auid": "1000",
"family": "2",
"local_address": "",
"local_port": "0",
"path": "/usr/bin/curl",
"pid": "30220",
"remote_address": "172.217.164.110"
},
"unixTime": 1527895545,
"hostIdentifier": "vagrant",
"name": "socket_events"
}
```
If you would like to log UNIX domain sockets use the hidden flag: `--audit_allow_unix`. This will put considerable strain on the system as many default actions use domain sockets. You will also need to explicitly select the `socket` column from the `socket_events` table.
## Troubleshooting Auditing on Linux
There are a few different methods to ensure you have configured auditing correctly.
1. Ensure you are supplied all of the necessary flags mentioned above in either a command-line argument or in your flagfile.
2. Verify auditd is not running if it is installed on the system.
3. Run `auditctl -s` if the binary is present on your system and verify that `enable` is not set to zero and the `pid` corresponds to a process for osquery
4. Verify that your osquery configuration has a query to `SELECT` from the process_events and/or socket_events tables
5. You may also run auditing using osqueryi:
```
$ osqueryi --audit_allow_config=true --audit_allow_sockets=true --audit_persist=true --disable_audit=false
 --events_expiry=1 --events_max=50000 --logger_plugin=filesystem --disable_events=false
```
If you would like to debug the raw audit events as osqueryd sees them, use the hidden flag `--audit_debug`. This will print all of the RAW audit lines to osquery's stdout.
## User events
On Linux, a companion table called `user_events` is included that provides several authentication-based events. If you are enabling process auditing it should be trivial to also include this table.
## macOS process auditing
osquery has support for OpenBSM audit on Darwin platforms. This feature is already enabled on all macOS installations but doesn't audit process execution or the root user with default settings. To start process auditing on macOS, edit the `audit_control` file in `/etc/security/`. An example configuration is provided below but the important flags are: `ex`, `pc`, `argv`, and `arge`. The `ex` flag will log `exec` events while `pc` logs `exec`, `fork`, and `exit`. If you don't need `fork` and `exit` you may leave that flag out however in future, getting parent pid may require `fork`. If you care about getting the arguments and environment variables you also need `argv` and `arge`. More about these flags can be found [here](https://www.freebsd.org/cgi/man.cgi?apropos=0&sektion=5&query=audit_control&manpath=FreeBSD+7.0-current&format=html). Note that it might require a reboot of the system for these new flags to take effect. `audit -s` should restart the system but your mileage may vary.
@ -54,10 +123,10 @@ member-clear-sflags-mask:has_authenticated
This section provides a brief overview of common and recommended optimizations for event-based tables. These optimizations also apply to the FIM events.
1. `--events_optimize=true` apply optimizations when `SELECT`ing from events-based tables, default enabled.
2. `--events_expiry` the lifetime of buffered events in seconds, default 86000.
3. `--events_max` the maximum number of events to buffer, default 1000.
1. `--events_optimize=true` apply optimizations when `SELECT`ing from events-based tables, enabled by default.
2. `--events_expiry` the lifetime of buffered events in seconds with a default value of 86000.
3. `--events_max` the maximum number of events to store in the buffer before expiring them with a default value of 1000.
The goal of optimizations are to protect the running process and system from impacting performance. By default these are all enabled, which is good for configuration and performance, but may introduce inconsistencies on highly-stressed systems using process auditing.
Optimizations work best when `SELECT`ing often from event-based tables. Otherwise the events are in a buffered state. When an event-based table is selected within the daemon, the backing storage maintaining event data is cleared according to the `--event_expiry` lifetime. Setting this value to `1` will auto-clear events event select, reducing all impact of the buffer.
Optimizations work best when `SELECT`ing often from event-based tables. Otherwise the events are in a buffered state. When an event-based table is selected within the daemon, the backing storage maintaining event data is cleared according to the `--event_expiry` lifetime. Setting this value to `1` will auto-clear events whenever a `SELECT` is performed against the table, reducing all impact of the buffer.