This will remove the use of current time for syslog.time and introduce
a new column called 'datetime'.
Events now uses an "optimize_id" alongside "optimize" to prevent returning
colliding events added within the same second as the previous genTable call.
This includes a minor SDK refactor as it move quite a few specialized
functions and facilities from core.h into system.h. There was a breaking point
for needing to frequently update core includes.
The new logger systemLog function allows a call site to bypass logging config
and write a line to the OS logger (aka syslog).
The OS X kernel subscribers have not been starting because they expect the
publisher thread to run before they begin configuration. Due to some recent
refactors the publisher thread creation now occurs after configuration.
The subscriber logic to check for a valid kernel connection is still valid.
This commit has two additional side-effects:
- The RocksDB plugin is modified to use 3 background merge threads.
- The OS X kernel publisher syncing thread is now non-blocking.
This commit adds an event-based virtual table implementation for
querying the linux syslog. It introduces an event publisher that
attaches to a named pipe to ingest CSV formatted syslog forwarded from
rsyslogd. An event subscriber/virtual table makes these log lines
available for queries. Currently, no additional processing is done on
the input data besides parsing.
Using this table requires a properly configured rsyslogd. Documentation
for this configuration is forthcoming in the wiki.
The refactor of config/packs was initiated because event subscribers needed
a method for toggling `::init` based on some configurable option. In the case
of auditd, turning on the support with `--disable_audit=false` used to start
auditing the EXECVE syscall. It was understandable that this would cause
latency based on the number of processes executing per measure of time.
A new `socket_events` table will do the same but for `bind` and `connect`. These
are less-obvious and for now, require a scan of /proc for socket tuples. In the
future this file descriptor to socket tuple will be faster.
