RocksDB is the default "database" plugin. Writes are normally kept in an
in-memory memtable. Writes that are not part of the event pubsub system can
be forced to sync to disk.
This changes several initialization steps:
- The daemon (and shell, though not needed) have a new --ephemeral flag.
- Events are now disabled in the shell by default, use --nodisable_events to
re-enable.
- RocksDB-based backing storage is now disabled in the shell by default.
The --ephemeral flag for the daemon is disabled by default and will allow
skipping configuration and database path sanity, and skipping pidfile checks.
This is intended to be used when debugging or monitoring the daemon process.
To make the RocksDB backing storage feature usage very clear we introduce a new
flag: --disable_database. The shell sets this to true unless overridden in
a flagfile or via command line arguments.
The OS X kernel subscribers have not been starting because they expect the
publisher thread to run before they begin configuration. Due to some recent
refactors the publisher thread creation now occurs after configuration.
The subscriber logic to check for a valid kernel connection is still valid.
This commit has two additional side-effects:
- The RocksDB plugin is modified to use 3 background merge threads.
- The OS X kernel publisher syncing thread is now non-blocking.
2. Introduce a SQLite-based database plugin
3. Refactor database usage to include local 'fast-calls'
4. Introduce an 'ephemeral' database plugin for testing (like a mock)
We found that not installing the headers for snappy caused RocksDB's
snappy detection to not find that snappy was installed:
https://goo.gl/YOWJl0
The snippet there requires that the headers are installed, not just the
library. By installing the headers, we can ensure that snappy is linked.
OR, alternatively, we could just leave it and not link snappy. It's
uncertain what the specific benefits of including snappy are for our
use-case. (CC @igorcanadi)
`make tests` fails with another osquery process running.
The backing-store check happens after a config plugin is setUp and
the initial load occures. This may involve calls to cached keys, the
check should occur pre-config initialize.
