Include table changes necessary for a Windows processes table and changes to other tables needed for daemon and shell to run. The Windows processes table uses WMI as a backend to gather information. This commit does not yet build these tables.
* Integrated filesystem operation abstraction code into filesystem.cpp
* Modified filesystem unit tests to be more platform agnostic
* Added append mode for PlatformFile
* Minor bug fixes in filesystem operations
This commit bumps the third-party SQLite to the 3.14.0 pre-release (18:59).
With 3.14.0 the LIKE and EQUALS constraint operators may be mixed within a
query. Previously these would fail to produce a valid set.
As part of the support, each virtual table should choose to bypass rowid-based
deduplication using the new "WITHOUT ROWID" create table epilog. This will
be appended to the schema if the table defines a PRIMARY KEY using index=True.
Some of the types in fileops tests were causing warnings to be thrown
during build, due to type mismatch. I've added a few local variables to
quiet these warnings.
* Implemented filesystem operations abstraction code
* Added filesystem operations abstraction unit tests
* Modified CMake configurations to support the building of the abstraction code and unit tests
The three new SQLite functions:
- split: Splits a column using a set of tokens and a selected index.
- regex_split: Similar to split but with a regex instead of tokens.
- inet_aton: Returns the IPv4 decimal value for a string-formatted address.
With a removeService method, combined with the abstracted thread start in
the Dispatcher API, services auto-remove when finished.
This will un-break the kernel communication tests. These tests only stop
when all their producer threads/services have ended.
This also promotes the OS X kernel build to 10.11.
This changes several initialization steps:
- The daemon (and shell, though not needed) have a new --ephemeral flag.
- Events are now disabled in the shell by default, use --nodisable_events to
re-enable.
- RocksDB-based backing storage is now disabled in the shell by default.
The --ephemeral flag for the daemon is disabled by default and will allow
skipping configuration and database path sanity, and skipping pidfile checks.
This is intended to be used when debugging or monitoring the daemon process.
To make the RocksDB backing storage feature usage very clear we introduce a new
flag: --disable_database. The shell sets this to true unless overridden in
a flagfile or via command line arguments.
Table options includes a change to the Registry::call API for TablePlugins.
When requesting route information or the 'columns' action, a new 'op' key is included.
Integrated process abstraction code into more locations
Defined new macros for abstracting across various platforms
Added GLOG_NO_ABBREVIATED_SEVERITIES for glog to support Windows
Fixed some minor CMake issues involving thrift
Updated gflags package; reflecting change in provision script
Preparing CMake config files for WIN32 support
This introduces a new testing-related flag to the osqueryi shell.
The --pack flag takes a single argument, the name of a configured pack.
To list the configured packs try:
$ osqueryi -A osquery_packs
To run all queries, immediately and serially, try:
$ osqueryi --pack NAME_OF_PACK
When selecting UUID as the host identifier the client follows:
- If host_identifier is set to hostname, return the hostname
- If UUID check a static UUID container
- If empty request the UUID from the backing storage, RocksDB
- If empty use the Apple hardware UUID or generate a UUID
- If the Apple UUID request fails generate a UUID
When no UUID is available in the backing store save the new UUID.
The Thrift library should be discovered simiarly to all others.
When using the build-macros for library discovery the caller can choose
how to discover static or dynamic links.
Added Windows support for process operations.
Added unit tests for process abstraction code for POSIX and Windows.
Modified CMake config files to support building the new code and unit tests.
The initialization of a logger plugin was confusing. The 'init' step was
introduced to allow a daemon to buffer status events before a logger plugin
is determined by external/remote configuration. The buffered statuses could
then be transferred via a medium other than Glog (the default). To determine
if Glog should continue to write statuses to the filesystem the 'init' method
returned a Status.
Logger plugins should now use a feature method override to select how status
logs should be handled.
The OS X kernel subscribers have not been starting because they expect the
publisher thread to run before they begin configuration. Due to some recent
refactors the publisher thread creation now occurs after configuration.
The subscriber logic to check for a valid kernel connection is still valid.
This commit has two additional side-effects:
- The RocksDB plugin is modified to use 3 background merge threads.
- The OS X kernel publisher syncing thread is now non-blocking.
The mobile 'LowBattery' crash logs provide little if any data of value,
and seem to be nothing more than notification events. We're filtering
these out of the crashes table.
This adds a new optimization feature that allows expensive tables to cache
their results between JOINs. Consider JOINing a list of open sockets, for each
process, then requesting to hash each process path. This query may hash
the same path multiple times.
Within-query caching allows the hash table to respond with the previous
result of the hash request as long as the requested computation was the result
of a single query. Subsequent queries will perform subsequent hashing.
This commit adds mobile device crashes to the list of crash logs parsed by the Crashes table as well as adding a lamdba to improve code reuse. The commit also adds a 'type' column to the table to indicate what kind of log this crash log was.
Renamed the crash_log table to crashes for future abstraction to other
operating systems. Also fixed how the table was parsing the most recent
stack trace and the registers. Register values are now all parsed into
one column 'registers', which will be a space delimited string of the
form:
register:value register:value ... register:value
in order to best allow for OS abstraction.
It is often helpful to know the local timezone of the machine. For this use
local_timezone, as the base timezone will use local or UTC depending on the
--utc flag. This will be default=UTC in osquery 1.8.0.
The datetime field is added to mimic ISO 8601, along with iso_8601.
The timestamp field remains as the time stamp used for logging (within osquery)
and commonly outside of osquery. The goal for adding multiple representations
is to allow joining/augmenting of other tables.
Beginning in version 1.8.0 all time uses will converge on an osquery-provided
getUnixTime() API call that returns, by default, UNIX time integers converted
to UTC/GMT. The 'time' table will respond with the parsed time for the
configuration. If the timezone is not UTC then osquery is using localtime.
This configuration option will affect the 'unix_time' response in the 'time'
table. Because of this configurable-effect the table is extended to include
'local_time' which is always the system local UNIX time.
