This commit contains the features specified in #1390 as well as a
refactoring of the general osquery configuration code.
The API for the config plugins hasn't changed, although now there's a
`genPack` method that config plugins can implement. If a plugin doesn't
implement `genPack`, then the map<string, string> format cannot be used.
The default config plugin, the filesystem plugin, now implements
`genPack`, so existing query packs code will continue to work as it
always has.
Now many other config plugins can implement custom pack handling for
what makes sense in their context. `genPacks` is not a pure virtual, so
it doesn't have to be implemented in your plugin if you don't want to
use it. Also, more importantly, all config plugins can use the standard
inline pack format if they want to use query packs. Which is awesome.
For more information, refer to #1390, the documentation and the doxygen
comments included with this pull requests, as well as the following
example config which is now supported, regardless of what config plugin
you're using:
```json
{
"options": {
"enable_monitor": "true"
},
"packs": {
"core_os_monitoring": {
"version": "1.4.5",
"discovery": [
"select pid from processes where name like '%osqueryd%';"
],
"queries": {
"kernel_modules": {
"query": "SELECT name, size FROM kernel_modules;",
"interval": 600
},
"system_controls": {
"query": "SELECT * FROM system_controls;",
"interval": 600,
"snapshot": true,
},
"usb_devices": {
"query": "SELECT * FROM usb_devices;",
"interval": 600
}
}
},
"osquery_internal_info": {
"version": "1.4.5",
"discovery": [
"select pid from processes where name like '%osqueryd%';"
],
"queries": {
"info": {
"query": "select i.*, p.resident_size, p.user_time, p.system_time, time.minutes as counter from osquery_info i, processes p, time where p.pid = i.pid;",
"interval": 60,
"snapshot": true
},
"registry": {
"query": "SELECT * FROM osquery_registry;",
"interval": 600,
"snapshot": true
},
"schedule": {
"query": "select name, interval, executions, output_size, wall_time, (user_time/executions) as avg_user_time, (system_time/executions) as avg_system_time, average_memory from osquery_schedule;",
"interval": 60,
"snapshot": true
}
}
}
}
}
```
The `osquery_packs` table was modified to remove the superfluous
columns which could already have been found in `osquery_schedule`. Two
more columns were added in their place, representing stats about pack's
discovery query execution history.
Notably, the internal API for the `osquery::Config` class has changed
rather dramatically as apart of the refactoring. We think this is an
improvement. While strictly adhering to the osquery config plugin
interface will have avoided any compatibility errors, advanced users may
notice compilation errors if they access config data directly. All
internal users of the config have obviously been updated. Yet another
reason to merge your code into mainline; we update it for you when we
refactor!
enum `SecItemClass` changed in 10.11 headers,
so don't instantiate with rvalue of int.
Update `SecKeychainSearchCreateFromAttributes` to match the stricter definition.
Fixes#1423
See Github issue #1301. FreeBSD (which also uses this table) by default has two
users which are UID 0 -- both `toor` and `root`. 19a2d64959 made it so that we
would only get the first one from `getpwent`, but this feature is undesirable
in cases where two different users share the same UID.
When strings match they will be populated into the "strings" column of
the table. The format is identifier:offset.
When a matching rule has tags defined the tags will be put into the
"tags" column of the table in a comma separated list.
When optimizing a table using query constraints an implementation should not add unneeded rows.
A user experience bug exists when selecting with an explicit non-existing pid/uid.
1. Redhat-based distributions were not reporting their version correct.
2. The file read API assumed stat would return an accurate file size.
This has been replaced with an attempt to seek to the end of the file.
X509 parsing is now handled by OpenSSL as there does seem to be a
memory leak in SecCertificateCopyValues of Security framework which resulted
in a performance hit when querying certificates.
key_usage and key_algorithm columns now display human readable strings
(e.g. Digital Signature, CRL Sign rsaEncryption)
than the raw flags and OIDs (e.g 0x86, 1.2.840.1).
This fixes#1032
POSIX-globbing will allow event publishers/subscribers to post-check
results against glob-syntax, fnpath matching, and POSIX C-regex.
These checks are anecdotally speedy.
We now have a Publisher to report on disk events and its metadata,
using the DiskArbitration framework on OS X. Currently disk appearance
and disappearance events are published for both physical and
virtual disks (DMG files). On an event trigger, disk properties are
parsed and that metadata is reported along with the action.
The Subscriber subscribes to virtual disk events currently.
This closes#1103.
`int osquery::filter(const struct dirent*)` seemed like a pretty generic
symbol to have in our symbol table, so I changed it to
`int msrScandirFilter(const struct dirent*)`
This infomation is primarily related to the performance of processor
cores. The information given constitutes only a small portion of
the information in the model specific register, but this table
has been designed so that more information may easily be added.
The table requires osquery be run as the root, and that the msr
kernel module is loaded. The table reads the msr data from /dev
1. Example queries will run with an (optional) integration test.
2. Fix bad accesses with OS X package BOMs
3. Move spec files from ./osquery/tables/specs to ./specs
4. Remove server parsers (netlib) from client builds.
This change allows QueryContext constraints to be checked for based on
operator type. This makes checks for the existence of an equality
operator allow enumeration.
Example:
if (context.constraints["pid"].exists(EQUALS)) {
pids = context.constraints["pid"].getAll(EQUALS);
} else {
osquery::procProcesses(pids);
}
This implements the following tables on FreeBSD:
process_envs
process_memory_map
process_open_files
process_open_sockets
processes
All the heavy lifting is done with libprocstat(3). All the tables follow
the same general principle. Use the common function, getProcesses() in
procstat.cpp, to get the processes and then generate the rows for each
process returned. There is also a procstatCleanup() function commonly
used across all the tables.
The one thing I am not able to test is the process_open_sockets table on
an IPv6 machine.
The user_groups table represents the association between user ids and group ids.
Darwin Issue:
Issues arise in darwin systems with users that are members of many groups due
to a bug in Apple's implementation of getgrouplist. If the number of groups a
user is a member of is greater than 64 a truncated association table may
be returned.
