Previously this only existed in the yara table, but it now exists in the
yara config parser land, which will compile signature groups upon
update. Now your signature groups can reference signature files using
paths relative to /var/osquery.
Mostly cleanup as it moved from a gist to get the word out to docs.
This does assume that the change from file_changes to file_events
in #1049 will happen.
This was causing a crash when executing a query using the yara table
from the command line, because YARA was never initialized properly, so
the thread index was whatever was left on the stack. Eventually YARA
would attempt to set a rule that matches using this thread index and
would explode in flames.
Fix it by moving the initialization to a place that is always called.
Remove modes: HTML, Tabs, Explain.
Remove stats (meaningless for virtual tables).
Use the osquery SQLite DB manager within meta and shell SQL
execution to allow registry/extensions non-locking access.
This allows existing runtime DB manipulators to prefer a locking
modifier. Currently these manipulators will fallback to a more
transient db instance. The effect was, no shell-accessible runtime
created virtual tables.
1. Minor refactoring.
- Generate one row per sigfile or sig_group.
- While here, when a signature file fails to compile, VLOG() it.
2. Bring in a couple of YARA tests.
Write a couple of tests for YARA functionality. Right now the only tests
make sure rules are compiled properly and that rules match where they
should and don't match where they shouldn't.
3. Allow sigfiles to be relative to /var/osquery.
- Also, only create a row if scanning happened.
4. Add pattern support to yara table.
- Also, optimize things so that rules are only compiled once.
1. Rename yara_matches to yara_events.
2. Add support for Config::getParser().
- This returns a ConfigPluginRef, which is the ConfigParser for the
given key.
- Being able to get the parser is useful because the
YARAConfigParserPlugin uses it to store the compiled rules as an
attribute.
3. Finish rename and use ConfigParserPlugin.
- Finish the table rename to yara_events.
- Use the new ConfigParserPlugin interface to parse the YARA
configuration. The file_paths and signatures are stored in the
ConfigParserPlugin named "yara" under the key "yara". The rules are
compiled and stored as a private attribute of the same
ConfigParserPlugin object.
Here is an example config using this new structure:
{
// Description of the YARA feature.
"yara": {
"signatures": {
// Each key is an arbitrary group name to give the signatures listed
"sig_group_1": [ "/Users/wxs/foo.sig", "/Users/wxs//bar.sig" ],
"sig_group_2": [ "/Users/wxs/baz.sig" ]
},
"file_paths": {
// Each key is a key from file_paths
// The value is a list of signature groups to run when an event fires
// These will be watched for and scanned when the event framework
// fire off an event to yara_events table
"system_binaries": [ "sig_group_1" ],
"tmp": [ "sig_group_1", "sig_group_2" ]
}
},
// Paths to watch for filesystem events
"file_paths": {
"system_binaries": [ "/usr/bin/%", "/usr/sbin/%" ],
"tmp": [ "/Users/wxs/tmp/%%" ]
}
}
- Currently the signature file must be an absolute path.
3. Move common YARA code to yara_utils.
- In preparation for the yara table (different from yara_events) I'm
moving the common YARA code into a separate place which is shared
between the two tables.
4. Add yara table.
- This allows you to do things like:
```sql
select * from yara where path="/bin/ls" and sigfile="/tmp/foo.sig";
select * from yara where path="/bin/ls" and sig_group="sig_group_1";
```
- The latter will use the signature grouping from the config.
5. Check for keys not existing.
