osquery-1/specs/yara.table

table_name("yara")
description("Track YARA matches for files or PIDs.")
schema([
    Column("path", TEXT, "The path scanned"),
    Column("matches", TEXT, "List of YARA matches"),
    Column("count", INTEGER, "Number of YARA matches"),
    Column("sig_group", TEXT, "Signature group used"),
    Column("sigfile", TEXT, "Signature file used"),
    Column("pattern", TEXT, "A pattern which can be used to match file paths"),
    Column("strings", TEXT, "Matching strings"),
    Column("tags", TEXT, "Matching tags"),
])
implementation("yara@genYara")
Major YARA refactor and enhancements 1. Rename yara_matches to yara_events. 2. Add support for Config::getParser(). - This returns a ConfigPluginRef, which is the ConfigParser for the given key. - Being able to get the parser is useful because the YARAConfigParserPlugin uses it to store the compiled rules as an attribute. 3. Finish rename and use ConfigParserPlugin. - Finish the table rename to yara_events. - Use the new ConfigParserPlugin interface to parse the YARA configuration. The file_paths and signatures are stored in the ConfigParserPlugin named "yara" under the key "yara". The rules are compiled and stored as a private attribute of the same ConfigParserPlugin object. Here is an example config using this new structure: { // Description of the YARA feature. "yara": { "signatures": { // Each key is an arbitrary group name to give the signatures listed "sig_group_1": [ "/Users/wxs/foo.sig", "/Users/wxs//bar.sig" ], "sig_group_2": [ "/Users/wxs/baz.sig" ] }, "file_paths": { // Each key is a key from file_paths // The value is a list of signature groups to run when an event fires // These will be watched for and scanned when the event framework // fire off an event to yara_events table "system_binaries": [ "sig_group_1" ], "tmp": [ "sig_group_1", "sig_group_2" ] } }, // Paths to watch for filesystem events "file_paths": { "system_binaries": [ "/usr/bin/%", "/usr/sbin/%" ], "tmp": [ "/Users/wxs/tmp/%%" ] } } - Currently the signature file must be an absolute path. 3. Move common YARA code to yara_utils. - In preparation for the yara table (different from yara_events) I'm moving the common YARA code into a separate place which is shared between the two tables. 4. Add yara table. - This allows you to do things like: ```sql select * from yara where path="/bin/ls" and sigfile="/tmp/foo.sig"; select * from yara where path="/bin/ls" and sig_group="sig_group_1"; ``` - The latter will use the signature grouping from the config. 5. Check for keys not existing. 2015-04-17 20:03:43 +00:00			`table_name("yara")`
			`description("Track YARA matches for files or PIDs.")`
			`schema([`
			`Column("path", TEXT, "The path scanned"),`
			`Column("matches", TEXT, "List of YARA matches"),`
			`Column("count", INTEGER, "Number of YARA matches"),`
			`Column("sig_group", TEXT, "Signature group used"),`
			`Column("sigfile", TEXT, "Signature file used"),`
YARA tests, SQL matching, sigfile loading 1. Minor refactoring. - Generate one row per sigfile or sig_group. - While here, when a signature file fails to compile, VLOG() it. 2. Bring in a couple of YARA tests. Write a couple of tests for YARA functionality. Right now the only tests make sure rules are compiled properly and that rules match where they should and don't match where they shouldn't. 3. Allow sigfiles to be relative to /var/osquery. - Also, only create a row if scanning happened. 4. Add pattern support to yara table. - Also, optimize things so that rules are only compiled once. 2015-04-22 23:41:51 +00:00			`Column("pattern", TEXT, "A pattern which can be used to match file paths"),`
Add tags and strings columns to YARA tables. When strings match they will be populated into the "strings" column of the table. The format is identifier:offset. When a matching rule has tags defined the tags will be put into the "tags" column of the table in a comma separated list. 2015-07-26 19:38:44 +00:00			`Column("strings", TEXT, "Matching strings"),`
			`Column("tags", TEXT, "Matching tags"),`
Major YARA refactor and enhancements 1. Rename yara_matches to yara_events. 2. Add support for Config::getParser(). - This returns a ConfigPluginRef, which is the ConfigParser for the given key. - Being able to get the parser is useful because the YARAConfigParserPlugin uses it to store the compiled rules as an attribute. 3. Finish rename and use ConfigParserPlugin. - Finish the table rename to yara_events. - Use the new ConfigParserPlugin interface to parse the YARA configuration. The file_paths and signatures are stored in the ConfigParserPlugin named "yara" under the key "yara". The rules are compiled and stored as a private attribute of the same ConfigParserPlugin object. Here is an example config using this new structure: { // Description of the YARA feature. "yara": { "signatures": { // Each key is an arbitrary group name to give the signatures listed "sig_group_1": [ "/Users/wxs/foo.sig", "/Users/wxs//bar.sig" ], "sig_group_2": [ "/Users/wxs/baz.sig" ] }, "file_paths": { // Each key is a key from file_paths // The value is a list of signature groups to run when an event fires // These will be watched for and scanned when the event framework // fire off an event to yara_events table "system_binaries": [ "sig_group_1" ], "tmp": [ "sig_group_1", "sig_group_2" ] } }, // Paths to watch for filesystem events "file_paths": { "system_binaries": [ "/usr/bin/%", "/usr/sbin/%" ], "tmp": [ "/Users/wxs/tmp/%%" ] } } - Currently the signature file must be an absolute path. 3. Move common YARA code to yara_utils. - In preparation for the yara table (different from yara_events) I'm moving the common YARA code into a separate place which is shared between the two tables. 4. Add yara table. - This allows you to do things like: ```sql select * from yara where path="/bin/ls" and sigfile="/tmp/foo.sig"; select * from yara where path="/bin/ls" and sig_group="sig_group_1"; ``` - The latter will use the signature grouping from the config. 5. Check for keys not existing. 2015-04-17 20:03:43 +00:00			`])`
			`implementation("yara@genYara")`