Commit Graph

972 Commits

Author SHA1 Message Date
Teddy Reed
467ecc20ae Merge pull request #1059 from theopolis/shell_improv
Various shell fixups
2015-04-27 17:29:02 -07:00
Teddy Reed
d5b9c0216b Merge pull request #1058 from theopolis/catching_registry
Apply vegas-style rules to call
2015-04-27 17:28:18 -07:00
Teddy Reed
2c5cbdee63 Various shell fixups 2015-04-27 16:40:05 -07:00
Teddy Reed
2b5b9683a4 Apply vegas-style rules to call 2015-04-27 15:08:03 -07:00
Teddy Reed
04f80f1ef3 Merge pull request #1057 from wxsBSD/yara_fix2
Move YARA initialization to setUp().
2015-04-27 14:42:32 -07:00
Teddy Reed
cd7d68c994 Use a fs glob in chrome-extensions searching
Chrome (non-Opera) may use multiple profiles before the
extensions dir. Use a glob before searching for extensions/versions.
2015-04-27 11:36:44 -07:00
Teddy Reed
beb28a3f96 Merge pull request #1051 from theopolis/slim_shell
Remove unpopular features from shell
2015-04-27 10:53:55 -07:00
Wesley Shields
bb392c42f6 Move YARA initialization to setUp().
This was causing a crash when executing a query using the yara table
from the command line, because YARA was never initialized properly, so
the thread index was whatever was left on the stack. Eventually YARA
would attempt to set a rule that matches using this thread index and
would explode in flames.

Fix it by moving the initialization to a place that is always called.
2015-04-27 13:45:20 -04:00
Teddy Reed
be65922569 Fast tests 2015-04-27 09:40:31 -07:00
Teddy Reed
149cc8594b Remove unpopular features from shell.
Remove modes: HTML, Tabs, Explain.
Remove stats (meaningless for virtual tables).

Use the osquery SQLite DB manager within meta and shell SQL
execution to allow registry/extensions non-locking access.

This allows existing runtime DB manipulators to prefer a locking
modifier. Currently these manipulators will fallback to a more
transient db instance. The effect was, no shell-accessible runtime
created virtual tables.
2015-04-26 18:54:27 -07:00
Mike Arpaia
b3540034f8 Merge pull request #1038 from marpaia/file_wildcard
[Fix #1013] wildcard support in file table
2015-04-26 16:10:50 -07:00
Teddy Reed
16447e67d6 [Fix #1040] Check for disabled event subs 2015-04-26 12:15:49 -07:00
Wesley Shields
67bf099207 YARA tests, SQL matching, sigfile loading
1. Minor refactoring.

- Generate one row per sigfile or sig_group.
- While here, when a signature file fails to compile, VLOG() it.

2. Bring in a couple of YARA tests.
Write a couple of tests for YARA functionality. Right now the only tests
make sure rules are compiled properly and that rules match where they
should and don't match where they shouldn't.

3. Allow sigfiles to be relative to /var/osquery.
- Also, only create a row if scanning happened.

4. Add pattern support to yara table.
- Also, optimize things so that rules are only compiled once.
2015-04-26 03:03:48 -07:00
Teddy Reed
fcde6c4bfc Move yara out of core/SDK into additional 2015-04-26 03:01:28 -07:00
Wesley Shields
a9f66fa38b Major YARA refactor and enhancements
1. Rename yara_matches to yara_events.

2. Add support for Config::getParser().
- This returns a ConfigPluginRef, which is the ConfigParser for the
  given key.
- Being able to get the parser is useful because the
  YARAConfigParserPlugin uses it to store the compiled rules as an
  attribute.

3. Finish rename and use ConfigParserPlugin.
- Finish the table rename to yara_events.
- Use the new ConfigParserPlugin interface to parse the YARA
  configuration. The file_paths and signatures are stored in the
  ConfigParserPlugin named "yara" under the key "yara". The rules are
  compiled and stored as a private attribute of the same
  ConfigParserPlugin object.

Here is an example config using this new structure:

{
  // Description of the YARA feature.
  "yara": {
    "signatures": {
      // Each key is an arbitrary group name to give the signatures listed
      "sig_group_1": [ "/Users/wxs/foo.sig", "/Users/wxs//bar.sig" ],
      "sig_group_2": [ "/Users/wxs/baz.sig" ]
    },
    "file_paths": {
      // Each key is a key from file_paths
      // The value is a list of signature groups to run when an event fires
      // These will be watched for and scanned when the event framework
      // fire off an event to yara_events table
      "system_binaries": [ "sig_group_1" ],
      "tmp": [ "sig_group_1", "sig_group_2" ]
    }
  },

  // Paths to watch for filesystem events
  "file_paths": {
    "system_binaries": [ "/usr/bin/%", "/usr/sbin/%" ],
    "tmp": [ "/Users/wxs/tmp/%%" ]
  }
}

- Currently the signature file must be an absolute path.

3. Move common YARA code to yara_utils.
- In preparation for the yara table (different from yara_events) I'm
  moving the common YARA code into a separate place which is shared
  between the two tables.

4. Add yara table.
- This allows you to do things like:

```sql
select * from yara where path="/bin/ls" and sigfile="/tmp/foo.sig";
select * from yara where path="/bin/ls" and sig_group="sig_group_1";
```

- The latter will use the signature grouping from the config.

5. Check for keys not existing.
2015-04-26 03:01:28 -07:00
Teddy Reed
a972b1b0b0 Merge pull request #1027 from sharvilshah/fde_linux
[Implement #933] Add LUKS/dm-crypt disk_encryption support for Linux
2015-04-25 12:43:05 -07:00
Mike Arpaia
f0302cedb6 clang-format in http config plugin 2015-04-25 12:06:48 -07:00
Teddy Reed
5e2ce5c2e9 Merge pull request #1039 from theopolis/fix_flags
Fix dameon flags loading from options
2015-04-25 01:28:30 -07:00
Teddy Reed
b1eee9503a Merge pull request #1021 from theopolis/rpm_files
Add rpm_package_files table
2015-04-25 01:27:55 -07:00
Teddy Reed
13d1ff031b Add rpm_package_files table 2015-04-25 01:18:55 -07:00
Sharvil Shah
f72dcb5d96 add libcrypysetup-dev library
moved disk_ecryption table spec to crossplatform

link libcryptsetup

implemented get cipher type and cipher_mode:

more idiomatic c++11

no need to explicitly call std::string constructor to convert char * to std::string

update cryptsetup sources for centos

add function prototype for older libcryptsetup which is in centos6

ifdef check for centos6 which uses older libcryptsetup

remove forward declared functions defined in libcryptsetup, stylistic changes
2015-04-24 17:01:14 -07:00
Teddy Reed
b90aeab2fe Fix dameon flags loading from options 2015-04-24 11:37:51 -07:00
mike@arpaia.co
7eb4402a78 VLOG instead of WARNING 2015-04-24 08:55:36 -07:00
mike@arpaia.co
dedee94441 Changing the column wildcard to pattern 2015-04-24 08:54:18 -07:00
Mike Arpaia
3db60378f4 [Fix #1013] wildcard support in file table
Now you can run a query like:

```
[localhost] linux (file_wildcard) * ./osquery/osqueryi
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
osquery - being built, with love, at Facebook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Connected to a transient in-memory database.
osquery> select path from file where wildcard = "/home/%/git/osquery/%";
+--------------------------------------------+
| path                                       |
+--------------------------------------------+
| /home/marpaia/git/osquery/.clang-format    |
| /home/marpaia/git/osquery/osquery.thrift   |
| /home/marpaia/git/osquery/PATENTS          |
| /home/marpaia/git/osquery/README.md        |
| /home/marpaia/git/osquery/Vagrantfile      |
| /home/marpaia/git/osquery/CONTRIBUTING.md  |
| /home/marpaia/git/osquery/mkdocs.yml       |
| /home/marpaia/git/osquery/Doxyfile         |
| /home/marpaia/git/osquery/.gitmodules      |
| /home/marpaia/git/osquery/requirements.txt |
| /home/marpaia/git/osquery/Makefile         |
| /home/marpaia/git/osquery/LICENSE          |
| /home/marpaia/git/osquery/.gitignore       |
| /home/marpaia/git/osquery/CMakeLists.txt   |
+--------------------------------------------+
```
2015-04-23 23:39:32 -07:00
Steven Hilder
0937a92cd1 Fix --csv flag in osqueryi shell 2015-04-23 21:32:14 +00:00
Javier Marcos
ddb41ae84a Adding tests to the prototocols table 2015-04-22 17:49:27 -07:00
Javier Marcos
93d2b58b60 Adding new table for /etc/protocols 2015-04-22 15:46:29 -07:00
Teddy Reed
ab4e78f875 Merge pull request #1005 from theopolis/osx_memmap
Towards OS X memory maps
2015-04-20 22:18:36 -07:00
Teddy Reed
b8db4359f3 Merge pull request #1004 from theopolis/fix_bugs
Lint fixes and clang analyze
2015-04-18 14:31:39 -07:00
Teddy Reed
6b9f412caa Towards OS X memory maps 2015-04-18 14:20:40 -07:00
Mitchell Grenier
cf6457ae94 Merge pull request #986 from jedi22/config_remote
Creating interfaces for remote logging
2015-04-17 16:28:15 -07:00
Mitchell Grenier
c47790d87d Creating interfaces for remote logging
First draft of the enrollment plugin and part of the config plugin.

Please comment on potential structure and functionality.

They way it's designed to work is this:
Both the config and logger plugins will call the enroll getKey function. getKey
is in charge of maintaining the state of having a key and doing the enrollment.

If enroll has a key, then it will instantly return doing no HTTP requests, if it
doesn't, then it will try a few times to get a key, if it's not successful, it
fails and returns an empty string, if it succeeds, then it will return the
client enrollment key for the requester to use.
2015-04-17 14:48:46 -07:00
Teddy Reed
c59ce0e4e4 Lint fixes and clang analyze 2015-04-17 09:18:46 -07:00
Javier Marcos
470d63c283 Merge pull request #1002 from javuto/osquery_scheduled_table
Adding osquery_scheduled table
2015-04-16 15:48:33 -07:00
Javier Marcos
1e505fe023 Adding renamed file 2015-04-16 15:29:10 -07:00
Javier Marcos
3c86ecd31c Changing name to osquery_schedule 2015-04-16 15:26:20 -07:00
Javier Marcos
6f2afd7be8 Adding osquery_scheduled table 2015-04-16 14:48:21 -07:00
Teddy Reed
c9e07ec2ba Add launchd_overrides table 2015-04-15 23:19:23 -07:00
Javier Marcos
4763d2a5ff Removing commented ifdef 2015-04-15 20:46:06 -07:00
Javier Marcos
7a2716b4b6 Adding distro and version to osquery_info 2015-04-15 16:34:12 -07:00
Teddy Reed
8fe0a214b2 Merge pull request #994 from theopolis/ubuntu_version
Fix Ubuntu os_version regex
2015-04-15 12:18:46 -07:00
Mike Arpaia
c37be342ec updating wiki link to read the docs 2015-04-15 01:02:58 -07:00
Teddy Reed
f7f1819a67 Fix Ubuntu-version regex 2015-04-14 22:43:44 -06:00
Teddy Reed
e6a436fc51 Merge pull request #988 from theopolis/config_parsers
Add a ConfigParserPlugin to extend config keys
2015-04-14 21:27:56 -07:00
Teddy Reed
2d3de51510 Restrict permissions on RocksDB paths 2015-04-14 21:07:21 -07:00
Teddy Reed
c3958259b8 Add a ConfigParserPlugin to extend config keys 2015-04-14 20:30:06 -07:00
mike@arpaia.co
233f672655 Request template classes
As discussed in the comments of #961. Included is an HTTP transport
(which works for HTTPS also) and a JSON serializer.
2015-04-13 10:32:56 -07:00
Teddy Reed
739d91c361 Performance linting 2015-04-11 19:50:35 -07:00
Teddy Reed
e1f0106710 Various fixes, checks 2015-04-11 15:57:12 -07:00