Commit Graph

992 Commits

Author SHA1 Message Date
Teddy Reed
70e3c190bb Easier build host-based sync 2015-05-05 15:15:45 -07:00
Teddy Reed
cdb112eccb Add a CMake variable for packages 2015-05-04 17:09:09 -07:00
Teddy Reed
d6eb63ae2f Merge pull request #1097 from theopolis/intel_perf_limits
Limit memory checks to worker allocations
2015-05-04 12:14:43 -07:00
Teddy Reed
fa35ee5f7b Merge pull request #1095 from theopolis/raw_sockets
[Fix #1080] Remove netlink, support raw sockets
2015-05-04 12:09:37 -07:00
Teddy Reed
5b60eb9fb8 Limit memory checks to worker allocations 2015-05-04 11:30:25 -07:00
Teddy Reed
893f678403 Linting and asan fixups 2015-05-04 11:00:21 -07:00
Teddy Reed
7da8b6f68a [Fix #1080] Remove netlink, support raw sockets 2015-05-04 10:57:49 -07:00
Teddy Reed
c63bf0451a Various exception hardening 2015-05-03 14:18:20 -07:00
Teddy Reed
e01a73b4f3 Schedule monitoring, doc updates, logger plugin fixes 2015-05-03 11:54:15 -07:00
Mike Arpaia
3311e17c06 [FIX #1082] Removing cpp-netlib from SDK 2015-05-01 14:00:10 -07:00
Sharvil Shah
2735e731de Implement --disable_tables runtime flag 2015-04-30 01:41:01 -07:00
Teddy Reed
13c4e27629 Merge pull request #1067 from theopolis/snapshots
[#966] Allow snapshot scheduled items
2015-04-29 18:47:24 -07:00
Teddy Reed
a4c3a869de Merge pull request #1073 from theopolis/file_events
Rename file_changes to file_events
2015-04-29 18:43:57 -07:00
Javier Marcos
e83b813399 Support RHEL6
This enables support for building osquery in RHEL6
2015-04-29 18:31:13 -07:00
Teddy Reed
9658d4377c Rename file_changes to file_events 2015-04-29 16:27:29 -07:00
Teddy Reed
c012d1c1d3 Merge pull request #1070 from wxsBSD/yara_relative
Make YARA rule compiling handle relative paths.
2015-04-29 15:56:17 -07:00
Teddy Reed
b66a350526 Allow snapshot scheduled items 2015-04-29 15:55:00 -07:00
Teddy Reed
d0bbb0bc4f Towards safer and shuffled unittests 2015-04-29 14:43:27 -07:00
Wesley Shields
546d298196 Move yara relative paths to /etc/osquery/yara. 2015-04-29 10:16:11 -04:00
Wesley Shields
82123d14d1 Make YARA rule compiling handle relative paths.
Previously this only existed in the yara table, but it now exists in the
yara config parser land, which will compile signature groups upon
update. Now your signature groups can reference signature files using
paths relative to /var/osquery.
2015-04-28 23:06:02 -04:00
Teddy Reed
467ecc20ae Merge pull request #1059 from theopolis/shell_improv
Various shell fixups
2015-04-27 17:29:02 -07:00
Teddy Reed
d5b9c0216b Merge pull request #1058 from theopolis/catching_registry
Apply vegas-style rules to call
2015-04-27 17:28:18 -07:00
Teddy Reed
2c5cbdee63 Various shell fixups 2015-04-27 16:40:05 -07:00
Teddy Reed
2b5b9683a4 Apply vegas-style rules to call 2015-04-27 15:08:03 -07:00
Teddy Reed
04f80f1ef3 Merge pull request #1057 from wxsBSD/yara_fix2
Move YARA initialization to setUp().
2015-04-27 14:42:32 -07:00
Teddy Reed
cd7d68c994 Use a fs glob in chrome-extensions searching
Chrome (non-Opera) may use multiple profiles before the
extensions dir. Use a glob before searching for extensions/versions.
2015-04-27 11:36:44 -07:00
Teddy Reed
beb28a3f96 Merge pull request #1051 from theopolis/slim_shell
Remove unpopular features from shell
2015-04-27 10:53:55 -07:00
Wesley Shields
bb392c42f6 Move YARA initialization to setUp().
This was causing a crash when executing a query using the yara table
from the command line, because YARA was never initialized properly, so
the thread index was whatever was left on the stack. Eventually YARA
would attempt to set a rule that matches using this thread index and
would explode in flames.

Fix it by moving the initialization to a place that is always called.
2015-04-27 13:45:20 -04:00
Teddy Reed
be65922569 Fast tests 2015-04-27 09:40:31 -07:00
Teddy Reed
149cc8594b Remove unpopular features from shell.
Remove modes: HTML, Tabs, Explain.
Remove stats (meaningless for virtual tables).

Use the osquery SQLite DB manager within meta and shell SQL
execution to allow registry/extensions non-locking access.

This allows existing runtime DB manipulators to prefer a locking
modifier. Currently these manipulators will fallback to a more
transient db instance. The effect was, no shell-accessible runtime
created virtual tables.
2015-04-26 18:54:27 -07:00
Mike Arpaia
b3540034f8 Merge pull request #1038 from marpaia/file_wildcard
[Fix #1013] wildcard support in file table
2015-04-26 16:10:50 -07:00
Teddy Reed
16447e67d6 [Fix #1040] Check for disabled event subs 2015-04-26 12:15:49 -07:00
Wesley Shields
67bf099207 YARA tests, SQL matching, sigfile loading
1. Minor refactoring.

- Generate one row per sigfile or sig_group.
- While here, when a signature file fails to compile, VLOG() it.

2. Bring in a couple of YARA tests.
Write a couple of tests for YARA functionality. Right now the only tests
make sure rules are compiled properly and that rules match where they
should and don't match where they shouldn't.

3. Allow sigfiles to be relative to /var/osquery.
- Also, only create a row if scanning happened.

4. Add pattern support to yara table.
- Also, optimize things so that rules are only compiled once.
2015-04-26 03:03:48 -07:00
Teddy Reed
fcde6c4bfc Move yara out of core/SDK into additional 2015-04-26 03:01:28 -07:00
Wesley Shields
a9f66fa38b Major YARA refactor and enhancements
1. Rename yara_matches to yara_events.

2. Add support for Config::getParser().
- This returns a ConfigPluginRef, which is the ConfigParser for the
  given key.
- Being able to get the parser is useful because the
  YARAConfigParserPlugin uses it to store the compiled rules as an
  attribute.

3. Finish rename and use ConfigParserPlugin.
- Finish the table rename to yara_events.
- Use the new ConfigParserPlugin interface to parse the YARA
  configuration. The file_paths and signatures are stored in the
  ConfigParserPlugin named "yara" under the key "yara". The rules are
  compiled and stored as a private attribute of the same
  ConfigParserPlugin object.

Here is an example config using this new structure:

{
  // Description of the YARA feature.
  "yara": {
    "signatures": {
      // Each key is an arbitrary group name to give the signatures listed
      "sig_group_1": [ "/Users/wxs/foo.sig", "/Users/wxs//bar.sig" ],
      "sig_group_2": [ "/Users/wxs/baz.sig" ]
    },
    "file_paths": {
      // Each key is a key from file_paths
      // The value is a list of signature groups to run when an event fires
      // These will be watched for and scanned when the event framework
      // fire off an event to yara_events table
      "system_binaries": [ "sig_group_1" ],
      "tmp": [ "sig_group_1", "sig_group_2" ]
    }
  },

  // Paths to watch for filesystem events
  "file_paths": {
    "system_binaries": [ "/usr/bin/%", "/usr/sbin/%" ],
    "tmp": [ "/Users/wxs/tmp/%%" ]
  }
}

- Currently the signature file must be an absolute path.

3. Move common YARA code to yara_utils.
- In preparation for the yara table (different from yara_events) I'm
  moving the common YARA code into a separate place which is shared
  between the two tables.

4. Add yara table.
- This allows you to do things like:

```sql
select * from yara where path="/bin/ls" and sigfile="/tmp/foo.sig";
select * from yara where path="/bin/ls" and sig_group="sig_group_1";
```

- The latter will use the signature grouping from the config.

5. Check for keys not existing.
2015-04-26 03:01:28 -07:00
Teddy Reed
a972b1b0b0 Merge pull request #1027 from sharvilshah/fde_linux
[Implement #933] Add LUKS/dm-crypt disk_encryption support for Linux
2015-04-25 12:43:05 -07:00
Mike Arpaia
f0302cedb6 clang-format in http config plugin 2015-04-25 12:06:48 -07:00
Teddy Reed
5e2ce5c2e9 Merge pull request #1039 from theopolis/fix_flags
Fix dameon flags loading from options
2015-04-25 01:28:30 -07:00
Teddy Reed
b1eee9503a Merge pull request #1021 from theopolis/rpm_files
Add rpm_package_files table
2015-04-25 01:27:55 -07:00
Teddy Reed
13d1ff031b Add rpm_package_files table 2015-04-25 01:18:55 -07:00
Sharvil Shah
f72dcb5d96 add libcrypysetup-dev library
moved disk_ecryption table spec to crossplatform

link libcryptsetup

implemented get cipher type and cipher_mode:

more idiomatic c++11

no need to explicitly call std::string constructor to convert char * to std::string

update cryptsetup sources for centos

add function prototype for older libcryptsetup which is in centos6

ifdef check for centos6 which uses older libcryptsetup

remove forward declared functions defined in libcryptsetup, stylistic changes
2015-04-24 17:01:14 -07:00
Teddy Reed
b90aeab2fe Fix dameon flags loading from options 2015-04-24 11:37:51 -07:00
mike@arpaia.co
7eb4402a78 VLOG instead of WARNING 2015-04-24 08:55:36 -07:00
mike@arpaia.co
dedee94441 Changing the column wildcard to pattern 2015-04-24 08:54:18 -07:00
Mike Arpaia
3db60378f4 [Fix #1013] wildcard support in file table
Now you can run a query like:

```
[localhost] linux (file_wildcard) * ./osquery/osqueryi
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
osquery - being built, with love, at Facebook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Connected to a transient in-memory database.
osquery> select path from file where wildcard = "/home/%/git/osquery/%";
+--------------------------------------------+
| path                                       |
+--------------------------------------------+
| /home/marpaia/git/osquery/.clang-format    |
| /home/marpaia/git/osquery/osquery.thrift   |
| /home/marpaia/git/osquery/PATENTS          |
| /home/marpaia/git/osquery/README.md        |
| /home/marpaia/git/osquery/Vagrantfile      |
| /home/marpaia/git/osquery/CONTRIBUTING.md  |
| /home/marpaia/git/osquery/mkdocs.yml       |
| /home/marpaia/git/osquery/Doxyfile         |
| /home/marpaia/git/osquery/.gitmodules      |
| /home/marpaia/git/osquery/requirements.txt |
| /home/marpaia/git/osquery/Makefile         |
| /home/marpaia/git/osquery/LICENSE          |
| /home/marpaia/git/osquery/.gitignore       |
| /home/marpaia/git/osquery/CMakeLists.txt   |
+--------------------------------------------+
```
2015-04-23 23:39:32 -07:00
Steven Hilder
0937a92cd1 Fix --csv flag in osqueryi shell 2015-04-23 21:32:14 +00:00
Javier Marcos
ddb41ae84a Adding tests to the prototocols table 2015-04-22 17:49:27 -07:00
Javier Marcos
93d2b58b60 Adding new table for /etc/protocols 2015-04-22 15:46:29 -07:00
Teddy Reed
ab4e78f875 Merge pull request #1005 from theopolis/osx_memmap
Towards OS X memory maps
2015-04-20 22:18:36 -07:00
Teddy Reed
b8db4359f3 Merge pull request #1004 from theopolis/fix_bugs
Lint fixes and clang analyze
2015-04-18 14:31:39 -07:00