Commit Graph

1095 Commits

Author SHA1 Message Date
Teddy Reed
4a6c002f62 Allow unit tests execs from project root 2015-05-23 13:12:31 -07:00
Teddy Reed
5969ae4fbf Clean up TLS-version from OpenSSL detection 2015-05-23 13:04:36 -07:00
Teddy Reed
700384dedc Minify tables namespace, extra CMake macros 2015-05-22 10:29:04 -07:00
Javier Marcos
9a4f611baf Merge pull request #1155 from javuto/osquery_packs_table
Osquery packs table
2015-05-21 20:32:45 -07:00
Javier Marcos
f86b2bc6f3 Adding checks to avoid duplicated queries in the schedule 2015-05-21 19:23:38 -07:00
Mike Arpaia
6f30c40041 Merge pull request #1152 from sharvilshah/xattr_parse_where_from
More thorough where_from parsing in extended_attributes
2015-05-21 16:32:32 -07:00
Javier Marcos
2b834a401a Fixing problem with extensions test, utility tables were added to core 2015-05-21 14:10:20 -07:00
Javier Marcos
886ad6e928 Added table for the packs and check for already scheduled queries 2015-05-21 13:42:45 -07:00
Sharvil Shah
a216ef2886 Use CoreServices Metadata API to parse kMDItemWhereFroms for file xattrs and now includes non-browser values too 2015-05-20 10:50:25 -07:00
Teddy Reed
4ff2fc1db2 Merge pull request #1151 from theopolis/crontab-fix
Include several search paths for user contabs
2015-05-20 10:47:32 -07:00
Javier Marcos
c6855fab43 Table for osquery packs 2015-05-19 18:44:28 -07:00
Teddy Reed
b3338dc5d2 Merge pull request #1146 from theopolis/tls
Towards TLS config/logging
2015-05-19 17:17:04 -07:00
Teddy Reed
2a1f496cc5 Towards TLS config/logging 2015-05-19 17:05:55 -07:00
Teddy Reed
983d107fe6 Search for cronstabs in /cron and /cron/crontabs 2015-05-19 15:51:03 -07:00
Ryan Steinmetz
949f84f3a8 Add mounts table support under FreeBSD
Cleanup blacklist entries for FreeBSD (mounts/users/groups)
2015-05-19 15:33:06 -07:00
Javier Marcos
65e6e38e0f Merge pull request #1143 from javuto/pack_config_changes
Support to load query packs as scheduled queries
2015-05-16 15:37:27 -07:00
Javier Marcos
47e680e825 Adding tests and implementing version checker 2015-05-15 22:25:19 -07:00
Teddy Reed
525c584a0b Merge pull request #1141 from theopolis/static_cryptsetup
Build libcryptsetup statically
2015-05-14 22:33:56 -07:00
Teddy Reed
9ee839b265 Build libcryptsetup statically 2015-05-14 19:36:00 -07:00
Javier Marcos
aa27159bb8 Proper update of the schedule and iterate all the packs 2015-05-14 17:20:00 -07:00
Javier Marcos
e170692db6 Top level key is packs 2015-05-13 23:10:44 -07:00
Javier Marcos
4d8b05d861 Adding parsed packs to schedule 2015-05-13 21:19:54 -07:00
Javier Marcos
9e9ab079ec Adding support for packs in configuration files 2015-05-13 13:55:01 -07:00
Blake Frantz
3a49fc46c8 Merge remote-tracking branch 'upstream/master' 2015-05-13 07:38:41 -07:00
Ryan Steinmetz
0777fa5fe2 - Add users/groups support for FreeBSD 2015-05-12 23:47:20 -07:00
Wesley Shields
81eac8e89a Fix build on FreeBSD. 2015-05-12 19:13:43 +00:00
Mike Arpaia
fff36af0af Removing trailing whitespace 2015-05-11 23:31:13 -07:00
Blake Frantz
805db480c5 Merge remote-tracking branch 'upstream/master' 2015-05-11 16:08:59 -07:00
Teddy Reed
5b43067c98 Merge pull request #1130 from theopolis/patch-134
[Fix #1125 #1126] Flag padding checks, config_check tests
2015-05-11 13:43:36 -07:00
Teddy Reed
771ed4da2f [Fix #1125 #1126] Flag padding checks, config_check tests 2015-05-11 10:37:16 -07:00
Mike Arpaia
25bd6e7b70 [Fix #1132] Headers to /usr/local/include 2015-05-11 09:36:15 -07:00
Blake Frantz
2e865a69d6 Merge remote-tracking branch 'upstream/master' 2015-05-10 14:38:33 -07:00
Blake Frantz
2c4ae6758a initial commit for adding support for amazon linux 2015.03 2015-05-10 11:42:30 -07:00
Teddy Reed
8235fd155f Merge pull request #1122 from theopolis/relax_deps
Relaxing iptables, EL-deps
2015-05-09 23:52:28 -07:00
Teddy Reed
3e9f40f73f [Fix #1121] Minify shell table/schema, add meta tests 2015-05-09 19:48:28 -07:00
Teddy Reed
98b52c39a1 elaxing iptables, EL-deps 2015-05-09 18:16:13 -07:00
Teddy Reed
b5be0212e2 Merge pull request #1120 from theopolis/iptables_best
Adding new table to display iptables filters, chains and rules
2015-05-08 20:10:34 -07:00
Javier Marcos
4f21090fb8 Adding new table to display iptables filters, chains and rules
Patching headers to avoid void pointers
Adding test for parsing ipt_ip entries
2015-05-08 19:11:49 -07:00
Teddy Reed
1de7cfb331 Use CMake find_package for python, fix ifaddrs on FreeBSD 2015-05-08 18:49:01 -07:00
Teddy Reed
24a638eaaf Remove cpp-netlib from make install 2015-05-08 14:00:09 -07:00
Teddy Reed
434ace85d5 Merge pull request #1113 from theopolis/http_tests
[Fix #1048] Cleaner additional tests
2015-05-08 11:54:25 -07:00
Teddy Reed
258dd62b24 Merge pull request #1114 from theopolis/rhel_centos_tables
RHEL table parity with CENTOS
2015-05-08 11:54:20 -07:00
Teddy Reed
bf1de3b95e Merge pull request #1110 from theopolis/build_freebsd
Towards building on FreeBSD/ports
2015-05-08 10:53:07 -07:00
Teddy Reed
6919065b4b RHEL table parity with CENTOS 2015-05-07 23:23:32 -07:00
Teddy Reed
c7b9114975 Towards building on FreeBSD/ports 2015-05-07 23:12:30 -07:00
Teddy Reed
311f9bd55f Cleaner additional tests 2015-05-07 22:07:14 -07:00
Teddy Reed
a7daa0ace5 Apply a safe permissions check to worker 2015-05-07 00:19:10 -07:00
Teddy Reed
a64270f324 Merge pull request #1106 from theopolis/dispatcher_hardening
Harden extensions/dispatcher tests
2015-05-06 21:07:46 -07:00
Teddy Reed
c50838922f Merge pull request #1102 from theopolis/sync_builds
Easier build host-based sync
2015-05-06 21:06:53 -07:00
Teddy Reed
ee872d3fbe Harden events tests 2015-05-06 20:33:39 -07:00
Jack Naglieri
8e3e7ef7be Fixed crontab parsing issue in RHEL 6.5.X 2015-05-06 13:03:34 -07:00
Teddy Reed
23933cefe8 Harden extensions/dispatcher tests 2015-05-05 23:34:10 -07:00
Teddy Reed
e6c838131b Limit the number of RocksDB log files 2015-05-05 16:14:24 -07:00
Teddy Reed
70e3c190bb Easier build host-based sync 2015-05-05 15:15:45 -07:00
Teddy Reed
cdb112eccb Add a CMake variable for packages 2015-05-04 17:09:09 -07:00
Teddy Reed
d6eb63ae2f Merge pull request #1097 from theopolis/intel_perf_limits
Limit memory checks to worker allocations
2015-05-04 12:14:43 -07:00
Teddy Reed
fa35ee5f7b Merge pull request #1095 from theopolis/raw_sockets
[Fix #1080] Remove netlink, support raw sockets
2015-05-04 12:09:37 -07:00
Teddy Reed
5b60eb9fb8 Limit memory checks to worker allocations 2015-05-04 11:30:25 -07:00
Teddy Reed
893f678403 Linting and asan fixups 2015-05-04 11:00:21 -07:00
Teddy Reed
7da8b6f68a [Fix #1080] Remove netlink, support raw sockets 2015-05-04 10:57:49 -07:00
Teddy Reed
c63bf0451a Various exception hardening 2015-05-03 14:18:20 -07:00
Teddy Reed
e01a73b4f3 Schedule monitoring, doc updates, logger plugin fixes 2015-05-03 11:54:15 -07:00
Mike Arpaia
3311e17c06 [FIX #1082] Removing cpp-netlib from SDK 2015-05-01 14:00:10 -07:00
Sharvil Shah
2735e731de Implement --disable_tables runtime flag 2015-04-30 01:41:01 -07:00
Teddy Reed
13c4e27629 Merge pull request #1067 from theopolis/snapshots
[#966] Allow snapshot scheduled items
2015-04-29 18:47:24 -07:00
Teddy Reed
a4c3a869de Merge pull request #1073 from theopolis/file_events
Rename file_changes to file_events
2015-04-29 18:43:57 -07:00
Javier Marcos
e83b813399 Support RHEL6
This enables support for building osquery in RHEL6
2015-04-29 18:31:13 -07:00
Teddy Reed
9658d4377c Rename file_changes to file_events 2015-04-29 16:27:29 -07:00
Teddy Reed
c012d1c1d3 Merge pull request #1070 from wxsBSD/yara_relative
Make YARA rule compiling handle relative paths.
2015-04-29 15:56:17 -07:00
Teddy Reed
b66a350526 Allow snapshot scheduled items 2015-04-29 15:55:00 -07:00
Teddy Reed
d0bbb0bc4f Towards safer and shuffled unittests 2015-04-29 14:43:27 -07:00
Wesley Shields
546d298196 Move yara relative paths to /etc/osquery/yara. 2015-04-29 10:16:11 -04:00
Wesley Shields
82123d14d1 Make YARA rule compiling handle relative paths.
Previously this only existed in the yara table, but it now exists in the
yara config parser land, which will compile signature groups upon
update. Now your signature groups can reference signature files using
paths relative to /var/osquery.
2015-04-28 23:06:02 -04:00
Teddy Reed
467ecc20ae Merge pull request #1059 from theopolis/shell_improv
Various shell fixups
2015-04-27 17:29:02 -07:00
Teddy Reed
d5b9c0216b Merge pull request #1058 from theopolis/catching_registry
Apply vegas-style rules to call
2015-04-27 17:28:18 -07:00
Teddy Reed
2c5cbdee63 Various shell fixups 2015-04-27 16:40:05 -07:00
Teddy Reed
2b5b9683a4 Apply vegas-style rules to call 2015-04-27 15:08:03 -07:00
Teddy Reed
04f80f1ef3 Merge pull request #1057 from wxsBSD/yara_fix2
Move YARA initialization to setUp().
2015-04-27 14:42:32 -07:00
Teddy Reed
cd7d68c994 Use a fs glob in chrome-extensions searching
Chrome (non-Opera) may use multiple profiles before the
extensions dir. Use a glob before searching for extensions/versions.
2015-04-27 11:36:44 -07:00
Teddy Reed
beb28a3f96 Merge pull request #1051 from theopolis/slim_shell
Remove unpopular features from shell
2015-04-27 10:53:55 -07:00
Wesley Shields
bb392c42f6 Move YARA initialization to setUp().
This was causing a crash when executing a query using the yara table
from the command line, because YARA was never initialized properly, so
the thread index was whatever was left on the stack. Eventually YARA
would attempt to set a rule that matches using this thread index and
would explode in flames.

Fix it by moving the initialization to a place that is always called.
2015-04-27 13:45:20 -04:00
Teddy Reed
be65922569 Fast tests 2015-04-27 09:40:31 -07:00
Teddy Reed
149cc8594b Remove unpopular features from shell.
Remove modes: HTML, Tabs, Explain.
Remove stats (meaningless for virtual tables).

Use the osquery SQLite DB manager within meta and shell SQL
execution to allow registry/extensions non-locking access.

This allows existing runtime DB manipulators to prefer a locking
modifier. Currently these manipulators will fallback to a more
transient db instance. The effect was, no shell-accessible runtime
created virtual tables.
2015-04-26 18:54:27 -07:00
Mike Arpaia
b3540034f8 Merge pull request #1038 from marpaia/file_wildcard
[Fix #1013] wildcard support in file table
2015-04-26 16:10:50 -07:00
Teddy Reed
16447e67d6 [Fix #1040] Check for disabled event subs 2015-04-26 12:15:49 -07:00
Wesley Shields
67bf099207 YARA tests, SQL matching, sigfile loading
1. Minor refactoring.

- Generate one row per sigfile or sig_group.
- While here, when a signature file fails to compile, VLOG() it.

2. Bring in a couple of YARA tests.
Write a couple of tests for YARA functionality. Right now the only tests
make sure rules are compiled properly and that rules match where they
should and don't match where they shouldn't.

3. Allow sigfiles to be relative to /var/osquery.
- Also, only create a row if scanning happened.

4. Add pattern support to yara table.
- Also, optimize things so that rules are only compiled once.
2015-04-26 03:03:48 -07:00
Teddy Reed
fcde6c4bfc Move yara out of core/SDK into additional 2015-04-26 03:01:28 -07:00
Wesley Shields
a9f66fa38b Major YARA refactor and enhancements
1. Rename yara_matches to yara_events.

2. Add support for Config::getParser().
- This returns a ConfigPluginRef, which is the ConfigParser for the
  given key.
- Being able to get the parser is useful because the
  YARAConfigParserPlugin uses it to store the compiled rules as an
  attribute.

3. Finish rename and use ConfigParserPlugin.
- Finish the table rename to yara_events.
- Use the new ConfigParserPlugin interface to parse the YARA
  configuration. The file_paths and signatures are stored in the
  ConfigParserPlugin named "yara" under the key "yara". The rules are
  compiled and stored as a private attribute of the same
  ConfigParserPlugin object.

Here is an example config using this new structure:

{
  // Description of the YARA feature.
  "yara": {
    "signatures": {
      // Each key is an arbitrary group name to give the signatures listed
      "sig_group_1": [ "/Users/wxs/foo.sig", "/Users/wxs//bar.sig" ],
      "sig_group_2": [ "/Users/wxs/baz.sig" ]
    },
    "file_paths": {
      // Each key is a key from file_paths
      // The value is a list of signature groups to run when an event fires
      // These will be watched for and scanned when the event framework
      // fire off an event to yara_events table
      "system_binaries": [ "sig_group_1" ],
      "tmp": [ "sig_group_1", "sig_group_2" ]
    }
  },

  // Paths to watch for filesystem events
  "file_paths": {
    "system_binaries": [ "/usr/bin/%", "/usr/sbin/%" ],
    "tmp": [ "/Users/wxs/tmp/%%" ]
  }
}

- Currently the signature file must be an absolute path.

3. Move common YARA code to yara_utils.
- In preparation for the yara table (different from yara_events) I'm
  moving the common YARA code into a separate place which is shared
  between the two tables.

4. Add yara table.
- This allows you to do things like:

```sql
select * from yara where path="/bin/ls" and sigfile="/tmp/foo.sig";
select * from yara where path="/bin/ls" and sig_group="sig_group_1";
```

- The latter will use the signature grouping from the config.

5. Check for keys not existing.
2015-04-26 03:01:28 -07:00
Teddy Reed
a972b1b0b0 Merge pull request #1027 from sharvilshah/fde_linux
[Implement #933] Add LUKS/dm-crypt disk_encryption support for Linux
2015-04-25 12:43:05 -07:00
Mike Arpaia
f0302cedb6 clang-format in http config plugin 2015-04-25 12:06:48 -07:00
Teddy Reed
5e2ce5c2e9 Merge pull request #1039 from theopolis/fix_flags
Fix dameon flags loading from options
2015-04-25 01:28:30 -07:00
Teddy Reed
b1eee9503a Merge pull request #1021 from theopolis/rpm_files
Add rpm_package_files table
2015-04-25 01:27:55 -07:00
Teddy Reed
13d1ff031b Add rpm_package_files table 2015-04-25 01:18:55 -07:00
Sharvil Shah
f72dcb5d96 add libcrypysetup-dev library
moved disk_ecryption table spec to crossplatform

link libcryptsetup

implemented get cipher type and cipher_mode:

more idiomatic c++11

no need to explicitly call std::string constructor to convert char * to std::string

update cryptsetup sources for centos

add function prototype for older libcryptsetup which is in centos6

ifdef check for centos6 which uses older libcryptsetup

remove forward declared functions defined in libcryptsetup, stylistic changes
2015-04-24 17:01:14 -07:00
Teddy Reed
b90aeab2fe Fix dameon flags loading from options 2015-04-24 11:37:51 -07:00
mike@arpaia.co
7eb4402a78 VLOG instead of WARNING 2015-04-24 08:55:36 -07:00
mike@arpaia.co
dedee94441 Changing the column wildcard to pattern 2015-04-24 08:54:18 -07:00
Mike Arpaia
3db60378f4 [Fix #1013] wildcard support in file table
Now you can run a query like:

```
[localhost] linux (file_wildcard) * ./osquery/osqueryi
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
osquery - being built, with love, at Facebook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Connected to a transient in-memory database.
osquery> select path from file where wildcard = "/home/%/git/osquery/%";
+--------------------------------------------+
| path                                       |
+--------------------------------------------+
| /home/marpaia/git/osquery/.clang-format    |
| /home/marpaia/git/osquery/osquery.thrift   |
| /home/marpaia/git/osquery/PATENTS          |
| /home/marpaia/git/osquery/README.md        |
| /home/marpaia/git/osquery/Vagrantfile      |
| /home/marpaia/git/osquery/CONTRIBUTING.md  |
| /home/marpaia/git/osquery/mkdocs.yml       |
| /home/marpaia/git/osquery/Doxyfile         |
| /home/marpaia/git/osquery/.gitmodules      |
| /home/marpaia/git/osquery/requirements.txt |
| /home/marpaia/git/osquery/Makefile         |
| /home/marpaia/git/osquery/LICENSE          |
| /home/marpaia/git/osquery/.gitignore       |
| /home/marpaia/git/osquery/CMakeLists.txt   |
+--------------------------------------------+
```
2015-04-23 23:39:32 -07:00
Steven Hilder
0937a92cd1 Fix --csv flag in osqueryi shell 2015-04-23 21:32:14 +00:00
Javier Marcos
ddb41ae84a Adding tests to the prototocols table 2015-04-22 17:49:27 -07:00