Commit Graph

1696 Commits

Author SHA1 Message Date
Teddy Reed
87ebb93bf7 Merge pull request #1045 from theopolis/yara_wip
YARA Set of commits from wxsBSD squashed
2015-04-26 03:14:04 -07:00
Wesley Shields
67bf099207 YARA tests, SQL matching, sigfile loading
1. Minor refactoring.

- Generate one row per sigfile or sig_group.
- While here, when a signature file fails to compile, VLOG() it.

2. Bring in a couple of YARA tests.
Write a couple of tests for YARA functionality. Right now the only tests
make sure rules are compiled properly and that rules match where they
should and don't match where they shouldn't.

3. Allow sigfiles to be relative to /var/osquery.
- Also, only create a row if scanning happened.

4. Add pattern support to yara table.
- Also, optimize things so that rules are only compiled once.
2015-04-26 03:03:48 -07:00
Teddy Reed
fcde6c4bfc Move yara out of core/SDK into additional 2015-04-26 03:01:28 -07:00
Wesley Shields
a9f66fa38b Major YARA refactor and enhancements
1. Rename yara_matches to yara_events.

2. Add support for Config::getParser().
- This returns a ConfigPluginRef, which is the ConfigParser for the
  given key.
- Being able to get the parser is useful because the
  YARAConfigParserPlugin uses it to store the compiled rules as an
  attribute.

3. Finish rename and use ConfigParserPlugin.
- Finish the table rename to yara_events.
- Use the new ConfigParserPlugin interface to parse the YARA
  configuration. The file_paths and signatures are stored in the
  ConfigParserPlugin named "yara" under the key "yara". The rules are
  compiled and stored as a private attribute of the same
  ConfigParserPlugin object.

Here is an example config using this new structure:

{
  // Description of the YARA feature.
  "yara": {
    "signatures": {
      // Each key is an arbitrary group name to give the signatures listed
      "sig_group_1": [ "/Users/wxs/foo.sig", "/Users/wxs//bar.sig" ],
      "sig_group_2": [ "/Users/wxs/baz.sig" ]
    },
    "file_paths": {
      // Each key is a key from file_paths
      // The value is a list of signature groups to run when an event fires
      // These will be watched for and scanned when the event framework
      // fire off an event to yara_events table
      "system_binaries": [ "sig_group_1" ],
      "tmp": [ "sig_group_1", "sig_group_2" ]
    }
  },

  // Paths to watch for filesystem events
  "file_paths": {
    "system_binaries": [ "/usr/bin/%", "/usr/sbin/%" ],
    "tmp": [ "/Users/wxs/tmp/%%" ]
  }
}

- Currently the signature file must be an absolute path.

3. Move common YARA code to yara_utils.
- In preparation for the yara table (different from yara_events) I'm
  moving the common YARA code into a separate place which is shared
  between the two tables.

4. Add yara table.
- This allows you to do things like:

```sql
select * from yara where path="/bin/ls" and sigfile="/tmp/foo.sig";
select * from yara where path="/bin/ls" and sig_group="sig_group_1";
```

- The latter will use the signature grouping from the config.

5. Check for keys not existing.
2015-04-26 03:01:28 -07:00
Mike Arpaia
a3bdc71c83 Merge pull request #1044 from shawndavenport/patch-1
Fix a broken link to the OS X install docs.
2015-04-25 20:18:52 -07:00
Shawn Davenport
abdf273f0a Fix install-osx link. 2015-04-25 19:57:07 -07:00
Teddy Reed
a972b1b0b0 Merge pull request #1027 from sharvilshah/fde_linux
[Implement #933] Add LUKS/dm-crypt disk_encryption support for Linux
2015-04-25 12:43:05 -07:00
Mike Arpaia
f0302cedb6 clang-format in http config plugin 2015-04-25 12:06:48 -07:00
Teddy Reed
5e2ce5c2e9 Merge pull request #1039 from theopolis/fix_flags
Fix dameon flags loading from options
2015-04-25 01:28:30 -07:00
Teddy Reed
b1eee9503a Merge pull request #1021 from theopolis/rpm_files
Add rpm_package_files table
2015-04-25 01:27:55 -07:00
Teddy Reed
13d1ff031b Add rpm_package_files table 2015-04-25 01:18:55 -07:00
Teddy Reed
c2a8187db5 Merge pull request #1042 from theopolis/rhel_easier
Simpler RHEL6 provision
2015-04-24 17:05:36 -07:00
Sharvil Shah
f72dcb5d96 add libcrypysetup-dev library
moved disk_ecryption table spec to crossplatform

link libcryptsetup

implemented get cipher type and cipher_mode:

more idiomatic c++11

no need to explicitly call std::string constructor to convert char * to std::string

update cryptsetup sources for centos

add function prototype for older libcryptsetup which is in centos6

ifdef check for centos6 which uses older libcryptsetup

remove forward declared functions defined in libcryptsetup, stylistic changes
2015-04-24 17:01:14 -07:00
Teddy Reed
5e08b8bf60 Simpler RHEL6 provision 2015-04-24 14:25:59 -07:00
Teddy Reed
b90aeab2fe Fix dameon flags loading from options 2015-04-24 11:37:51 -07:00
Teddy Reed
85b6d979d7 Merge pull request #1035 from steven-hilder/fixCsvFlag
Fix --csv flag in osqueryi shell
2015-04-23 16:57:14 -07:00
Teddy Reed
bfb8f64d37 Merge pull request #1024 from sharvilshah/faster_vagrant
Use all available CPUs when provisioning a Vagrant box
2015-04-23 16:56:03 -07:00
Steven Hilder
0937a92cd1 Fix --csv flag in osqueryi shell 2015-04-23 21:32:14 +00:00
Sharvil Shah
6383a9917f Support setting v.cpus via ENV variable, default fallback to 2
Update docs: mention V_CPUS ENV variable for vagrant

Use OSQUERY_BUILD_CPUS as the ENV var instead of V_CPUS
2015-04-23 14:15:16 -07:00
Javier Marcos
6f447ffedb Merge pull request #1031 from javuto/etc_protocols_table
Adding new table for /etc/protocols
2015-04-22 18:18:03 -07:00
Teddy Reed
328bc062aa Merge pull request #1034 from theopolis/fix_cmake_centos7
Build cmake with gcc to avoid gnu++1y
2015-04-22 18:14:54 -07:00
Teddy Reed
b2dc8b7264 Build cmake with gcc to avoid gnu++1y 2015-04-22 17:58:08 -07:00
Javier Marcos
ddb41ae84a Adding tests to the prototocols table 2015-04-22 17:49:27 -07:00
Javier Marcos
93d2b58b60 Adding new table for /etc/protocols 2015-04-22 15:46:29 -07:00
Mike Arpaia
e1c5b5b596 Merge pull request #1028 from arirubinstein/master
Add a missing Page attribute to fix broken mkdocs build
2015-04-22 11:00:41 -07:00
Ari Rubinstein
7f50484f1d Add a missing Page attribute to fix broken mkdocs build 2015-04-22 10:01:42 -07:00
Teddy Reed
b8a54dbcc9 Merge pull request #1026 from sharvilshah/doc_updates
Minor doc updates
2015-04-22 08:36:55 -07:00
Sharvil Shah
b14874aacd update links to osquery.rtfd and better formatting 2015-04-21 20:36:35 -07:00
Teddy Reed
ab4e78f875 Merge pull request #1005 from theopolis/osx_memmap
Towards OS X memory maps
2015-04-20 22:18:36 -07:00
Mike Arpaia
b409049050 Merge pull request #1010 from marpaia/readme
Changing read the docs URL to be https
2015-04-19 21:59:52 -07:00
Mike Arpaia
8e7d10b956 Changing read the docs URL to be https 2015-04-19 12:49:33 -07:00
Teddy Reed
767ac367b6 Merge pull request #1008 from theopolis/docs_3
Documentation updates, separate config/logging pages
2015-04-19 08:23:07 -07:00
Teddy Reed
8930f9e692 Documentation updates, separate config/logging pages
Mostly minor documentation/wiki/guide fixes.
The breaks down the "using osqueryd" page into more of a summary
of what the daemon does from a schedule/logging perspective.

The bulk of the "using osqueryd" page now exists in the configuration
deployment page and the new "logging" deployment page.
2015-04-18 22:09:25 -07:00
Teddy Reed
b8db4359f3 Merge pull request #1004 from theopolis/fix_bugs
Lint fixes and clang analyze
2015-04-18 14:31:39 -07:00
Teddy Reed
6b9f412caa Towards OS X memory maps 2015-04-18 14:20:40 -07:00
Mitchell Grenier
cf6457ae94 Merge pull request #986 from jedi22/config_remote
Creating interfaces for remote logging
2015-04-17 16:28:15 -07:00
Mitchell Grenier
c47790d87d Creating interfaces for remote logging
First draft of the enrollment plugin and part of the config plugin.

Please comment on potential structure and functionality.

They way it's designed to work is this:
Both the config and logger plugins will call the enroll getKey function. getKey
is in charge of maintaining the state of having a key and doing the enrollment.

If enroll has a key, then it will instantly return doing no HTTP requests, if it
doesn't, then it will try a few times to get a key, if it's not successful, it
fails and returns an empty string, if it succeeds, then it will return the
client enrollment key for the requester to use.
2015-04-17 14:48:46 -07:00
Teddy Reed
c59ce0e4e4 Lint fixes and clang analyze 2015-04-17 09:18:46 -07:00
Javier Marcos
470d63c283 Merge pull request #1002 from javuto/osquery_scheduled_table
Adding osquery_scheduled table
2015-04-16 15:48:33 -07:00
Javier Marcos
1e505fe023 Adding renamed file 2015-04-16 15:29:10 -07:00
Javier Marcos
3c86ecd31c Changing name to osquery_schedule 2015-04-16 15:26:20 -07:00
Javier Marcos
6f2afd7be8 Adding osquery_scheduled table 2015-04-16 14:48:21 -07:00
Teddy Reed
1236f22501 Merge pull request #999 from theopolis/launchd_overrides_2
Add launchd_overrides table
2015-04-16 13:35:22 -07:00
Teddy Reed
cf479abdc8 [Fix #1000] Bump third-party SQLite 3.8.9 2015-04-16 12:53:12 -07:00
Teddy Reed
c9e07ec2ba Add launchd_overrides table 2015-04-15 23:19:23 -07:00
Mike Arpaia
a8506d15e8 Merge pull request #997 from facebook/platform_osquery_info
[#989] Adding distro and version to osquery_info
2015-04-15 21:28:35 -07:00
Javier Marcos
4763d2a5ff Removing commented ifdef 2015-04-15 20:46:06 -07:00
Javier Marcos
7a2716b4b6 Adding distro and version to osquery_info 2015-04-15 16:34:12 -07:00
Teddy Reed
1e584c9b93 Merge pull request #992 from theopolis/debug_target
[Fix #991] Use separate targets for opt/debug builds
2015-04-15 12:19:21 -07:00
Teddy Reed
8fe0a214b2 Merge pull request #994 from theopolis/ubuntu_version
Fix Ubuntu os_version regex
2015-04-15 12:18:46 -07:00