Commit Graph

1242 Commits

Author SHA1 Message Date
Mike Arpaia
fb2f33d770 Removing the ptvalue typedef
Removing the ptvalue typedef in favor of just using `pt::ptree::value_type`
2015-09-02 12:50:24 -07:00
Mike Arpaia
f92fa761f7 Merge pull request #1484 from theopolis/faster_plist
Add plist parsing benchmarks and refactor slightly for perf wins
2015-09-02 12:48:44 -07:00
Mike Arpaia
a140333441 [fix #1390] query pack re-org
This commit contains the features specified in #1390 as well as a
refactoring of the general osquery configuration code.

The API for the config plugins hasn't changed, although now there's a
`genPack` method that config plugins can implement. If a plugin doesn't
implement `genPack`, then the map<string, string> format cannot be used.
The default config plugin, the filesystem plugin, now implements
`genPack`, so existing query packs code will continue to work as it
always has.

Now many other config plugins can implement custom pack handling for
what makes sense in their context. `genPacks` is not a pure virtual, so
it doesn't have to be implemented in your plugin if you don't want to
use it. Also, more importantly, all config plugins can use the standard
inline pack format if they want to use query packs. Which is awesome.

For more information, refer to #1390, the documentation and the doxygen
comments included with this pull requests, as well as the following
example config which is now supported, regardless of what config plugin
you're using:

```json
{
  "options": {
    "enable_monitor": "true"
  },
  "packs": {
    "core_os_monitoring": {
        "version": "1.4.5",
        "discovery": [
          "select pid from processes where name like '%osqueryd%';"
        ],
        "queries": {
          "kernel_modules": {
              "query": "SELECT name, size FROM kernel_modules;",
              "interval": 600
          },
          "system_controls": {
              "query": "SELECT * FROM system_controls;",
              "interval": 600,
              "snapshot": true,
          },
          "usb_devices": {
              "query": "SELECT * FROM usb_devices;",
              "interval": 600
          }
        }
    },
    "osquery_internal_info": {
        "version": "1.4.5",
        "discovery": [
          "select pid from processes where name like '%osqueryd%';"
        ],
        "queries": {
          "info": {
              "query": "select i.*, p.resident_size, p.user_time, p.system_time, time.minutes as counter from osquery_info i, processes p, time where p.pid = i.pid;",
              "interval": 60,
              "snapshot": true
          },
          "registry": {
              "query": "SELECT * FROM osquery_registry;",
              "interval": 600,
              "snapshot": true
          },
          "schedule": {
              "query": "select name, interval, executions, output_size, wall_time, (user_time/executions) as avg_user_time, (system_time/executions) as avg_system_time, average_memory from osquery_schedule;",
              "interval": 60,
              "snapshot": true
          }
        }
    }
  }
}
```

The `osquery_packs` table was modified to remove the superfluous
columns which could already have been found in `osquery_schedule`. Two
more columns were added in their place, representing stats about pack's
discovery query execution history.

Notably, the internal API for the `osquery::Config` class has changed
rather dramatically as apart of the refactoring. We think this is an
improvement. While strictly adhering to the osquery config plugin
interface will have avoided any compatibility errors, advanced users may
notice compilation errors if they access config data directly. All
internal users of the config have obviously been updated. Yet another
reason to merge your code into mainline; we update it for you when we
refactor!
2015-09-02 10:56:26 -07:00
Teddy Reed
5f56490835 Add plist parsing benchmarks and refactor slightly for perf wins 2015-09-01 18:56:09 -07:00
Teddy Reed
4dd77a43a7 Remove cpp-netlib from third-party, prefer deps-build 2015-08-31 09:27:01 -07:00
Teddy Reed
776de9c4d1 Merge pull request #1477 from theopolis/table_xp_meta
XProtect meta virtual table and safari_extensions column additions
2015-08-30 21:31:35 -07:00
Teddy Reed
906d19927f [#1418] Use libarchive to parse Safari extension bundles 2015-08-29 23:59:41 -07:00
Teddy Reed
9ca040c54f Merge pull request #1478 from sharvilshah/shell_cleanups
Shell Cleanups
2015-08-29 23:52:55 -07:00
Sharvil Shah
fc6865b8a9 Remove unused functions/macros/variables from shell and misc hardening 2015-08-29 22:08:01 -07:00
Teddy Reed
cd1d39b323 Merge pull request #1407 from theopolis/tls_customization
Add 'hidden' flags to customize TLS plugins
2015-08-28 17:21:49 -07:00
Javier Marcos
74be3d1da0 Removing dots at the end of log entries 2015-08-28 16:50:44 -07:00
Javier Marcos
086ab40f83 Merge pull request #1473 from javuto/this_is_real_magic
Adding magic table to check for libmagic data
2015-08-28 14:03:10 -07:00
Teddy Reed
0e16f56c8d Add 'hidden' flags to customize TLS plugins 2015-08-28 12:57:53 -07:00
Javier Marcos
1a50977a23 Adding magic table to check for libmagic data 2015-08-28 12:49:46 -07:00
Teddy Reed
88c7ad35a2 Merge pull request #1471 from theopolis/process_start_fix
[Fix #1453] Use second precision for process start times
2015-08-28 11:48:25 -07:00
Teddy Reed
2433d9e06c [#1418] Include XProtect's meta list of plugin versions, and blacklisted extensions 2015-08-28 11:46:21 -07:00
Teddy Reed
014e504fba [Fix #1432] Improve OS X USB device reporting 2015-08-27 16:36:54 -07:00
Teddy Reed
3c114c3439 [Fix #1453] Use second precision for process start times 2015-08-27 15:47:06 -07:00
Teddy Reed
d47cac7434 Merge pull request #1461 from blackfist/master
Attempts to add a cli flag --enroll_secret_env
2015-08-26 14:57:00 -07:00
Kevin Thompson
e8772f2603 Adds an enroll_secret_env flag that allows the user to specify that
the enroll secret for TLS enrollment is stored in an environment
variable rather than a file.
2015-08-25 21:11:19 -05:00
Mathieu Kooiman
b151ecedc2 Refs https://github.com/facebook/osquery/issues/320
Add provisioning scripts to build osquery on Debian Wheezy and Debian Jessie.
2015-08-20 20:57:22 +02:00
Teddy Reed
bdadc9753b Additional OS X table performance improvements 2015-08-18 01:35:10 -07:00
Teddy Reed
ff926730a9 Remove VirtualTable matrix rotation 2015-08-17 16:58:54 -07:00
Teddy Reed
5bf30a779d RocksDB usage speedups 2015-08-15 20:43:53 -07:00
Teddy Reed
43cf5f1a0a Merge pull request #1448 from theopolis/strol-speedup
Speedup type conversions, yara, and 10.10 symbols at runtime
2015-08-14 11:01:46 -07:00
Teddy Reed
68d7a6e0be Speedup type conversions, yara, and 10.10 symbols at runtime 2015-08-13 18:04:03 -07:00
Teddy Reed
634dfe7da1 Merge pull request #1438 from sharvilshah/fix_homebrew_version
[Fix #1434] version reporting for homewbrew_packages
2015-08-12 11:30:21 -07:00
Sharvil Shah
b190f5f99a Fix #1433, os_version reporting for 10.11 2015-08-11 14:03:27 -07:00
Sharvil Shah
369040e69b Fix version reporting for homewbrew_packages. Fixes #1434 2015-08-11 01:50:40 -07:00
Michael O'Farrell
eefccf27b1 Switch boost lexical casts to strtol. This should be faster than a boost lexical cast. 2015-08-07 16:33:32 -07:00
Sharvil Shah
64588be88b Fix build on OS X 10.11
enum `SecItemClass` changed in 10.11 headers,
so don't instantiate with rvalue of int.

Update `SecKeychainSearchCreateFromAttributes` to match the stricter definition.

Fixes #1423
2015-08-05 18:29:29 -07:00
Teddy Reed
1eea02ed9b Merge pull request #1419 from theopolis/sql_optimizations
Several small optimizations around internal SQL queries
2015-08-03 16:11:36 -07:00
Teddy Reed
a11dfcc222 Merge pull request #1422 from theopolis/options_on_packs
Apply query options to pack queries
2015-08-03 15:50:05 -07:00
Teddy Reed
f86c9e7778 Apply query options to pack queries 2015-08-03 15:33:55 -07:00
Teddy Reed
67b0f51ab5 Several small optimizations around internal SQL queries 2015-08-03 07:56:55 -07:00
Michael O'Farrell
5d0e4be6a1 Merge pull request #1335 from mofarrell/kernel-file-events
Added kernel file access events.
2015-07-31 15:22:11 -07:00
Michael O'Farrell
9f2b318778 Added kernel file access events. 2015-07-31 15:06:46 -07:00
Mike Arpaia
a45c794f52 building on 10.9 2015-07-31 11:57:39 -07:00
osquery
ae8305e00e Revert "Remove OS X 10.9 code path since we no longer support it"
This reverts commit 05bbe2ce06.
2015-07-31 11:44:34 -07:00
Michael O'Farrell
b0289adcf5 Merge pull request #1414 from theopolis/env_limits
Add optional environment variable whitelist to process_events
2015-07-30 18:17:31 -07:00
Teddy Reed
dc82ffa636 Add optional environment variable whitelist to process_events 2015-07-30 16:05:11 -07:00
Michael O'Farrell
8c8c591195 Merge pull request #1404 from mofarrell/load-kernel
Added loading of kernel.
2015-07-30 15:20:33 -07:00
Michael O'Farrell
eaf7de08df Added loading of kernel. 2015-07-30 14:36:46 -07:00
Michael O'Farrell
9e20d5904d Merge pull request #1412 from theopolis/use_sigkill
Use SIGKILL on OS X
2015-07-30 10:55:56 -07:00
Michael O'Farrell
f694149584 Merge pull request #1411 from mofarrell/benchmark-means
Benchmark using mean across 5 runs.
2015-07-29 18:00:35 -07:00
Teddy Reed
8082a0b5ac Use SIGKILL on OS X 2015-07-29 17:05:45 -07:00
Michael O'Farrell
346743e87f Benchmark using mean across 5 runs. 2015-07-29 16:50:19 -07:00
Chris Down
260df0d6d0 linux users table: Do not drop users with duplicate UIDs
See Github issue #1301. FreeBSD (which also uses this table) by default has two
users which are UID 0 -- both `toor` and `root`. 19a2d64959 made it so that we
would only get the first one from `getpwent`, but this feature is undesirable
in cases where two different users share the same UID.
2015-07-29 09:00:47 -07:00
Teddy Reed
fa36a8918b Merge pull request #1401 from theopolis/tests_and_benchmarks
Various additional tests and benchmarks
2015-07-28 13:20:46 -07:00
Teddy Reed
ff9cb71628 Various additional tests and benchmarks 2015-07-28 12:26:17 -07:00