Commit Graph

2254 Commits

Author SHA1 Message Date
Scott Piper
5e7d0d6a37 Added system_info table 2015-09-09 10:26:16 -07:00
Mike Arpaia
9929c61c94 Merge pull request #1500 from marpaia/remote
Client-side implementation of distributed queries
2015-09-08 15:02:32 -07:00
Mike Arpaia
aaa03a1058 Distributed queries client-side 2015-09-08 13:33:48 -07:00
Mike Arpaia
2a20804f59 Merge pull request #1499 from marpaia/flub
Removing remnants of a refactoring from Christmas Past
2015-09-07 17:34:20 -04:00
Mike Arpaia
07283817cb Removing remnants of a refactoring from Christmas Past 2015-09-04 11:33:33 -07:00
Mike Arpaia
041cc6784e Merge pull request #1495 from marpaia/config-md5
Config MD5 a bit more deterministic
2015-09-03 22:24:22 -07:00
Mike Arpaia
de58353131 Config MD5 a bit more deterministic
```
$ ./build/darwin/osquery/osqueryi --config_path=/asdfasdfadfs
E0903 11:45:02.050308 1990836992 init.cpp:370] Error reading config: config file does not exist
Using a virtual database. Need help, type '.help'
osquery> .mode line
osquery> .all osquery_info
           pid = 33700
       version = 1.5.2-43-gb06fa92
    config_md5 =
  config_valid = 0
   config_path = /asdfasdfadfs
    extensions = active
build_platform = darwin
  build_distro = 10.10
osquery> .exit

$ ./build/darwin/osquery/osqueryi
osquery> .mode line
osquery> .all osquery_info
           pid = 33781
       version = 1.5.2-43-gb06fa92
    config_md5 = 8a432ac93d3de080c62d77ba99b89783
  config_valid = 1
   config_path = /var/osquery/osquery.conf
    extensions = active
build_platform = darwin
  build_distro = 10.10
osquery> .exit
```
2015-09-03 22:03:40 -07:00
Teddy Reed
b06fa92e76 Merge pull request #1463 from theopolis/linux_audit2
Add a Linux audit event publisher
2015-09-03 09:38:15 -07:00
Teddy Reed
2813d3ab87 Add a Linux audit event publisher 2015-09-03 08:45:02 -07:00
Teddy Reed
ba7cef3f78 Merge pull request #1493 from theopolis/fix_1492
[Fix #1492] Fix firefox key counting and spec typo
2015-09-02 23:49:55 -07:00
Teddy Reed
01e040a01c Merge pull request #1491 from theopolis/cleanups3
Static analysis cleanups, static libmagic
2015-09-02 23:49:13 -07:00
Teddy Reed
7a15d25796 [Fix #1492] Fix firefox key counting and spec typo 2015-09-02 19:50:36 -07:00
Teddy Reed
bb2b5f594b Static analysis cleanups, static libmagic 2015-09-02 16:55:20 -07:00
Mike Arpaia
d5db0ccd17 Merge pull request #1485 from marpaia/ptvalue
Removing the ptvalue typedef
2015-09-02 14:53:12 -07:00
Mike Arpaia
fb2f33d770 Removing the ptvalue typedef
Removing the ptvalue typedef in favor of just using `pt::ptree::value_type`
2015-09-02 12:50:24 -07:00
Mike Arpaia
f92fa761f7 Merge pull request #1484 from theopolis/faster_plist
Add plist parsing benchmarks and refactor slightly for perf wins
2015-09-02 12:48:44 -07:00
Mike Arpaia
673e9869c0 Merge pull request #1474 from marpaia/packs-A-F
query pack re-org
2015-09-02 12:44:08 -07:00
Mike Arpaia
a140333441 [fix #1390] query pack re-org
This commit contains the features specified in #1390 as well as a
refactoring of the general osquery configuration code.

The API for the config plugins hasn't changed, although now there's a
`genPack` method that config plugins can implement. If a plugin doesn't
implement `genPack`, then the map<string, string> format cannot be used.
The default config plugin, the filesystem plugin, now implements
`genPack`, so existing query packs code will continue to work as it
always has.

Now many other config plugins can implement custom pack handling for
what makes sense in their context. `genPacks` is not a pure virtual, so
it doesn't have to be implemented in your plugin if you don't want to
use it. Also, more importantly, all config plugins can use the standard
inline pack format if they want to use query packs. Which is awesome.

For more information, refer to #1390, the documentation and the doxygen
comments included with this pull requests, as well as the following
example config which is now supported, regardless of what config plugin
you're using:

```json
{
  "options": {
    "enable_monitor": "true"
  },
  "packs": {
    "core_os_monitoring": {
        "version": "1.4.5",
        "discovery": [
          "select pid from processes where name like '%osqueryd%';"
        ],
        "queries": {
          "kernel_modules": {
              "query": "SELECT name, size FROM kernel_modules;",
              "interval": 600
          },
          "system_controls": {
              "query": "SELECT * FROM system_controls;",
              "interval": 600,
              "snapshot": true,
          },
          "usb_devices": {
              "query": "SELECT * FROM usb_devices;",
              "interval": 600
          }
        }
    },
    "osquery_internal_info": {
        "version": "1.4.5",
        "discovery": [
          "select pid from processes where name like '%osqueryd%';"
        ],
        "queries": {
          "info": {
              "query": "select i.*, p.resident_size, p.user_time, p.system_time, time.minutes as counter from osquery_info i, processes p, time where p.pid = i.pid;",
              "interval": 60,
              "snapshot": true
          },
          "registry": {
              "query": "SELECT * FROM osquery_registry;",
              "interval": 600,
              "snapshot": true
          },
          "schedule": {
              "query": "select name, interval, executions, output_size, wall_time, (user_time/executions) as avg_user_time, (system_time/executions) as avg_system_time, average_memory from osquery_schedule;",
              "interval": 60,
              "snapshot": true
          }
        }
    }
  }
}
```

The `osquery_packs` table was modified to remove the superfluous
columns which could already have been found in `osquery_schedule`. Two
more columns were added in their place, representing stats about pack's
discovery query execution history.

Notably, the internal API for the `osquery::Config` class has changed
rather dramatically as apart of the refactoring. We think this is an
improvement. While strictly adhering to the osquery config plugin
interface will have avoided any compatibility errors, advanced users may
notice compilation errors if they access config data directly. All
internal users of the config have obviously been updated. Yet another
reason to merge your code into mainline; we update it for you when we
refactor!
2015-09-02 10:56:26 -07:00
Teddy Reed
5f56490835 Add plist parsing benchmarks and refactor slightly for perf wins 2015-09-01 18:56:09 -07:00
Teddy Reed
8c75b15b5d Merge pull request #1480 from theopolis/remove_benchmark
Remove benchmark from third-party, prefer deps-build
2015-08-31 23:59:58 -07:00
Teddy Reed
d63510f8bd Remove benchmark from third-party, prefer deps-build 2015-08-31 15:01:13 -07:00
Teddy Reed
7ac1b993fc Merge pull request #1479 from theopolis/remove_netlib
Remove cpp-netlib from third-party, prefer deps-build
2015-08-31 14:24:37 -07:00
Teddy Reed
4dd77a43a7 Remove cpp-netlib from third-party, prefer deps-build 2015-08-31 09:27:01 -07:00
Teddy Reed
776de9c4d1 Merge pull request #1477 from theopolis/table_xp_meta
XProtect meta virtual table and safari_extensions column additions
2015-08-30 21:31:35 -07:00
Teddy Reed
906d19927f [#1418] Use libarchive to parse Safari extension bundles 2015-08-29 23:59:41 -07:00
Teddy Reed
9ca040c54f Merge pull request #1478 from sharvilshah/shell_cleanups
Shell Cleanups
2015-08-29 23:52:55 -07:00
Sharvil Shah
fc6865b8a9 Remove unused functions/macros/variables from shell and misc hardening 2015-08-29 22:08:01 -07:00
Teddy Reed
cd1d39b323 Merge pull request #1407 from theopolis/tls_customization
Add 'hidden' flags to customize TLS plugins
2015-08-28 17:21:49 -07:00
Javier Marcos
8aed4e95c3 Merge pull request #1476 from javuto/removing_dots
Removing dots at the end of log entries
2015-08-28 17:20:45 -07:00
Javier Marcos
74be3d1da0 Removing dots at the end of log entries 2015-08-28 16:50:44 -07:00
Javier Marcos
086ab40f83 Merge pull request #1473 from javuto/this_is_real_magic
Adding magic table to check for libmagic data
2015-08-28 14:03:10 -07:00
Teddy Reed
0e16f56c8d Add 'hidden' flags to customize TLS plugins 2015-08-28 12:57:53 -07:00
Javier Marcos
1a50977a23 Adding magic table to check for libmagic data 2015-08-28 12:49:46 -07:00
Teddy Reed
88c7ad35a2 Merge pull request #1471 from theopolis/process_start_fix
[Fix #1453] Use second precision for process start times
2015-08-28 11:48:25 -07:00
Teddy Reed
2433d9e06c [#1418] Include XProtect's meta list of plugin versions, and blacklisted extensions 2015-08-28 11:46:21 -07:00
Teddy Reed
14ee346eea Merge pull request #1472 from theopolis/usb_osx
[Fix #1432] Improve OS X USB device reporting
2015-08-28 10:10:23 -07:00
Teddy Reed
4decfed255 Merge pull request #1470 from theopolis/ubuntu_libgcrypt
[#1455] Add libgcrypt11 as a package dependency for debian builds
2015-08-27 16:40:51 -07:00
Teddy Reed
656e2558eb Merge pull request #1469 from sharvilshah/fix_make_docs
Fix doxygen docs generation
2015-08-27 16:40:41 -07:00
Teddy Reed
014e504fba [Fix #1432] Improve OS X USB device reporting 2015-08-27 16:36:54 -07:00
Teddy Reed
3c114c3439 [Fix #1453] Use second precision for process start times 2015-08-27 15:47:06 -07:00
Teddy Reed
263219daa7 [#1455] Add libgcrypt11 as a package dependency for debian builds 2015-08-27 13:28:32 -07:00
Sharvil Shah
e5155a56a6 Fix doxygen docs generation.
`make docs` and `genapi.py` expects build/docs directory to exist.
The directory wouldn't exist if `make distclean` is run, this change
creates the docs directory if it doesn't exist.
2015-08-27 13:20:45 -07:00
Teddy Reed
d47cac7434 Merge pull request #1461 from blackfist/master
Attempts to add a cli flag --enroll_secret_env
2015-08-26 14:57:00 -07:00
Teddy Reed
0ce8d5dc49 Merge pull request #1467 from sharvilshah/readme_query_update
[Doc Update] Fix example query in README
2015-08-26 14:18:15 -07:00
Sharvil Shah
33f626590c Fix example query to get process details for processes listening on all interfaces.
As it previously stood, the query was listening all process pids and
for each searching the matching pid that was listening.
It now joins process details with against each pid with a listening socket.

Hat tip to @theopolis (https://github.com/facebook/osquery/issues/1465#issuecomment-134835685)
2015-08-26 13:53:24 -07:00
Kevin Thompson
e8772f2603 Adds an enroll_secret_env flag that allows the user to specify that
the enroll secret for TLS enrollment is stored in an environment
variable rather than a file.
2015-08-25 21:11:19 -05:00
Teddy Reed
68cb40bae1 Merge pull request #1464 from mathieuk/update_thirdparty_commit
Update to latest commit on the third-party submodule
2015-08-25 09:02:24 -07:00
Teddy Reed
9a675878d5 Merge pull request #1458 from DavidGosselin/fedoraSupport
Support for Building on Fedora 21
2015-08-25 09:02:08 -07:00
Dave Gosselin
6899127e1e Add support for building on Fedora 21 2015-08-25 11:10:56 -04:00
Mathieu Kooiman
81cd751596 Update to latest commit on the third-party submodule 2015-08-25 09:05:50 +02:00