valitydev/osquery-1
14
0
Fork 0
mirror of https://github.com/valitydev/osquery-1.git synced 2024-11-08 02:18:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,506 Commits 2 Branches 109 Tags 25 MiB
12716496aa
Commit Graph

1420 Commits

Author SHA1 Message Date
Teddy Reed
284dac71de Write helpful DB access/open error to verbose log 2015-09-20 10:35:26 -07:00
Teddy Reed
946ab354ff Merge pull request #1517 from theopolis/fix_yara 
Fix YARA sigfile caching
 2015-09-20 10:34:29 -07:00
Teddy Reed
d042967f43 Fix YARA sigfile caching 2015-09-20 00:06:57 -07:00
Jack Naglieri
9c1e114728 Fix os_version table regex for REDHAT_BASED systems. Updating centos6/7 and freebsd10 Vagrant boxes. 2015-09-18 14:47:08 -07:00
Mike Arpaia
a0795f300b Merge pull request #1512 from theopolis/schedule_tracking 
Scheduled query success tracking
 2015-09-17 13:39:04 -07:00
Teddy Reed
c51d214ddd Scheduled query success tracking 2015-09-16 23:31:07 -07:00
Mike Arpaia
73045e4974 Moving packs to top level include directory 
I could've swore that I did this already, but this moves
`include/osquery/config/packs.h` to `include/osquery/packs.h`.
 2015-09-16 15:51:05 -07:00
Teddy Reed
333f2ce8c8 [#1506] Silent kext loading messages from syslog 2015-09-16 13:13:56 -07:00
Mike Arpaia
3d81223dfb Merge pull request #1508 from marpaia/distributed_test_fixes 
Making distributed tests more awesome and less flaky
 2015-09-16 12:05:51 -07:00
Mike Arpaia
dc6e395b77 Only log to warning if the config can't be read by the daemon 
fix #1504
 2015-09-16 10:54:38 -07:00
Mike Arpaia
41ef6798c6 Making distributed tests more awesome and less flaky 
Distributed tests were failing every now and then because the test
plugin didn't implement retry's and the test server wasn't always
starting up fast enough. I fixed this by refactoring the tests to use
the real TLS plugin, which has retry logic. This required some mangling
of the configuration options, which should serve as a good reference as
well.
 2015-09-16 10:36:34 -07:00
Teddy Reed
7852c356ec Merge pull request #1494 from theopolis/signals 
[#1488] Use signal handlers for teardown and reloading
 2015-09-15 16:14:40 -07:00
Teddy Reed
65162e7239 Merge pull request #1501 from sharvilshah/sysinfo_updates 
Update system_info table to include CPU type, CPU cores and total memory
 2015-09-14 20:02:56 -04:00
Teddy Reed
7c2a625ef2 Use signal handlers for teardown and reloading 2015-09-14 16:57:00 -07:00
Teddy Reed
944e3de206 Merge pull request #1496 from theopolis/events_table 
[#1487] Add osquery_events table to track pubsub stats
 2015-09-14 15:27:35 -04:00
Sharvil Shah
28143f64f0 Update system_info table: adds CPU type, CPU cores and total memory. 
This change adds following columns to `system_info` table:

    cpu_type, cpu_subtype, cpu_brand, cpu_physical_cores,
    cpu_logical_cores, physical_memory, hardware_model

Here's an example output of those columns:

```
              cpu_type = x86_64h
           cpu_subtype = Intel x86-64h Haswell
             cpu_brand = Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
    cpu_physical_cores = 4
     cpu_logical_cores = 8
       physical_memory = 17179869184
        hardware_model = MacBookPro11,3
```
 2015-09-10 14:44:48 -07:00
Matthew White
28d456a2f1 Fix build for Ubuntu Lucid 2015-09-10 13:55:59 -04:00
Scott Piper
5e7d0d6a37 Added system_info table 2015-09-09 10:26:16 -07:00
Mike Arpaia
9929c61c94 Merge pull request #1500 from marpaia/remote 
Client-side implementation of distributed queries
 2015-09-08 15:02:32 -07:00
Mike Arpaia
aaa03a1058 Distributed queries client-side 2015-09-08 13:33:48 -07:00
Mike Arpaia
07283817cb Removing remnants of a refactoring from Christmas Past 2015-09-04 11:33:33 -07:00
Mike Arpaia
de58353131 Config MD5 a bit more deterministic 
```
$ ./build/darwin/osquery/osqueryi --config_path=/asdfasdfadfs
E0903 11:45:02.050308 1990836992 init.cpp:370] Error reading config: config file does not exist
Using a virtual database. Need help, type '.help'
osquery> .mode line
osquery> .all osquery_info
           pid = 33700
       version = 1.5.2-43-gb06fa92
    config_md5 =
  config_valid = 0
   config_path = /asdfasdfadfs
    extensions = active
build_platform = darwin
  build_distro = 10.10
osquery> .exit

$ ./build/darwin/osquery/osqueryi
osquery> .mode line
osquery> .all osquery_info
           pid = 33781
       version = 1.5.2-43-gb06fa92
    config_md5 = 8a432ac93d3de080c62d77ba99b89783
  config_valid = 1
   config_path = /var/osquery/osquery.conf
    extensions = active
build_platform = darwin
  build_distro = 10.10
osquery> .exit
```
 2015-09-03 22:03:40 -07:00
Teddy Reed
b57040db60 Add osquery_events table to track pubsub stats 2015-09-03 15:10:53 -07:00
Teddy Reed
2813d3ab87 Add a Linux audit event publisher 2015-09-03 08:45:02 -07:00
Teddy Reed
ba7cef3f78 Merge pull request #1493 from theopolis/fix_1492 
[Fix #1492] Fix firefox key counting and spec typo
 2015-09-02 23:49:55 -07:00
Teddy Reed
01e040a01c Merge pull request #1491 from theopolis/cleanups3 
Static analysis cleanups, static libmagic
 2015-09-02 23:49:13 -07:00
Teddy Reed
7a15d25796 [Fix #1492] Fix firefox key counting and spec typo 2015-09-02 19:50:36 -07:00
Teddy Reed
bb2b5f594b Static analysis cleanups, static libmagic 2015-09-02 16:55:20 -07:00
Mike Arpaia
fb2f33d770 Removing the ptvalue typedef 
Removing the ptvalue typedef in favor of just using `pt::ptree::value_type`
 2015-09-02 12:50:24 -07:00
Mike Arpaia
f92fa761f7 Merge pull request #1484 from theopolis/faster_plist 
Add plist parsing benchmarks and refactor slightly for perf wins
 2015-09-02 12:48:44 -07:00
Mike Arpaia
a140333441 [fix #1390] query pack re-org 
This commit contains the features specified in #1390 as well as a
refactoring of the general osquery configuration code.

The API for the config plugins hasn't changed, although now there's a
`genPack` method that config plugins can implement. If a plugin doesn't
implement `genPack`, then the map<string, string> format cannot be used.
The default config plugin, the filesystem plugin, now implements
`genPack`, so existing query packs code will continue to work as it
always has.

Now many other config plugins can implement custom pack handling for
what makes sense in their context. `genPacks` is not a pure virtual, so
it doesn't have to be implemented in your plugin if you don't want to
use it. Also, more importantly, all config plugins can use the standard
inline pack format if they want to use query packs. Which is awesome.

For more information, refer to #1390, the documentation and the doxygen
comments included with this pull requests, as well as the following
example config which is now supported, regardless of what config plugin
you're using:

```json
{
  "options": {
    "enable_monitor": "true"
  },
  "packs": {
    "core_os_monitoring": {
        "version": "1.4.5",
        "discovery": [
          "select pid from processes where name like '%osqueryd%';"
        ],
        "queries": {
          "kernel_modules": {
              "query": "SELECT name, size FROM kernel_modules;",
              "interval": 600
          },
          "system_controls": {
              "query": "SELECT * FROM system_controls;",
              "interval": 600,
              "snapshot": true,
          },
          "usb_devices": {
              "query": "SELECT * FROM usb_devices;",
              "interval": 600
          }
        }
    },
    "osquery_internal_info": {
        "version": "1.4.5",
        "discovery": [
          "select pid from processes where name like '%osqueryd%';"
        ],
        "queries": {
          "info": {
              "query": "select i.*, p.resident_size, p.user_time, p.system_time, time.minutes as counter from osquery_info i, processes p, time where p.pid = i.pid;",
              "interval": 60,
              "snapshot": true
          },
          "registry": {
              "query": "SELECT * FROM osquery_registry;",
              "interval": 600,
              "snapshot": true
          },
          "schedule": {
              "query": "select name, interval, executions, output_size, wall_time, (user_time/executions) as avg_user_time, (system_time/executions) as avg_system_time, average_memory from osquery_schedule;",
              "interval": 60,
              "snapshot": true
          }
        }
    }
  }
}
```

The `osquery_packs` table was modified to remove the superfluous
columns which could already have been found in `osquery_schedule`. Two
more columns were added in their place, representing stats about pack's
discovery query execution history.

Notably, the internal API for the `osquery::Config` class has changed
rather dramatically as apart of the refactoring. We think this is an
improvement. While strictly adhering to the osquery config plugin
interface will have avoided any compatibility errors, advanced users may
notice compilation errors if they access config data directly. All
internal users of the config have obviously been updated. Yet another
reason to merge your code into mainline; we update it for you when we
refactor!
 2015-09-02 10:56:26 -07:00
Teddy Reed
5f56490835 Add plist parsing benchmarks and refactor slightly for perf wins 2015-09-01 18:56:09 -07:00
Teddy Reed
4dd77a43a7 Remove cpp-netlib from third-party, prefer deps-build 2015-08-31 09:27:01 -07:00
Teddy Reed
776de9c4d1 Merge pull request #1477 from theopolis/table_xp_meta 
XProtect meta virtual table and safari_extensions column additions
 2015-08-30 21:31:35 -07:00
Teddy Reed
906d19927f [#1418] Use libarchive to parse Safari extension bundles 2015-08-29 23:59:41 -07:00
Teddy Reed
9ca040c54f Merge pull request #1478 from sharvilshah/shell_cleanups 
Shell Cleanups
 2015-08-29 23:52:55 -07:00
Sharvil Shah
fc6865b8a9 Remove unused functions/macros/variables from shell and misc hardening 2015-08-29 22:08:01 -07:00
Teddy Reed
cd1d39b323 Merge pull request #1407 from theopolis/tls_customization 
Add 'hidden' flags to customize TLS plugins
 2015-08-28 17:21:49 -07:00
Javier Marcos
74be3d1da0 Removing dots at the end of log entries 2015-08-28 16:50:44 -07:00
Javier Marcos
086ab40f83 Merge pull request #1473 from javuto/this_is_real_magic 
Adding magic table to check for libmagic data
 2015-08-28 14:03:10 -07:00
Teddy Reed
0e16f56c8d Add 'hidden' flags to customize TLS plugins 2015-08-28 12:57:53 -07:00
Javier Marcos
1a50977a23 Adding magic table to check for libmagic data 2015-08-28 12:49:46 -07:00
Teddy Reed
88c7ad35a2 Merge pull request #1471 from theopolis/process_start_fix 
[Fix #1453] Use second precision for process start times
 2015-08-28 11:48:25 -07:00
Teddy Reed
2433d9e06c [#1418] Include XProtect's meta list of plugin versions, and blacklisted extensions 2015-08-28 11:46:21 -07:00
Teddy Reed
014e504fba [Fix #1432] Improve OS X USB device reporting 2015-08-27 16:36:54 -07:00
Teddy Reed
3c114c3439 [Fix #1453] Use second precision for process start times 2015-08-27 15:47:06 -07:00
Teddy Reed
d47cac7434 Merge pull request #1461 from blackfist/master 
Attempts to add a cli flag --enroll_secret_env
 2015-08-26 14:57:00 -07:00
Kevin Thompson
e8772f2603 Adds an enroll_secret_env flag that allows the user to specify that 
the enroll secret for TLS enrollment is stored in an environment
variable rather than a file.
 2015-08-25 21:11:19 -05:00
Mathieu Kooiman
b151ecedc2 Refs https://github.com/facebook/osquery/issues/320 
Add provisioning scripts to build osquery on Debian Wheezy and Debian Jessie.
 2015-08-20 20:57:22 +02:00
Teddy Reed
bdadc9753b Additional OS X table performance improvements 2015-08-18 01:35:10 -07:00
Teddy Reed
ff926730a9 Remove VirtualTable matrix rotation 2015-08-17 16:58:54 -07:00
Teddy Reed
5bf30a779d RocksDB usage speedups 2015-08-15 20:43:53 -07:00
Teddy Reed
43cf5f1a0a Merge pull request #1448 from theopolis/strol-speedup 
Speedup type conversions, yara, and 10.10 symbols at runtime
 2015-08-14 11:01:46 -07:00
Teddy Reed
68d7a6e0be Speedup type conversions, yara, and 10.10 symbols at runtime 2015-08-13 18:04:03 -07:00
Teddy Reed
634dfe7da1 Merge pull request #1438 from sharvilshah/fix_homebrew_version 
[Fix #1434] version reporting for homewbrew_packages
 2015-08-12 11:30:21 -07:00
Sharvil Shah
b190f5f99a Fix #1433, os_version reporting for 10.11 2015-08-11 14:03:27 -07:00
Sharvil Shah
369040e69b Fix version reporting for homewbrew_packages. Fixes #1434 2015-08-11 01:50:40 -07:00
Michael O'Farrell
eefccf27b1 Switch boost lexical casts to strtol. This should be faster than a boost lexical cast. 2015-08-07 16:33:32 -07:00
Sharvil Shah
64588be88b Fix build on OS X 10.11 
enum `SecItemClass` changed in 10.11 headers,
so don't instantiate with rvalue of int.

Update `SecKeychainSearchCreateFromAttributes` to match the stricter definition.

Fixes #1423
 2015-08-05 18:29:29 -07:00
Teddy Reed
1eea02ed9b Merge pull request #1419 from theopolis/sql_optimizations 
Several small optimizations around internal SQL queries
 2015-08-03 16:11:36 -07:00
Teddy Reed
a11dfcc222 Merge pull request #1422 from theopolis/options_on_packs 
Apply query options to pack queries
 2015-08-03 15:50:05 -07:00
Teddy Reed
f86c9e7778 Apply query options to pack queries 2015-08-03 15:33:55 -07:00
Teddy Reed
67b0f51ab5 Several small optimizations around internal SQL queries 2015-08-03 07:56:55 -07:00
Michael O'Farrell
5d0e4be6a1 Merge pull request #1335 from mofarrell/kernel-file-events 
Added kernel file access events.
 2015-07-31 15:22:11 -07:00
Michael O'Farrell
9f2b318778 Added kernel file access events. 2015-07-31 15:06:46 -07:00
Mike Arpaia
a45c794f52 building on 10.9 2015-07-31 11:57:39 -07:00
osquery
ae8305e00e Revert "Remove OS X 10.9 code path since we no longer support it" 
This reverts commit 05bbe2ce06.
 2015-07-31 11:44:34 -07:00
Michael O'Farrell
b0289adcf5 Merge pull request #1414 from theopolis/env_limits 
Add optional environment variable whitelist to process_events
 2015-07-30 18:17:31 -07:00
Teddy Reed
dc82ffa636 Add optional environment variable whitelist to process_events 2015-07-30 16:05:11 -07:00
Michael O'Farrell
8c8c591195 Merge pull request #1404 from mofarrell/load-kernel 
Added loading of kernel.
 2015-07-30 15:20:33 -07:00
Michael O'Farrell
eaf7de08df Added loading of kernel. 2015-07-30 14:36:46 -07:00
Michael O'Farrell
9e20d5904d Merge pull request #1412 from theopolis/use_sigkill 
Use SIGKILL on OS X
 2015-07-30 10:55:56 -07:00
Michael O'Farrell
f694149584 Merge pull request #1411 from mofarrell/benchmark-means 
Benchmark using mean across 5 runs.
 2015-07-29 18:00:35 -07:00
Teddy Reed
8082a0b5ac Use SIGKILL on OS X 2015-07-29 17:05:45 -07:00
Michael O'Farrell
346743e87f Benchmark using mean across 5 runs. 2015-07-29 16:50:19 -07:00
Chris Down
260df0d6d0 linux users table: Do not drop users with duplicate UIDs 
See Github issue #1301. FreeBSD (which also uses this table) by default has two
users which are UID 0 -- both `toor` and `root`. 19a2d64959 made it so that we
would only get the first one from `getpwent`, but this feature is undesirable
in cases where two different users share the same UID.
 2015-07-29 09:00:47 -07:00
Teddy Reed
fa36a8918b Merge pull request #1401 from theopolis/tests_and_benchmarks 
Various additional tests and benchmarks
 2015-07-28 13:20:46 -07:00
Teddy Reed
ff9cb71628 Various additional tests and benchmarks 2015-07-28 12:26:17 -07:00
Michael O'Farrell
93a65eaf04 Merge pull request #1400 from mofarrell/process-events-env-arg 
Adding environment variables and arguments for process events.
 2015-07-27 17:54:06 -07:00
Michael O'Farrell
3f87d5832f Adding environment variables and arguments for process events. 2015-07-27 15:48:47 -07:00
Wesley Shields
698e226b80 Add tags and strings columns to YARA tables. 
When strings match they will be populated into the "strings" column of
the table. The format is identifier:offset.

When a matching rule has tags defined the tags will be put into the
"tags" column of the table in a comma separated list.
 2015-07-27 08:20:24 -04:00
Teddy Reed
e2553e26b1 Merge pull request #1391 from theopolis/1374 
[Fix #1374] Allow subscription subclassing
 2015-07-26 13:46:19 -07:00
Alex Gaynor
e9dca0ef4d Fixed #1392 -- removed non-existant modes from .mode's help 2015-07-26 13:34:08 -04:00
Teddy Reed
d2effc539c [Fix #1374] Allow subscription subclassing 2015-07-26 01:48:27 -07:00
Teddy Reed
af13c1b7ea Silence google benchmark CMake output, remove benchmark tests 2015-07-24 09:52:29 -07:00
Teddy Reed
cce8a6aab3 Merge pull request #1384 from theopolis/table_cleanups 
Remove some non-warning/error log lines from tables
 2015-07-24 00:32:11 -07:00
Teddy Reed
2d7ce9341a Remove some non-warning/error log lines from tables 2015-07-24 00:09:06 -07:00
Teddy Reed
928f46c00f Merge pull request #1379 from theopolis/fix_1369 
[Fix #1369] Limit IOKit HID events
 2015-07-23 18:26:04 -07:00
Teddy Reed
5e3a86d2a8 Merge pull request #1376 from theopolis/fix_1367 
[Fix #1367] Disable user-controlled FIFO reads
 2015-07-23 18:25:52 -07:00
Teddy Reed
220fa0bd92 Merge pull request #1383 from theopolis/fix_1381 
[Fix #1381] Add documentation/install for daemon+Homebrew
 2015-07-23 18:25:40 -07:00
Teddy Reed
264ec99bd3 Merge pull request #1378 from mlw/fix-ubuntu10-string-concat-crash 
Support for older GCC compiler
 2015-07-23 18:25:05 -07:00
Michael O'Farrell
66b075a685 Merge pull request #1377 from mofarrell/benchmark 
Added benchmarking targets.
 2015-07-23 17:37:56 -07:00
Michael O'Farrell
a65f8dd93c Added benchmarking targets. 2015-07-23 17:07:42 -07:00
Teddy Reed
81aa36ecc7 [Fix #1381] Add documentation/install for daemon+Homebrew 2015-07-23 16:05:59 -07:00
Javier Marcos
f91a96f590 Fixing problem with versionChecker and adding usecase to tests 2015-07-23 14:21:43 -07:00
Teddy Reed
7c330f0bf8 [Fix #1369] Limit IOKit HID events 2015-07-23 11:52:23 -07:00
Matthew White
1c3587b95a Changed where string concat was being performed to support older GCC compiler 2015-07-23 08:56:26 -07:00
Teddy Reed
ad94eaf0b8 [Fix #1367] Disable user-controlled FIFO reads 2015-07-22 10:15:39 -07:00
Teddy Reed
fc24682816 Fix profile platform bug in leaks checking 2015-07-20 02:06:52 -07:00
Teddy Reed
e8cb919f03 Merge pull request #1364 from theopolis/harden_applications 
[Fix #1357] Use OS X LS API for app listing
 2015-07-20 01:14:07 -07:00
Mike Arpaia
5ccfe886ba Merge pull request #1363 from theopolis/less_rows 
[Fix #1303] Only emit rows when appropriate for processes/users.
 2015-07-19 20:36:26 -07:00
Mike Arpaia
8a760d52db Merge pull request #1361 from theopolis/dot_plist 
[Fix #1355] Allow plist keys with '.'
 2015-07-19 20:34:19 -07:00
Mike Arpaia
74021459c2 Merge pull request #1359 from theopolis/mutable_config_parser 
Allow ConfigParserPlugins to update the ConfigData.
 2015-07-19 20:34:11 -07:00
Mike Arpaia
4d05b54647 Merge pull request #1358 from theopolis/optimize_events 
Optimize event publisher database namespace lookups.
 2015-07-19 20:34:03 -07:00
Teddy Reed
dd7990b719 [Fix #1357] Use OS X LS API for app listing 
Attempt to use OS X's LaunchServices to get a list of applications.
Fall back to basic directory traversal of well-known application paths.
 2015-07-19 20:22:48 -07:00
Teddy Reed
5249e74146 [Fix #1303] Only emit rows when appropriate for processes/users. 
When optimizing a table using query constraints an implementation should not add unneeded rows.
A user experience bug exists when selecting with an explicit non-existing pid/uid.
 2015-07-19 20:20:04 -07:00
Teddy Reed
8eaf389010 Optimize event publisher database namespace lookups. 
Previously, event publishers used a canonicalized 'type' name for async callbacks.
This type was used to lookup the publisher plugin in the registry as well as for backing store namespacing.
The type is still used but subscribers, which made heavy used of the lookup, store the value locally.
This prevents unneeded publisher plugin allocation when adding events.
 2015-07-19 17:10:42 -07:00
Teddy Reed
95775be1d9 [Fix #1355] Allow plist keys with '.' 
Boost property trees are level delimited using '.' characters.
An Apple property list may contain keys with '.' characters, so the plist conversion must use iterators and raw node appends.
 2015-07-19 16:24:43 -07:00
Teddy Reed
bcdbb40f0c [Fix #1356] Tokenize process environ by '\0' on Linux 2015-07-19 14:34:49 -07:00
Teddy Reed
2109ae85b7 Allow ConfigParserPlugins to update the ConfigData. 
Previously, `ConfigParserPlugin`s could only maintain an internal derived object called `data_`.
Then parts of the code that knew to use the plugin's data would call `getParsedData` and provide the name of the plugin.

Parser plugins can now request a mutable version of the `ConfigData` using `::mutableConfigData`.
This requires a lock on the `ConfigDataInstance` and must be provided to their mutable accessor.

Acess to a mutable config enables parsers to make modifications to internal config structures like options and the query schedule.
 2015-07-18 15:08:51 -07:00
Teddy Reed
a713d09f0e Install additional configs for HB/packages 2015-07-17 16:07:22 -07:00
Teddy Reed
6104aaebfe Add optional TLS config plugin refresh 2015-07-17 14:59:08 -07:00
Teddy Reed
c36fbda274 Merge pull request #1349 from theopolis/centos_version 
[Fix #1319] CentOS version reporting and file read error
 2015-07-17 09:07:29 -07:00
Teddy Reed
270b4da540 [Fix #1339] Add kernel-build to packages when used 2015-07-16 15:23:29 -07:00
Teddy Reed
f06820f578 [Fix #1319] CentOS version reporting and file read error 
1. Redhat-based distributions were not reporting their version correct.
2. The file read API assumed stat would return an accurate file size.
This has been replaced with an attempt to seek to the end of the file.
 2015-07-16 14:16:51 -07:00
Teddy Reed
4cb6e37f1d Merge pull request #1338 from theopolis/join_bug 
Fix broken JOIN predicate passing
 2015-07-16 11:45:33 -07:00
Teddy Reed
deecef81c5 Fix broken JOIN predicate passing 2015-07-16 11:29:56 -07:00
Mike Arpaia
9eeb224ce7 clang-format authorizations files 2015-07-16 11:09:16 -07:00
Mike Arpaia
333f0c5799 Merge pull request #1345 from achmiel/fix_symlinks 
Updated the readFile function to correctly handle symbolic links
 2015-07-15 23:21:35 -07:00
Artur Chmiel
ac9a320218 Updated the readFile function to correctly handle symbolic links 2015-07-16 07:55:12 +02:00
Mike Arpaia
485a7f78fb Merge pull request #1318 from tburgin/master 
Added authdb table
 2015-07-15 21:51:53 -07:00
Teddy Reed
5f6577deb2 [Fix #1341] Add osqueryctl to make install target 2015-07-15 11:32:55 -07:00
Tom Burgin
e8d3e45cea Added authorization_mechanisms and authorizations tables 2015-07-15 14:25:19 -04:00
Michael O'Farrell
0eba0776e5 Merge pull request #1336 from mofarrell/master 
Kernel publisher only log info when not connected.  [Fix #1334]
 2015-07-14 20:27:45 -07:00
Michael O'Farrell
019defc788 Kernel publisher only log info when not connected. [Fix #1334] 2015-07-14 20:10:50 -07:00
Teddy Reed
263090e8f2 [Fix #1332] Check mode for links in readFile 
1. "really" check for links in readFile
2. Apply the same restrictions and flag ACLs to file hashing.
 2015-07-14 14:24:52 -07:00
Teddy Reed
c269bbeaf3 Rollup of build changes 2015-07-14 13:45:53 -07:00
Michael O'Farrell
276891ad00 Merge pull request #1330 from mofarrell/kernel 
Kernel!!!
 2015-07-13 17:29:08 -07:00
Michael O'Farrell
58ec6415d3 Created a basic publisher system for kernel events in the kernel extension. 2015-07-13 16:42:55 -07:00
Teddy Reed
3bd6b64b8b Silence OS X OpenSSL-related deprecations 2015-07-13 10:14:47 -07:00
Teddy Reed
1d336ccdb0 Merge pull request #1321 from sharvilshah/cert_parsing_fixes 
[Fix #1032] Better/faster performance when querying certificates on OS X
 2015-07-13 09:02:44 -07:00
Sharvil Shah
1ac6702f32 Better/faster performance when querying certificates on OS X 
X509 parsing is now handled by OpenSSL as there does seem to be a
memory leak in SecCertificateCopyValues of Security framework which resulted
in a performance hit when querying certificates.

key_usage and key_algorithm columns now display human readable strings
(e.g. Digital Signature, CRL Sign rsaEncryption)
than the raw flags and OIDs (e.g 0x86, 1.2.840.1).

This fixes #1032
 2015-07-12 11:18:53 -07:00
Mike Arpaia
370290d103 Merge pull request #1312 from theopolis/fix_getifaddres 
Fix getifaddrs result / data checking
 2015-07-10 01:46:55 -04:00
Teddy Reed
1de9eb68bc Enable registry exceptions for benchmarking/testing 2015-07-08 23:35:49 -07:00
Teddy Reed
d3424f5831 Fix getifaddrs checking 2015-07-08 22:37:35 -07:00
Michael O'Farrell
4bbb591b37 Added kernel process events table. 2015-07-08 13:47:07 -07:00
Michael O'Farrell
0284b9e60d Merge branch 'master' into kernel 
Conflicts:
	mkdocs.yml
 2015-07-08 10:26:32 -07:00
Michael O'Farrell
ba28b47239 Merge pull request #1298 from theopolis/event_streams 
Event index time and streaming
 2015-07-07 18:27:35 -07:00
Teddy Reed
ab56011881 Apply FIM pattern matching to inotify 2015-07-07 18:18:45 -07:00
Teddy Reed
0854c3ddc3 Merge pull request #1292 from theopolis/memory_tweaks 
Some tweaks to estimated scratch/heap for SQLite and RocksDB
 2015-07-07 08:11:30 -07:00
Teddy Reed
f48619ed28 [#1285, #1276] Faster, optimized subscriber results 2015-07-07 00:59:28 -07:00
Teddy Reed
41002b829c Merge pull request #1299 from timzimmermann/date 
Add date information to time table
 2015-07-07 00:46:32 -07:00
Teddy Reed
d2685cfa41 [#1142] Move path resolution into publisher logic 2015-07-07 00:45:55 -07:00
Teddy Reed
bf65e3d2d6 Event index time and streaming 2015-07-07 00:44:57 -07:00
Tim Zimmermann
0c3b123cb1 Add date information to time table 
The fix also includes the time in ISO 8601 format
as well as the format returned by C++'s asctime().
See #1297.
 2015-07-07 00:00:50 -07:00
Ari Rubinstein
be72e42bf1 Fix version string for TLS plugins 
Before, osqueryd would send `osquery/OSQUERY_BUILD_VERSION` as the user agent and appeared broken.  I copied the logic from the osquery version table and used that var here also so the user agent now reads 1.4.7
 2015-07-06 22:12:26 -07:00
Teddy Reed
dd9fa25d78 [Fix #1171, #1089] Add configurable max reads 
There are 3 new options that control how files are read:
--read_max: controls the maximum size, in bytes, for file reads. If a file is larger than `read_max` the read will fail.
--read_user_max: similar to `read_max` but applies additional limitations to user-controlled files.
--read_user_links: a boolean control to enable/disable following symlinks for user-controlled files.

Important highlights:
If files exceed the configured max, those reads will fail.
The `read_max` will override `read_user_max` if it is set lower.
A default integer value of `0` will disable the limitations.

The default `read_max` is set to 50M and the default `read_user_max` is 10M.
 2015-07-06 00:49:43 -07:00
Ryan Steinmetz
6f6bd8cabc - Fix build under FreeBSD 2015-07-03 19:47:47 -04:00
Teddy Reed
e73a867b75 Merge pull request #1269 from theopolis/fsevents_symlinks 
[Fix #1063] Allow configure-time symlink resolution in FSEvents
 2015-07-03 00:37:58 -07:00
Mike Arpaia
4f94c0034c Merge pull request #1290 from timzimmermann/uptime 
Uptime
 2015-07-03 00:23:44 -07:00
Tim Zimmermann
fa988b4e56 Add uptime table 
The table contains information about the time passed since the last boot.
 2015-07-02 22:32:48 -07:00
Michael O'Farrell
a712cd5036 Fix processes table to report gid correctly. 2015-07-02 17:03:25 -07:00
Teddy Reed
546aaa885d [Fix #1063] Allow configure-time symlink resolution in FSEvents 2015-07-02 16:50:27 -07:00
Teddy Reed
7aac5fd358 Replace custom wildcarding with POSIX-glob 
POSIX-globbing will allow event publishers/subscribers to post-check
results against glob-syntax, fnpath matching, and POSIX C-regex.
These checks are anecdotally speedy.
 2015-07-02 13:53:16 -07:00
Teddy Reed
a8813ab7d8 Some tweaks to estimated scratch/heap for SQLite and RocksDB 2015-07-02 13:52:39 -07:00
Teddy Reed
e24614c959 Merge pull request #1286 from theopolis/relay_status_logs 
[#1277] Forward status logs to osqueryd workers
 2015-07-02 10:33:58 -07:00
Michael O'Farrell
a00fb638c2 Added kernel event publisher. 2015-07-01 17:40:42 -07:00
Mike Arpaia
ba89b67cc5 Install snappy headers instead of just the library 
We found that not installing the headers for snappy caused RocksDB's
snappy detection to not find that snappy was installed:
https://goo.gl/YOWJl0

The snippet there requires that the headers are installed, not just the
library. By installing the headers, we can ensure that snappy is linked.

OR, alternatively, we could just leave it and not link snappy. It's
uncertain what the specific benefits of including snappy are for our
use-case. (CC @igorcanadi)
 2015-07-01 16:14:06 -07:00
Teddy Reed
79de0a5def [#1277] Forward status logs to osqueryd workers 
If watcher processes generate warning or error status logs they
will "relay" to the worker processes upon successful sanity check.
 2015-07-01 15:26:26 -07:00
Michael O'Farrell
1ab7040d83 Kernel extension fixes for daemon shutdown process. 2015-06-30 18:00:25 -07:00
Michael O'Farrell
a7bd4bd3db Merge pull request #1278 from facebook/master 
Merge branch 'master' into kernel
 2015-06-30 13:12:16 -07:00
Mike Arpaia
a2ec9d5885 rename osquery::getConfig to osquery::makeTLSConfigRequest 2015-06-29 23:33:40 -07:00
Teddy Reed
0d6ab16281 Yara events was not building 2015-06-29 14:45:31 -07:00
Michael O'Farrell
680ffd3bc8 Added a gangsta test (gtest) for the kernel communications. 
This test does not evaluate the functionality of the kernel
communication unless the KERNEL_TEST flag was set during the build.
The test will not succeed unless the tests are being run as root.
 2015-06-29 12:12:54 -07:00
Teddy Reed
d339877c01 Merge branch 'master' into kernel 2015-06-28 11:30:14 -07:00
Teddy Reed
6011ad06eb Fix small issue with printing 2015-06-28 11:18:35 -07:00
Teddy Reed
8db6ca4a3f [Fix #1198] Add a small retry to ext watcher 2015-06-28 02:12:50 -07:00
Michael O'Farrell
f4e05b992a Merge branch 'master' into kernel 
Conflicts:
	mkdocs.yml
 2015-06-26 17:04:42 -07:00
Michael O'Farrell
89fb4fbaf0 Moved kernel userland code into the osquery directory structure. 
Test cpp files are dead.
 2015-06-25 12:38:39 -07:00
Teddy Reed
e7ed68e187 [Fix #1198] Faster death/timeout checks in extensions tests 2015-06-25 02:53:53 -07:00
Teddy Reed
6437ddb82d Merge pull request #1235 from sharvilshah/remove_os_x_10_9_code 
Remove OS X 10.9 code path
 2015-06-24 15:18:32 -07:00
Mike Arpaia
7d5cb221dd Merge pull request #1239 from marpaia/1237-segfault 
Check for nullptr in CreatePropertyFromCertificate
 2015-06-24 08:25:25 -07:00
Mike Arpaia
d6389dc64d Check for nullptr in CreatePropertyFromCertificate 2015-06-23 21:45:46 -07:00
Sharvil Shah
05bbe2ce06 Remove OS X 10.9 code path since we no longer support it 2015-06-22 20:49:34 -07:00
Teddy Reed
040d9d5fd1 Merge pull request #1216 from sharvilshah/osx_mount_events 
[Implement #1103] DMG Mount Events
 2015-06-22 12:38:32 -07:00
Sharvil Shah
f676ba7642 Implements disk_events and the related publisher and subscriber. 
We now have a Publisher to report on disk events and its metadata,
using the DiskArbitration framework on OS X. Currently disk appearance
and disappearance events are published for both physical and
virtual disks (DMG files). On an event trigger, disk properties are
parsed and that metadata is reported along with the action.

The Subscriber subscribes to virtual disk events currently.

This closes #1103.
 2015-06-22 11:09:18 -07:00
Teddy Reed
37188f788b Fixups in tables, add DOUBLE, shell extensions 2015-06-22 04:17:23 -04:00
Teddy Reed
55f270ff97 OS X application duti/scheme listing table 2015-06-21 14:08:21 -04:00
Mike Arpaia
be85046d32 typo in keychain_acls table where path was being returned as app_path 2015-06-21 13:52:01 -04:00
Mike Arpaia
0a83572f08 Table to enumerate keychain ACLs 2015-06-20 14:59:07 -04:00
Mike Arpaia
fe8b25f443 Merge pull request #1218 from theopolis/osx_sandboxes 
Add application sandbox container metadata
 2015-06-19 11:01:03 -04:00
Teddy Reed
09ea12a2a7 Add application sandbox container metadata 2015-06-19 01:53:09 -04:00
Teddy Reed
fcc875ca47 Merge pull request #1212 from theopolis/syslog_plugin 
[#1207] Add syslog plugin
 2015-06-18 19:49:16 -04:00
Teddy Reed
b24cf6f20d Add syslog plugin 2015-06-18 15:59:40 -04:00
Teddy Reed
f74af5a063 [Fix #1205] Prevent wrapping when calculating average schedule memory 2015-06-13 02:25:24 -07:00
Teddy Reed
e7ab2fc47b Limit scope of git/tag version defines. 
Harden plist parsing against internal fuzzing tests.
Improve file/stream read speeds.
 2015-06-12 10:10:20 -07:00
Teddy Reed
d143b22cfa [Fix #1202] Replace argv[*] with spaces, fallback to path in [0] 2015-06-11 20:58:17 -07:00
Teddy Reed
b56e9efd47 Merge pull request #1199 from theopolis/fix_open_sockets 
Process open sockets on Linux needs '['
 2015-06-07 14:04:45 -07:00
Teddy Reed
49eb22ef44 Process open sockets on Linux was added '[' 2015-06-07 13:28:17 -07:00
Teddy Reed
e57d15da86 Merge pull request #1195 from theopolis/feature-nice 
Various table perf improvements and TLS docs
 2015-06-06 15:19:31 -07:00
Teddy Reed
727f5b091f Various table perf improvements and TLS docs 2015-06-05 22:03:15 -07:00
Teddy Reed
4c80891010 Fix FSEvents multiplexing actions 2015-06-05 17:36:29 -07:00
Teddy Reed
1168b6ef3b Fix the watchdog/scheduler limit tracking 2015-06-04 17:43:37 -07:00
Teddy Reed
4e59bcf4c1 Merge pull request #1191 from theopolis/feature-backoffs 
[#1190] Schedule queries without logging removed results
 2015-06-04 14:58:19 -07:00
Teddy Reed
e244883ea4 [#1190] Schedule queries without logging removed results 2015-06-04 13:53:55 -07:00
Mike Arpaia
ea70781f25 Merge pull request #1188 from marpaia/msr_format 
Formatting the callback function in the model_specific_register table
 2015-06-04 12:17:19 -07:00
Teddy Reed
a70828c2a4 Merge pull request #1187 from sharvilshah/xattr_update 
Extended Attributes: Use LaunchServices API for quarantine data
 2015-06-03 22:38:17 -07:00
Sharvil Shah
065fe6412d Use LaunchServices (part of CoreServices) to grab quarantine properties instead of manually parsing the colon separated attribute data. 
Fall back to deprecated LaunchService API for OS X 10.9 Mavericks.

Added tests for extended_attributes

Better error handling and cleanup
 2015-06-03 22:18:45 -07:00
Teddy Reed
8e2b7e1281 Merge pull request #1189 from theopolis/tooling 
Update tooling/profiling paths and use a better random seed
 2015-06-03 22:15:22 -07:00
Teddy Reed
c934ad0df3 Update tooling/profiling paths 2015-06-03 21:22:12 -07:00