Commit Graph

521 Commits

Author SHA1 Message Date
Teddy Reed
1e9feba506 pidfile: Update pidfile to /var/run on Linux and fbsd (#3457) 2017-07-07 17:57:31 -07:00
Lambda Conjecture
ecb9e2ccf2 Add Epoch marker to scheduled query results (#3378) 2017-07-07 17:56:03 -07:00
Teddy Reed
f8f5718297 watcher: Do not initialize the config in watcher (#3403) 2017-06-13 17:26:34 -07:00
ryanheffernan
80aaed8b05 [Fix #3313] Adding driver version + adding malloc/pointer safety to drivers table (#3319) 2017-06-05 09:06:49 -07:00
Teddy Reed
eb4536dceb config: Only reconfigure if content changes (#3356) 2017-05-30 19:22:41 -07:00
Teddy Reed
a7162daea6 logger: Allow logString and logSnapshot to fast-track (#3362) 2017-05-29 23:49:37 -07:00
Teddy Reed
49ed383017 config: Unify the config refresh logic (#3351) 2017-05-29 14:09:44 -07:00
Teddy Reed
b38a62be8b config: Rename getInstance to get for consistency (#3350) 2017-05-28 23:04:53 -07:00
lxcode
044947528d Set some paths for FreeBSD. (#3337) 2017-05-26 15:24:33 -07:00
Mitchell Grenier
600a5d017a Add an sql function for carving paths (#3317) 2017-05-26 11:19:43 -07:00
Teddy Reed
775a4cdcce flags: Allow custom flags in configuration (#3301) 2017-05-25 21:29:31 -07:00
Mitchell Grenier
fe1418f240 Adding a config block to create views (#3306) 2017-05-24 21:03:15 -07:00
Teddy Reed
b427310241 deps: Rebuild the world, static and hidden (#3299) 2017-05-21 10:59:19 -07:00
Nick Anderson
95d916e24b [Fix #3268] extensions autoload now correctly spawns extension processes (#3269) 2017-05-11 16:51:48 -07:00
Nick Anderson
8fdb0a6945 Adding distributed workid to carver (#3252) 2017-05-05 15:14:21 -07:00
Seshu Pasam
4cfb31452c Docker support (#3241) 2017-05-05 09:53:12 -07:00
Teddy Reed
66c177945e logger: Require an open DB for status logs (#3205) 2017-04-23 13:16:12 -07:00
Teddy Reed
65ef94f053 logger: Fixes to allow plugins access to hostIDs (#3197) 2017-04-22 18:24:25 -07:00
Grigoriy Chudnov
e3a2445480 Add database init retry (#3179) 2017-04-21 15:32:40 -07:00
Teddy Reed
649afcfff1 events: Use generator-tables for event subscribers (#3150) 2017-04-12 21:45:41 -07:00
Teddy Reed
67440ab101 tests: Rewrite and add filesystem tests (#3156) 2017-04-10 21:32:16 -07:00
yying
31b78cb996 Adding hostIdentifier, calendarTime, unixTime to status logging (#3146) 2017-04-10 11:56:23 -07:00
Teddy Reed
76fe5d748c logging: Allow Glog reentrancy (#3142) 2017-04-06 15:57:44 -07:00
Teddy Reed
cea8da9829 freebsd: Add lldpd to dependencies (#3124) 2017-04-02 13:04:10 -07:00
Teddy Reed
bc85f726ad events: Execute schedule before expiring (#3091) 2017-03-21 12:38:04 -07:00
Teddy Reed
69bb69fd6d events: Inspect schedule and improve tests (#3087) 2017-03-20 22:03:09 -07:00
Teddy Reed
7b6d026820 rocksdb: Update to 5.0 and use DeleteRange for expirations (#3066) 2017-03-17 12:40:07 -07:00
Teddy Reed
3a5aca9c94 Add getQueryTables to inspect tables scanned (#3056) 2017-03-15 18:48:01 -07:00
Teddy Reed
1e71f4aab8 Introduce generator/yield-style virtual tables (#3060) 2017-03-15 18:46:42 -07:00
Teddy Reed
5604a7a99b freebsd: Rollup of build and source fixes (#3022) 2017-02-26 18:25:49 -08:00
Santosh Ananthakrishnan
d3adaedbb1 Allow reloading filesystem config with --config_refresh (#2967) 2017-02-26 17:45:06 -08:00
Teddy Reed
d1d21cda78 darwin: Use boost shared_mutex for OS X (#3003) 2017-02-15 14:17:40 -08:00
Teddy Reed
0e57003e74 packages: Use /var/osquery on OS X for home (#2976) 2017-02-07 12:22:41 -08:00
Mike Arpaia
7a019d8226 Bundle cURL PEM into packages (#2950) 2017-02-02 20:46:13 -08:00
Mike Arpaia
2ad1d8839f Introduce two new host identifier options (#2944) 2017-01-27 17:56:50 -08:00
Teddy Reed
487f7ee59b extensions: Select ephemeral path for shell socket (#2945) 2017-01-27 15:56:40 -08:00
Teddy Reed
58ed5cc628 Introduce scheduler reload feature (#2917) 2017-01-25 17:48:33 -08:00
lambda-conjecture
721dd1ed62 Fix column order and repeated columns in distributed query (#2926) 2017-01-20 22:52:47 -08:00
Mike Arpaia
b445d7215d Superfluous "d" in a doc block (#2911) 2017-01-16 19:18:59 -08:00
Teddy Reed
f9599d60d0 Move Mutex to shared_timed_mutex and add ReadLock (#2915) 2017-01-15 02:16:40 -08:00
Teddy Reed
0178419085 Add a TLS config plugin test that runs the scheduler (#2898) 2017-01-10 19:52:58 -08:00
Teddy Reed
0e9733f94c Simplify Registry and plugin concepts (#2887) 2017-01-07 12:21:35 -08:00
Jonathan Lee
383e07e5be [Fix #2734] Remove OpenSSL link dependency for osquery core (#2750) 2016-12-22 00:37:59 -08:00
Marc LeBourdais
aa08d487c9 Keep a local reference to the process name to avoid memory corruption (#2847) 2016-12-07 20:20:17 -08:00
lambda-conjecture
cd761d1749 Fix deadlock in RocksDB log callback (#2749) 2016-12-02 23:24:08 -08:00
Teddy Reed
1b21e5173e [Fix #2798] Update gflags on OS X/Linux to 2.2.0 (#2800) 2016-11-30 15:59:18 -08:00
Teddy Reed
555d59eff6 Remove fsuid permissions drop for OS X and Linux (#2805) 2016-11-28 22:05:08 -08:00
Teddy Reed
deed140080 [#1773] Introduce multi-pack configuration syntax (#2787) 2016-11-22 09:35:03 -08:00
Teddy Reed
6ead016cbb [Fix #2656] Refactor events_optimize to act per-query (#2665) 2016-11-05 22:03:45 -07:00
Teddy Reed
530f2933e3 [Fix #2704] Various distributed code cleanups (#2719) 2016-11-03 23:54:55 -07:00
Teddy Reed
e775fe1ea2 [Fix #2579] Remove database_in_memory flag (#2716) 2016-11-03 09:55:10 -07:00
Teddy Reed
a3acf2a3e5 Fix Config TLS plugin default verb (#2708) 2016-11-02 17:08:44 -07:00
Teddy Reed
4c8fdf5d17 Fix UDEV publisher unit tests LSAN bug (#2693) 2016-10-30 11:15:55 -07:00
Teddy Reed
b04736631a Add --extensions_require feature (#2672) 2016-10-24 18:13:44 -07:00
Teddy Reed
b814fd54dc [Fix #2674] Add SQLite prepare lock to shell_exec (#2677) 2016-10-24 08:25:38 -07:00
Teddy Reed
02b21d00c3 Add dropToUser method to privileges dropper interface (#2624) 2016-10-14 15:25:54 -07:00
Nick Anderson
616d9f5953 Adding support for Windows platform_info table (#2611) 2016-10-12 09:01:32 -07:00
Teddy Reed
0b1713423c [Fix #1690] Use INDEX options in constraint cost evaluation (#2593) 2016-10-05 15:44:21 -07:00
Zachary Wasserman
6e7c1cdf32 Fix doxygen string in core.h causing make docs to fail (#2585) 2016-10-02 16:52:44 -07:00
Nick Anderson
b69981584f Fixing COM memory leak (#2583) 2016-10-02 12:15:17 -07:00
Teddy Reed
ff2e1a182f Initialize VirtualTableContent attributes (#2530) 2016-09-26 23:52:25 -07:00
Teddy Reed
4d1451c9b4 Add extensions SDK incompatibility checking (#2527) 2016-09-26 17:32:41 -07:00
Teddy Reed
17b89fc182 Refactor events and remove 10/3600 indexes (#2523) 2016-09-25 22:19:31 -07:00
Teddy Reed
bcd90070ae Remove time-override for events add API (#2508)
This will remove the use of current time for syslog.time and introduce
a new column called 'datetime'.

Events now uses an "optimize_id" alongside "optimize" to prevent returning
colliding events added within the same second as the previous genTable call.
2016-09-23 16:46:02 -07:00
Zachary Wasserman
9701c55d96 Add active column to osquery_packs table (#2475) 2016-09-19 13:00:11 -07:00
Mitchell Grenier
61c9da1c42 Buffer the distributed queries to RocksDB for greater reliability (#2452) 2016-09-08 15:40:14 -07:00
Teddy Reed
81d8a4aa68 Emit warnings for improper 'osquery' table usage (#2432) 2016-09-02 19:13:37 -07:00
yying
84e6a3401a Reducing compiler warnings and fails on warn in VS (#2433) 2016-09-02 15:04:03 -07:00
Nick Anderson
7c90823a0c Upgrade LLVM to 3.8.1 on Linux (#2436) (#2435) 2016-09-02 14:53:04 -07:00
yying
d347c847e1 Support for extensions (#2363) 2016-08-31 16:45:06 -07:00
Teddy Reed
65dd56e113 Introduce table 'attributes' (#2431) 2016-08-31 15:32:20 -07:00
Teddy Reed
0b3f6af306 Improve status logging when using multi-loggers (#2422) 2016-08-29 06:59:55 -07:00
Teddy Reed
1bff276fcf Increase TLS client timeout from 4s to 32s (#2410) 2016-08-27 13:12:48 +01:00
yying
2f1cad864d Support for building static osquery executable (#2398) 2016-08-22 23:27:12 +01:00
Nick Anderson
fe7b8d98f9 Adding getSystemRoot function (#2386)
Adding a cross platform function for getting the OS root
and returning it as a boost::fs::path
2016-08-18 09:32:34 -07:00
Teddy Reed
58fd284f05 Improve dispatcher tests (#2358)
This improves dispatcher tests by allowing units to act like component
tests and use embedded std::thread-based osquery APIs. A unit may force
a 'service' to run by joining the Dispatcher before deconstructing.
2016-08-14 15:41:53 -07:00
Teddy Reed
9a7b930a94 [Fix #2281] Avoid ctor ambiguity in table header (#2345) 2016-08-10 20:55:18 -07:00
Teddy Reed
f3f605e26a Introduce a PLATFORM_MASK and isPlatform (#2334)
Along with the platform defines and platform string defines provided by
CMake to the build, add a PLATFORM_MASK define.

Use this define as a platform-type mask with the PlatformType enum.
2016-08-09 20:27:42 -07:00
Teddy Reed
7eab0f39bd Fix race conditions in Linux inotify publisher (#2309)
1. This adds several mutexes to the inotify publisher and its tests.
2. A fix for Linux 4.1 and LLVM TSAN is applied to CMake logic.
2016-07-31 22:41:37 -07:00
Teddy Reed
e4dd78ec10 Build documentation for brew-based build (#2305) 2016-07-31 17:10:13 -07:00
yying
50487c6880 Changes to make pidfile work on Windows (#2297) 2016-07-28 16:04:34 -07:00
yying
0ef284b8e7 Changes to make osqueryd/osqueryi mostly build sans cputime/uptime tables (#2283) 2016-07-25 11:58:55 -07:00
Zachary Wasserman
b71276a1ea Fix up doxygen docs and require correct docs in CI (#2280)
This change causes Doxygen to error if there are unrecognized commands, or
undocumented/incorrect parameters in documented functions. It does not require
that every function be fully documented, just that those that are do not have
errors.

Old documentation with problems was fixed to comply.
2016-07-25 12:21:26 -04:00
yying
2fb3797c53 Changes to support building a osquery Windows service. (#2278) 2016-07-22 13:29:37 -07:00
Teddy Reed
c22f6147ea Move OSQUERY_HOME into core and use as filesystem config default (#2275) 2016-07-21 13:28:23 -07:00
Teddy Reed
394bce27d5 Fix mismatched free/delete in QueryContext dtor (#2259) 2016-07-19 19:27:48 -07:00
artemdinaburg
78e1cf7ab4 Transition __attribute__((constructor)) to a more platform independent approach (#2233) 2016-07-14 14:19:33 -07:00
Teddy Reed
45530c0496 Slight performance improvments (#2242) 2016-07-12 19:16:50 -07:00
Teddy Reed
48cb4d555d Add systemLog API (#2229)
This includes a minor SDK refactor as it move quite a few specialized
functions and facilities from core.h into system.h. There was a breaking point
for needing to frequently update core includes.

The new logger systemLog function allows a call site to bypass logging config
and write a line to the OS logger (aka syslog).
2016-07-07 15:16:28 -07:00
Teddy Reed
6852122af9 Force RocksDB to sync writes for non-event domains (#2228)
RocksDB is the default "database" plugin. Writes are normally kept in an
in-memory memtable. Writes that are not part of the event pubsub system can
be forced to sync to disk.
2016-07-07 14:08:12 -07:00
Teddy Reed
8dfe79ad83 Use a noexcept lexical_cast for SQL type conversions (#2212) 2016-07-01 11:37:19 -07:00
artemdinaburg
864d77b180 Windows Daemon/Shell: Windows Processes Table (#2184)
Include table changes necessary for a Windows processes table and changes to other tables needed for daemon and shell to run. The Windows processes table uses WMI as a backend to gather information. This commit does not yet build these tables.
2016-06-28 11:07:34 -07:00
Teddy Reed
a9ed73c68d Add removeService to Dispatcher API (#2116)
With a removeService method, combined with the abstracted thread start in
the Dispatcher API, services auto-remove when finished.

This will un-break the kernel communication tests. These tests only stop
when all their producer threads/services have ended.

This also promotes the OS X kernel build to 10.11.
2016-05-27 11:17:06 -07:00
Teddy Reed
10664a4945 Add --ephemeral for daemons and disable shell events (#2111)
This changes several initialization steps:
- The daemon (and shell, though not needed) have a new --ephemeral flag.
- Events are now disabled in the shell by default, use --nodisable_events to
  re-enable.
- RocksDB-based backing storage is now disabled in the shell by default.

The --ephemeral flag for the daemon is disabled by default and will allow
skipping configuration and database path sanity, and skipping pidfile checks.
This is intended to be used when debugging or monitoring the daemon process.

To make the RocksDB backing storage feature usage very clear we introduce a new
flag: --disable_database. The shell sets this to true unless overridden in
a flagfile or via command line arguments.
2016-05-20 08:39:04 -07:00
Teddy Reed
c4acfe89e5 Introduce table aliases (#2104) 2016-05-19 09:40:43 -07:00
Teddy Reed
b28c4d8d0f Introduce table options (#2101)
Table options includes a change to the Registry::call API for TablePlugins.
When requesting route information or the 'columns' action, a new 'op' key is included.
2016-05-18 12:23:52 -07:00
yying
26ad131c38 Building osquery unit tests on Windows 10 (#2100)
Integrated process abstraction code into more locations
Defined new macros for abstracting across various platforms
Added GLOG_NO_ABBREVIATED_SEVERITIES for glog to support Windows
Fixed some minor CMake issues involving thrift
Updated gflags package; reflecting change in provision script
Preparing CMake config files for WIN32 support
2016-05-17 12:39:11 -07:00
Teddy Reed
77273f6500 Add logEvent API to logger plugins (#2088) 2016-05-13 19:48:40 -07:00
Teddy Reed
ce2ba6d9c8 Refactor logStatus and logger initialization (#2081)
The initialization of a logger plugin was confusing. The 'init' step was
introduced to allow a daemon to buffer status events before a logger plugin
is determined by external/remote configuration. The buffered statuses could
then be transferred via a medium other than Glog (the default). To determine
if Glog should continue to write statuses to the filesystem the 'init' method
returned a Status.

Logger plugins should now use a feature method override to select how status
logs should be handled.
2016-05-11 12:05:09 -07:00
Teddy Reed
947799d28e Introduce within-query caching (#2077)
This adds a new optimization feature that allows expensive tables to cache
their results between JOINs. Consider JOINing a list of open sockets, for each
process, then requesting to hash each process path. This query may hash
the same path multiple times.

Within-query caching allows the hash table to respond with the previous
result of the hash request as long as the requested computation was the result
of a single query. Subsequent queries will perform subsequent hashing.
2016-05-09 10:32:33 -07:00