valitydev/osquery-1
14
0
Fork 0
mirror of https://github.com/valitydev/osquery-1.git synced 2024-11-08 10:23:54 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
3,983 Commits 2 Branches 109 Tags 25 MiB
7b321cef60
Commit Graph

277 Commits

Author SHA1 Message Date
Teddy Reed
7e9088e008 [#2542] Introduce --enable_syslog to explicit enable syslog ingestion (#2543) 2016-09-27 17:35:21 -07:00
Zachary Wasserman
9216ed8275 Make syslog rate limit configurable by flag (#2526) 2016-09-26 17:31:22 -07:00
Teddy Reed
17b89fc182 Refactor events and remove 10/3600 indexes (#2523) 2016-09-25 22:19:31 -07:00
Teddy Reed
bcd90070ae Remove time-override for events add API (#2508) 
This will remove the use of current time for syslog.time and introduce
a new column called 'datetime'.

Events now uses an "optimize_id" alongside "optimize" to prevent returning
colliding events added within the same second as the previous genTable call.
 2016-09-23 16:46:02 -07:00
yying
84e6a3401a Reducing compiler warnings and fails on warn in VS (#2433) 2016-09-02 15:04:03 -07:00
Teddy Reed
080bc5ed88 Improve verbose logging for several linux event publishers (#2421) 2016-08-29 14:26:25 -07:00
Teddy Reed
05a795d80a Count subscriber events correctly in osquery_events (#2419) 
This also changes the osquery_events API by renaming restarts to refreshes.
 2016-08-29 06:57:24 -07:00
Teddy Reed
987368221f Remove several raw strings that confuse static analysis (#2367) 2016-08-15 14:52:11 -07:00
Teddy Reed
dd3020df79 [Fix #2319] Emit verbose log when Linux audit is immutable (#2347) 2016-08-12 18:30:21 -07:00
artemdinaburg
d8bfe962aa Fix Windows under 1.8 build system (#2333) 2016-08-10 14:06:47 -07:00
Teddy Reed
33c1afa4b8 Allow the non-blocking kernel-test publisher to drop 5% (#2336) 2016-08-10 08:45:37 -07:00
Teddy Reed
1c4d6397fa OS X IOKit utilities refactor to allow SKIP_TABLES (#2335) 2016-08-09 20:49:56 -07:00
Teddy Reed
f3f605e26a Introduce a PLATFORM_MASK and isPlatform (#2334) 
Along with the platform defines and platform string defines provided by
CMake to the build, add a PLATFORM_MASK define.

Use this define as a platform-type mask with the PlatformType enum.
 2016-08-09 20:27:42 -07:00
Teddy Reed
7eab0f39bd Fix race conditions in Linux inotify publisher (#2309) 
1. This adds several mutexes to the inotify publisher and its tests.
2. A fix for Linux 4.1 and LLVM TSAN is applied to CMake logic.
 2016-07-31 22:41:37 -07:00
Teddy Reed
870c5bd9f9 Clean up verbose logging for OS X kernel extension (#2276) 2016-07-21 14:29:17 -07:00
yying
547e8f961c CMake configuration file changes to support Windows (#2258) 2016-07-20 23:48:55 -07:00
artemdinaburg
78e1cf7ab4 Transition __attribute__((constructor)) to a more platform independent approach (#2233) 2016-07-14 14:19:33 -07:00
Teddy Reed
7f304a0934 Various fixups and best practices (#2237) 2016-07-11 09:45:57 -07:00
Teddy Reed
48cb4d555d Add systemLog API (#2229) 
This includes a minor SDK refactor as it move quite a few specialized
functions and facilities from core.h into system.h. There was a breaking point
for needing to frequently update core includes.

The new logger systemLog function allows a call site to bypass logging config
and write a line to the OS logger (aka syslog).
 2016-07-07 15:16:28 -07:00
Nick Anderson
cf30388705 Moved test_utils to it's own directory out of core. Updated references (#2154) 2016-06-09 10:49:26 -07:00
Teddy Reed
866ff13fc3 Fix OS X kernel extension autoload (#2151) 2016-06-08 11:14:36 -07:00
yying
26ad131c38 Building osquery unit tests on Windows 10 (#2100) 
Integrated process abstraction code into more locations
Defined new macros for abstracting across various platforms
Added GLOG_NO_ABBREVIATED_SEVERITIES for glog to support Windows
Fixed some minor CMake issues involving thrift
Updated gflags package; reflecting change in provision script
Preparing CMake config files for WIN32 support
 2016-05-17 12:39:11 -07:00
Teddy Reed
77273f6500 Add logEvent API to logger plugins (#2088) 2016-05-13 19:48:40 -07:00
Teddy Reed
57c6b2a521 Revive the OS X kernel-based publishers (#2083) 
The OS X kernel subscribers have not been starting because they expect the
publisher thread to run before they begin configuration. Due to some recent
refactors the publisher thread creation now occurs after configuration.

The subscriber logic to check for a valid kernel connection is still valid.

This commit has two additional side-effects:
- The RocksDB plugin is modified to use 3 background merge threads.
- The OS X kernel publisher syncing thread is now non-blocking.
 2016-05-11 11:47:42 -07:00
Zachary Wasserman
98cdd3643f Add linux syslog virtual table 
This commit adds an event-based virtual table implementation for
querying the linux syslog. It introduces an event publisher that
attaches to a named pipe to ingest CSV formatted syslog forwarded from
rsyslogd. An event subscriber/virtual table makes these log lines
available for queries. Currently, no additional processing is done on
the input data besides parsing.

Using this table requires a properly configured rsyslogd. Documentation
for this configuration is forthcoming in the wiki.
 2016-03-30 13:36:57 -07:00
Teddy Reed
d2d1431061 Move dispatcher to public API 2016-03-21 15:27:51 -07:00
Teddy Reed
482eecfab1 Protect udev publisher from fast interrupts 2016-03-20 18:46:34 -07:00
Teddy Reed
15a998e54f Use the default shutdown flow within extensions 2016-03-20 01:45:49 -07:00
Teddy Reed
d7c2f88289 Enhance publisher resource locking on OS X 2016-03-18 16:14:15 -07:00
Teddy Reed
c62a0f41b6 Various cleanups 2016-03-18 10:40:07 -07:00
Teddy Reed
3e103e69ba Merge pull request #1931 from ilovezfs/iokitlib-header-casesensitivity 
IOKitLib.h not IOKitlib.h
 2016-03-16 22:32:03 -07:00
Teddy Reed
593f024514 Merge pull request #1936 from theopolis/events_ex 
Expire data when record is before expire time
 2016-03-16 12:58:21 -07:00
Teddy Reed
7040780863 Expire data when record is before expire time 2016-03-16 12:35:06 -07:00
Teddy Reed
bb20a968d9 Merge pull request #1930 from theopolis/debug 
Build debug packages
 2016-03-15 11:12:56 -07:00
ilovezfs
52e7d55600 IOKitLib.h not IOKitlib.h 
As with all other appearances of IOKitLib.h in the osquery sources, use
the capitalization "IOKitLib.h" not "IOKitlib.h" to avoid build failure
on case-sensitive file systems.
 2016-03-15 09:43:11 -07:00
Teddy Reed
42222bd4a5 Build debug packages 2016-03-15 08:58:01 -07:00
Teddy Reed
0ba2861cf9 [Fix #1920] Detach thread before joining/clearing (terminate) 2016-03-13 12:15:18 -07:00
Teddy Reed
59274e59c6 Remove boost::thread from fsevents tests 2016-03-12 00:30:05 -08:00
Teddy Reed
21c7ab642b Remove boost::thread from inotify tests 2016-03-12 00:15:58 -08:00
Teddy Reed
3de52846d0 Remove boost::thread 2016-03-11 11:50:44 -08:00
Teddy Reed
afd17f8134 1. Reorganize RocksDB database handle into a plugin 
2. Introduce a SQLite-based database plugin
3. Refactor database usage to include local 'fast-calls'
4. Introduce an 'ephemeral' database plugin for testing (like a mock)
 2016-03-06 20:40:16 -08:00
Teddy Reed
897b2225b1 Add fstests and reduce SQLite scope 2016-02-23 17:09:02 -08:00
Baraa Hamodi
21c2237eca [osquery] Update copyright headers to new format. 2016-02-11 11:48:58 -08:00
Teddy Reed
4031e299bb Cleanup/stabilize file_events-related APIs 2016-02-10 22:50:38 -08:00
Teddy Reed
7f37304c77 Refactor dispatcher shutdown logic 2016-02-05 01:29:42 -08:00
Teddy Reed
0b263114c1 Merge pull request #1818 from theopolis/inotify_access_limit 
[#1814] Do not stat inotify access subscriptions
 2016-02-03 22:08:51 -08:00
Teddy Reed
77ceca4693 [#1814] Do not stat inotify access subscriptions 2016-02-03 18:13:44 -08:00
Teddy Reed
8947dac232 [Fix #1814] Various fixes for Linux inotify 2016-02-03 17:00:41 -08:00
Teddy Reed
724ca51e16 Lower severity of failed publishers 2016-02-01 16:42:21 -08:00
Teddy Reed
f05cc345d3 Add an events_max limit for event buffering 2016-02-01 08:38:58 -08:00
Teddy Reed
832c3cfcce Unify build script and fix EVENTS benchmarks 2016-01-12 17:09:52 -08:00
Teddy Reed
c2b78faa09 Merge pull request #1740 from theopolis/events_improvements 
Fix double event subscriber select
 2015-12-17 22:32:57 -08:00
Teddy Reed
c4f3db1613 Fix double event subscriber select 2015-12-17 19:23:26 -08:00
Teddy Reed
f9faf0bea7 [Fix #1735] Limit OPENED and access-related events 2015-12-17 15:42:32 -08:00
Teddy Reed
db3782bc7f Do not add (self) events for FSEvents 2015-12-16 13:32:39 -08:00
Teddy Reed
31dfad2515 Fix unhelpful subscriber verbose error for process_file_events 2015-12-14 15:09:52 -08:00
Teddy Reed
98eb6a5055 Reorganize file_events into process_file_events 2015-12-11 00:58:22 -08:00
Teddy Reed
9d394065e3 [#1636] Add simple sharding to packs and pack queries 2015-12-10 10:01:53 -08:00
Teddy Reed
9f79d74c60 Add canary path on empty FSEvents subscription set 2015-12-09 00:14:08 -08:00
Teddy Reed
309944c586 Configuration triggered publisher reconfiguration 2015-12-08 14:03:35 -08:00
Teddy Reed
6602a59b7d Change EventSubscriber API to include subscription references 2015-12-07 22:22:04 -08:00
Teddy Reed
b7650e5291 Remove passwd_changes and user_data from event callbacks 2015-12-07 17:47:38 -08:00
Teddy Reed
5370fef950 Merge pull request #1678 from theopolis/audit_user_events 
[#1497] Add user_events table based on audit user-type messages
 2015-11-23 21:31:37 -08:00
Teddy Reed
07fd718e00 Add user_events table based on audit user-type messages 2015-11-23 18:13:31 -08:00
Teddy Reed
6748fdb024 Rewrite OS X hardware events to use IOKit proper 2015-11-21 19:31:05 -08:00
Teddy Reed
41ba637030 Linux inotify should accept non-glob dirs 2015-11-04 13:46:47 -08:00
Teddy Reed
cd4de8023f Merge pull request #1630 from theopolis/fix_1626 
[Fix #1626] Add schedule blacklist and protect DBHandle
 2015-11-03 21:05:29 -08:00
Teddy Reed
edea3d6edd [Fix #1626] Add schedule blacklist and protect DBHandle 2015-11-03 20:50:22 -08:00
Teddy Reed
5728c93392 Remove specific filenames from RocksDB IOErrors 2015-11-02 15:12:52 -08:00
Teddy Reed
a1a9131174 Optimize socket_events and Linux users 2015-11-02 10:37:56 -08:00
Teddy Reed
d27a7ecc4c Fix clang warnings, promote warnings to errors 2015-11-01 02:12:07 -08:00
Teddy Reed
2cf7543181 [Fix #1611] Prevent fs links in inotify path search 2015-10-29 23:19:07 -07:00
Teddy Reed
a3067fcbb5 Fix auditd message parsing 2015-10-27 16:56:42 -07:00
Teddy Reed
ba4eeb6a80 [#1600] Put inotify into a mod-only watch mode 2015-10-27 16:42:21 -07:00
Teddy Reed
b81b6de6ae This refactors a bit of config/packs and adds a socket_events table to Linux. 
The refactor of config/packs was initiated because event subscribers needed
a method for toggling `::init` based on some configurable option. In the case
of auditd, turning on the support with `--disable_audit=false` used to start
auditing the EXECVE syscall. It was understandable that this would cause
latency based on the number of processes executing per measure of time.

A new `socket_events` table will do the same but for `bind` and `connect`. These
are less-obvious and for now, require a scan of /proc for socket tuples. In the
future this file descriptor to socket tuple will be faster.
 2015-10-27 15:13:02 -07:00
Robert C. Seacord
1d9695ac31 eliminated some warnings from Clang 3.7 analyze mode 2015-10-21 06:02:58 +00:00
Robert C. Seacord
7a87be9ada more sign coversion errors 2015-10-20 06:08:01 +00:00
Robert C. Seacord
e57828aac3 changes for integer sign problems 2015-10-17 00:18:35 +00:00
Robert C. Seacord
acb2f6f628 eliminating diagnostics, mostly for comparisons between signed and unsigned operations 2015-10-16 16:10:37 +00:00
Teddy Reed
3be0994933 [Fix #1570] Check for invalid apt sources 
This fixes a crash identified by @endrazine.
When apt sources data in /etc/apt/sources.list or /etc/apt/sources.list.d/{*}.list contain invalid data/lines the cache_file.GetPkgCache(); call will fail and cache will be nullptr. Subsequent usage results in a SIGSEV.

To reproduce the fault try:

$ zzuf -I /etc/ -r 0.01:0.1 -s 0:1000 -v \
 ./build/trusty/osquery/osqueryi --registry_exceptions=true --verbose \
 "select count(*) from apt_sources"

Signed-off-by: Jonathan Brossard
 2015-10-15 15:20:26 -07:00
Teddy Reed
bbac2cf07f [#1529] Allow DB Readonly with RocksDB lite 2015-09-28 01:50:32 -07:00
Teddy Reed
bb65ec49ac [#1488] Shutdown Linux event publishers responsibly 2015-09-22 23:06:23 -07:00
Teddy Reed
97ca0e627a [#1488] Stop OS X event publishers with SIGINT 2015-09-21 22:02:27 -07:00
Teddy Reed
333f2ce8c8 [#1506] Silent kext loading messages from syslog 2015-09-16 13:13:56 -07:00
Teddy Reed
b57040db60 Add osquery_events table to track pubsub stats 2015-09-03 15:10:53 -07:00
Teddy Reed
2813d3ab87 Add a Linux audit event publisher 2015-09-03 08:45:02 -07:00
Teddy Reed
bb2b5f594b Static analysis cleanups, static libmagic 2015-09-02 16:55:20 -07:00
Teddy Reed
cd1d39b323 Merge pull request #1407 from theopolis/tls_customization 
Add 'hidden' flags to customize TLS plugins
 2015-08-28 17:21:49 -07:00
Javier Marcos
74be3d1da0 Removing dots at the end of log entries 2015-08-28 16:50:44 -07:00
Teddy Reed
0e16f56c8d Add 'hidden' flags to customize TLS plugins 2015-08-28 12:57:53 -07:00
Teddy Reed
5bf30a779d RocksDB usage speedups 2015-08-15 20:43:53 -07:00
Michael O'Farrell
5d0e4be6a1 Merge pull request #1335 from mofarrell/kernel-file-events 
Added kernel file access events.
 2015-07-31 15:22:11 -07:00
Michael O'Farrell
9f2b318778 Added kernel file access events. 2015-07-31 15:06:46 -07:00
Michael O'Farrell
b0289adcf5 Merge pull request #1414 from theopolis/env_limits 
Add optional environment variable whitelist to process_events
 2015-07-30 18:17:31 -07:00
Teddy Reed
dc82ffa636 Add optional environment variable whitelist to process_events 2015-07-30 16:05:11 -07:00
Michael O'Farrell
8c8c591195 Merge pull request #1404 from mofarrell/load-kernel 
Added loading of kernel.
 2015-07-30 15:20:33 -07:00
Michael O'Farrell
eaf7de08df Added loading of kernel. 2015-07-30 14:36:46 -07:00
Michael O'Farrell
346743e87f Benchmark using mean across 5 runs. 2015-07-29 16:50:19 -07:00
Teddy Reed
fa36a8918b Merge pull request #1401 from theopolis/tests_and_benchmarks 
Various additional tests and benchmarks
 2015-07-28 13:20:46 -07:00
Teddy Reed
ff9cb71628 Various additional tests and benchmarks 2015-07-28 12:26:17 -07:00