Commit Graph

348 Commits

Author SHA1 Message Date
Teddy Reed
3be0994933 [Fix #1570] Check for invalid apt sources
This fixes a crash identified by @endrazine.
When apt sources data in /etc/apt/sources.list or /etc/apt/sources.list.d/{*}.list contain invalid data/lines the cache_file.GetPkgCache(); call will fail and cache will be nullptr. Subsequent usage results in a SIGSEV.

To reproduce the fault try:

$ zzuf -I /etc/ -r 0.01:0.1 -s 0:1000 -v \
 ./build/trusty/osquery/osqueryi --registry_exceptions=true --verbose \
 "select count(*) from apt_sources"

Signed-off-by: Jonathan Brossard
2015-10-15 15:20:26 -07:00
Teddy Reed
bbac2cf07f [#1529] Allow DB Readonly with RocksDB lite 2015-09-28 01:50:32 -07:00
Teddy Reed
bb65ec49ac [#1488] Shutdown Linux event publishers responsibly 2015-09-22 23:06:23 -07:00
Teddy Reed
97ca0e627a [#1488] Stop OS X event publishers with SIGINT 2015-09-21 22:02:27 -07:00
Teddy Reed
333f2ce8c8 [#1506] Silent kext loading messages from syslog 2015-09-16 13:13:56 -07:00
Teddy Reed
b57040db60 Add osquery_events table to track pubsub stats 2015-09-03 15:10:53 -07:00
Teddy Reed
2813d3ab87 Add a Linux audit event publisher 2015-09-03 08:45:02 -07:00
Teddy Reed
bb2b5f594b Static analysis cleanups, static libmagic 2015-09-02 16:55:20 -07:00
Teddy Reed
cd1d39b323 Merge pull request #1407 from theopolis/tls_customization
Add 'hidden' flags to customize TLS plugins
2015-08-28 17:21:49 -07:00
Javier Marcos
74be3d1da0 Removing dots at the end of log entries 2015-08-28 16:50:44 -07:00
Teddy Reed
0e16f56c8d Add 'hidden' flags to customize TLS plugins 2015-08-28 12:57:53 -07:00
Teddy Reed
5bf30a779d RocksDB usage speedups 2015-08-15 20:43:53 -07:00
Michael O'Farrell
5d0e4be6a1 Merge pull request #1335 from mofarrell/kernel-file-events
Added kernel file access events.
2015-07-31 15:22:11 -07:00
Michael O'Farrell
9f2b318778 Added kernel file access events. 2015-07-31 15:06:46 -07:00
Michael O'Farrell
b0289adcf5 Merge pull request #1414 from theopolis/env_limits
Add optional environment variable whitelist to process_events
2015-07-30 18:17:31 -07:00
Teddy Reed
dc82ffa636 Add optional environment variable whitelist to process_events 2015-07-30 16:05:11 -07:00
Michael O'Farrell
8c8c591195 Merge pull request #1404 from mofarrell/load-kernel
Added loading of kernel.
2015-07-30 15:20:33 -07:00
Michael O'Farrell
eaf7de08df Added loading of kernel. 2015-07-30 14:36:46 -07:00
Michael O'Farrell
346743e87f Benchmark using mean across 5 runs. 2015-07-29 16:50:19 -07:00
Teddy Reed
fa36a8918b Merge pull request #1401 from theopolis/tests_and_benchmarks
Various additional tests and benchmarks
2015-07-28 13:20:46 -07:00
Teddy Reed
ff9cb71628 Various additional tests and benchmarks 2015-07-28 12:26:17 -07:00
Michael O'Farrell
93a65eaf04 Merge pull request #1400 from mofarrell/process-events-env-arg
Adding environment variables and arguments for process events.
2015-07-27 17:54:06 -07:00
Michael O'Farrell
3f87d5832f Adding environment variables and arguments for process events. 2015-07-27 15:48:47 -07:00
Teddy Reed
d2effc539c [Fix #1374] Allow subscription subclassing 2015-07-26 01:48:27 -07:00
Teddy Reed
928f46c00f Merge pull request #1379 from theopolis/fix_1369
[Fix #1369] Limit IOKit HID events
2015-07-23 18:26:04 -07:00
Michael O'Farrell
66b075a685 Merge pull request #1377 from mofarrell/benchmark
Added benchmarking targets.
2015-07-23 17:37:56 -07:00
Michael O'Farrell
a65f8dd93c Added benchmarking targets. 2015-07-23 17:07:42 -07:00
Teddy Reed
7c330f0bf8 [Fix #1369] Limit IOKit HID events 2015-07-23 11:52:23 -07:00
Teddy Reed
8eaf389010 Optimize event publisher database namespace lookups.
Previously, event publishers used a canonicalized 'type' name for async callbacks.
This type was used to lookup the publisher plugin in the registry as well as for backing store namespacing.
The type is still used but subscribers, which made heavy used of the lookup, store the value locally.
This prevents unneeded publisher plugin allocation when adding events.
2015-07-19 17:10:42 -07:00
Teddy Reed
6104aaebfe Add optional TLS config plugin refresh 2015-07-17 14:59:08 -07:00
Teddy Reed
270b4da540 [Fix #1339] Add kernel-build to packages when used 2015-07-16 15:23:29 -07:00
Michael O'Farrell
019defc788 Kernel publisher only log info when not connected. [Fix #1334] 2015-07-14 20:10:50 -07:00
Michael O'Farrell
58ec6415d3 Created a basic publisher system for kernel events in the kernel extension. 2015-07-13 16:42:55 -07:00
Michael O'Farrell
4bbb591b37 Added kernel process events table. 2015-07-08 13:47:07 -07:00
Michael O'Farrell
0284b9e60d Merge branch 'master' into kernel
Conflicts:
	mkdocs.yml
2015-07-08 10:26:32 -07:00
Teddy Reed
ab56011881 Apply FIM pattern matching to inotify 2015-07-07 18:18:45 -07:00
Teddy Reed
f48619ed28 [#1285, #1276] Faster, optimized subscriber results 2015-07-07 00:59:28 -07:00
Teddy Reed
d2685cfa41 [#1142] Move path resolution into publisher logic 2015-07-07 00:45:55 -07:00
Teddy Reed
bf65e3d2d6 Event index time and streaming 2015-07-07 00:44:57 -07:00
Teddy Reed
546aaa885d [Fix #1063] Allow configure-time symlink resolution in FSEvents 2015-07-02 16:50:27 -07:00
Teddy Reed
7aac5fd358 Replace custom wildcarding with POSIX-glob
POSIX-globbing will allow event publishers/subscribers to post-check
results against glob-syntax, fnpath matching, and POSIX C-regex.
These checks are anecdotally speedy.
2015-07-02 13:53:16 -07:00
Michael O'Farrell
a00fb638c2 Added kernel event publisher. 2015-07-01 17:40:42 -07:00
Michael O'Farrell
1ab7040d83 Kernel extension fixes for daemon shutdown process. 2015-06-30 18:00:25 -07:00
Michael O'Farrell
680ffd3bc8 Added a gangsta test (gtest) for the kernel communications.
This test does not evaluate the functionality of the kernel
communication unless the KERNEL_TEST flag was set during the build.
The test will not succeed unless the tests are being run as root.
2015-06-29 12:12:54 -07:00
Michael O'Farrell
f4e05b992a Merge branch 'master' into kernel
Conflicts:
	mkdocs.yml
2015-06-26 17:04:42 -07:00
Michael O'Farrell
89fb4fbaf0 Moved kernel userland code into the osquery directory structure.
Test cpp files are dead.
2015-06-25 12:38:39 -07:00
Sharvil Shah
f676ba7642 Implements disk_events and the related publisher and subscriber.
We now have a Publisher to report on disk events and its metadata,
using the DiskArbitration framework on OS X. Currently disk appearance
and disappearance events are published for both physical and
virtual disks (DMG files). On an event trigger, disk properties are
parsed and that metadata is reported along with the action.

The Subscriber subscribes to virtual disk events currently.

This closes #1103.
2015-06-22 11:09:18 -07:00
Teddy Reed
4c80891010 Fix FSEvents multiplexing actions 2015-06-05 17:36:29 -07:00
Teddy Reed
8b3686a58a TLS plugin workflow tests 2015-05-26 19:55:00 -07:00
Teddy Reed
2a1f496cc5 Towards TLS config/logging 2015-05-19 17:05:55 -07:00
Mike Arpaia
fff36af0af Removing trailing whitespace 2015-05-11 23:31:13 -07:00
Teddy Reed
ee872d3fbe Harden events tests 2015-05-06 20:33:39 -07:00
Teddy Reed
d0bbb0bc4f Towards safer and shuffled unittests 2015-04-29 14:43:27 -07:00
Teddy Reed
be65922569 Fast tests 2015-04-27 09:40:31 -07:00
Teddy Reed
16447e67d6 [Fix #1040] Check for disabled event subs 2015-04-26 12:15:49 -07:00
Teddy Reed
c59ce0e4e4 Lint fixes and clang analyze 2015-04-17 09:18:46 -07:00
Teddy Reed
739d91c361 Performance linting 2015-04-11 19:50:35 -07:00
Teddy Reed
090f7e71f1 Fix event subscriber time 2015-04-04 13:28:45 -07:00
Teddy Reed
2b20d3dde0 Merge yara subscribers 2015-04-03 00:48:13 -07:00
Wesley Shields
a9644d22c2 Implement YARA table.
Currently only for OS X, will port to others soon.

Also need to add tests.

Remove old comment and add loading message.

Implement YARA table for Linux.

Use mask properly.

Use the various masks to specify the kinds of events we are interested
in. This removes the need to do the dirty "DELETED" check when the event
fires.

Make getYARAFiles return a const map.

Switch to LOG(WARNING) and emit error number.

Add vim .swp files to .gitignore.

Add yara_utils.(c|h).

Start to condense common code between the Linux and Darwin YARA tables
into a yara_utils.h. Right now it includes a function to compile rules
and store the results back in the map, indexed by category. It also has
the callback used by YARA when a rule is processed. I can not move much
more than that for the row creation code because the structures used in
the event callback are slightly different.

Include a better error message.

The errors are still printed by the compiler callback, but this will
allow my future work to return a Status from the event initialization to
print a useful message in summary.

Make Subscriber init() return Status.

Each EventSubscriber::init() now returns a Status. If the init() fails
for any reason the EventSubscriber is still stored but the failure is
tracked.

EventSubscribers now have a state member, which represents the current
state of the subscriber. The current supported states are:
uninitialized, running, paused, failed. Currently the only meaningful
ones are running and failed, but I put paused in there as a
forward-looking feature.

Subscriptions now have a subscriber_name member. This is used in
EventPublisherPlugin::fire() as a lookup to get the EventSubscriber and
check the state. If the EventSubscriber is not running the event will
not fire.

Only the EventSubscribers on OS X are using this. I'll do the Linux
implementation next.

Chase the init() changes to Linux.

This brings the Linux YARA table in line with the OS X one.

Require a EventSubscriberID when creating a subscription.

Now that Subscriptions are "tied" to EventSubscribers you must create a
Subscription with the name of the Subscriber it is for. This is because
when the event fires the list of Subscriptions is walked and the name is
used to lookup the EventSubscriber and make sure it is in the running
state.

Fix various tests.

Some tests would fire an event with only a Subscription, which is no
longer a valid thing to do. For these tests an EventSubscription is
created and registered in the EventFactory.

When Subscriptions are created pass the name of the EventSubscriber to
them. In some cases where no event is ever fired it is fine to pass a
bogus name.

Fix inotify tests.

Move a test down so the class is defined and make sure to create an
EventSubscriber and use it properly.

Add support for yara to provision.sh.

Right now this grabs yara 3.3.0 and applies the patch to fix min() and max(),
which is commit fc4696c8b725be1ac099d340359c8d550d116041 in the yara repo.

This has been tested under Ubuntu 14.04 only.

Remove NOMINMAX.

This is no longer necessary after the patch was backported to 3.3.0.

Revert "Add support for yara to provision.sh."

This reverts commit a8bd371498c0979f070adeff23d05571882ac3f1.

Use vendored YARA code in third-party.

This switches to using the YARA code contained in third-party, including
the patch to fix min/max macros.

Fix mismerge.

Remove unused function after merge.

Well, soon to be unused as soon as I fix up the Linux YARA table. ;)

Chase config changes.

Make the Linux YARA table use ConfigDataInstance along with files() and
yaraFiles().
2015-04-03 00:47:39 -07:00
Teddy Reed
2ba009de2d Merge pull request #889 from theopolis/config_update
Support specific config source async updating
2015-03-24 16:55:12 -07:00
Teddy Reed
5b227c8e3d Scheduler organization for async config updates 2015-03-24 16:27:35 -07:00
Teddy Reed
eee5b7d462 Reduce restart times for event runloops 2015-03-24 16:27:07 -07:00
Teddy Reed
4c4cba73c8 [Implement #593] Add a RLOG, TLOG helper macro 2015-03-08 15:20:36 -07:00
Mitchell Grenier
fc09924a59 clang format 2015-03-02 16:48:01 -08:00
Mitchell Grenier
68ea487137 Addressing theopolis changes 2015-03-02 15:43:31 -08:00
Mitchell Grenier
0031c6ed57 Fixed many bugs. inotify and fsevents should be same now 2015-02-27 17:28:51 -08:00
Mitchell Grenier
70c82b5a40 Linux inotify more closely resembles fsevents and is generally more awesome 2015-02-25 16:43:37 -08:00
Teddy Reed
247e57f2d6 Removed reinterpret plugin casts 2015-02-19 14:23:15 -08:00
Mitchell Grenier
182c69d4af Added ability to specify files to watch with wildcards 2015-02-19 12:43:23 -08:00
Teddy Reed
1f8dacec3c Add flag aliasing, logger/flag tests 2015-02-17 16:26:14 -08:00
Teddy Reed
9eeda1f02c Safer compile flags 2015-02-11 10:45:04 -08:00
Teddy Reed
23864f220d [Fix #704] Events sleep with dispatcher's interruptableSleep 2015-02-10 12:51:26 -08:00
Teddy Reed
993e2c4577 Changes to flags, extensions now loaded with shell/daemon 2015-02-06 09:40:49 -08:00
Teddy Reed
ed9bae29b7 Organizing headers/build for SDK 2015-02-03 14:59:32 -08:00
Teddy Reed
b0a91e1058 Fixing threading assumptions for FSEvents runloop 2015-02-01 05:12:28 -07:00
Teddy Reed
5072b40997 Fix missing virtual destructors for event APIs 2015-02-01 04:32:18 -07:00
Teddy Reed
e4b369917b Unref udev monitor during events tearDown 2015-02-01 03:00:09 -07:00
Teddy Reed
f96b498ae3 Remove EventFactory::deregister... in favor of ::end 2015-02-01 02:20:09 -07:00
Teddy Reed
bd620853aa Verbose log when table row is missing a column 2015-02-01 02:20:09 -07:00
Teddy Reed
d39f1fae95 Minor registry documentation, using macros for create/add 2015-02-01 02:20:09 -07:00
Teddy Reed
ab08bc76a8 Towards a new registry 2015-02-01 02:20:09 -07:00
Teddy Reed
c4fb5d45ed Added make analyze (clang-analyze) and fixed output 2015-01-31 03:09:30 -08:00
Teddy Reed
9c1faec090 Isolate glog include and depend on libglog for #652 2015-01-21 13:37:06 -08:00
Teddy Reed
367709429e Treat IOKit HID failures as warnings 2015-01-13 17:25:11 -08:00
Teddy Reed
465db46628 Fix shouldFire pubsub virtual 2015-01-11 19:51:54 -08:00
Teddy Reed
df3029e880 [Fix #559] Detach event publisher threads when ending 2015-01-05 19:07:08 -08:00
Teddy Reed
9b0adcc47f [Fix #560] Improve config tests 2015-01-01 22:05:03 -08:00
Teddy Reed
94811f3ee8 Removed 'core' tables as a build dependency 2014-12-25 12:46:59 -08:00
Teddy Reed
b2dca55539 Build leaner libosquery, allow control over spec/impl 2014-12-23 20:07:12 -08:00
Theodore M. Reed
53d683a3b3 Remove tables dependency from CMake build 2014-12-23 14:37:07 -08:00
mike@arpaia.co
b9f732c31f Updating the license comment to be the correct open source header
As per t5494224, all of the license headers in osquery needed to be updated
to reflect the correct open source header style.
2014-12-18 10:52:55 -08:00
Teddy Reed
dd2eaf248a Fixing Linux syntax errors and tests for Events 2.0 2014-12-15 16:47:09 -08:00
Teddy Reed
6de14466db Events 2.0 using pbr 2014-12-15 11:55:05 -08:00
Teddy Reed
fcdf49d17f WIP migrating Linux Events 2014-12-15 00:43:28 -08:00
Teddy Reed
17efa0b3d6 Migrate subscribers on OSX 2014-12-15 00:25:28 -08:00
Teddy Reed
fbd56663d9 Migrate fsevents to events 2.0 2014-12-14 22:17:38 -08:00
Teddy Reed
d927495209 Support casted subscribes 2014-12-14 21:20:20 -08:00
Teddy Reed
c1e37b73fb Non-static event type and name IDs 2014-12-14 18:03:41 -08:00
Teddy Reed
d2a93cf8c1 Remove EventSubscriber macros 2014-12-14 17:05:07 -07:00
Teddy Reed
0d00e4b0e9 Remove EventPublisher macros 2014-12-14 04:43:31 -07:00
Teddy Reed
cd20ed6b77 Prevent IOKitHID value subscriptions 2014-12-11 18:19:05 -08:00
Teddy Reed
acccfa94e2 IOKit HID events and OSX hardware_events table 2014-12-11 18:06:08 -08:00
Teddy Reed
7b56fa605d PCI/USB parity 2014-12-10 19:51:18 -08:00
Teddy Reed
2ebbbf6f98 Linux udev events 2014-12-08 14:13:47 -08:00
Teddy Reed
19695d40aa Add expiration to events 2014-12-06 18:28:03 -07:00
Teddy Reed
7b16e45f55 Improve pubsub unittests 2014-12-05 16:18:05 -07:00
Teddy Reed
7c738c8497 Codemod to improve include search paths 2014-12-03 15:14:02 -08:00
Teddy Reed
59367b41af Predicate constraints for Linux 2014-11-29 22:36:06 -08:00
Teddy Reed
ba86d68e68 Rebuild generated files when templates change. 2014-11-29 22:36:06 -08:00
mike@arpaia.co
5855dab22b fixing two missing semi-colon issues with clang-format 2014-11-25 09:05:16 -08:00
mike@arpaia.co
8f50cae3aa clang-format on the codebase
Periodic clang-format run.
2014-11-25 09:05:16 -08:00
Li-Wen Hsu
4f8006ad02 Add dummy table implementations for FreeBSD 2014-11-19 05:07:59 +08:00
Li-Wen Hsu
c00074e117 linux/inotify_tests.cpp is Linux only 2014-11-17 04:27:45 +08:00
Li-Wen Hsu
1ad47bbafb Make room for FreeBSD events 2014-11-16 01:41:50 +08:00
Teddy Reed
0d8b9d3eaa Use SQLite types 2014-11-12 11:07:24 -08:00
Teddy Reed
b0ff403d3d Fixing librpm API usage leaks 2014-11-10 01:48:07 -08:00
Teddy Reed
2f6369ad99 Fix boost filesystem path for inotify 2014-11-03 23:37:45 -08:00
yetanotherhacker
8cee7e0b3c Spelling fixes in comments and output. 2014-10-30 04:27:00 -04:00
Teddy Reed
6e60612520 Using clang-format 3.5 2014-10-27 17:37:36 -07:00
Teddy Reed
9d6efc83b8 [events] Fix SCNetwork runloop thrashing 2014-10-25 07:01:57 -07:00
Teddy Reed
21a0fd1aec Merge pull request #207 from facebook/scnetwork_publisher
[events] OSX SCNetwork Publisher
2014-10-16 16:27:35 -07:00
Teddy Reed
55ef15fa3d [events] OSX SCNetwork Publisher 2014-10-07 16:00:28 -07:00
Teddy Reed
ded0717e94 [events] Additional INotify tests 2014-10-07 12:27:25 -07:00
Teddy Reed
8213e7dcbc [events] Improve inotify 2014-10-06 14:37:44 -07:00
Teddy Reed
37352f862a [events] Formatting from name changes 2014-10-04 13:29:17 -07:00
Teddy Reed
a36117670b Revert "disabling inotify_tests" 2014-10-03 17:02:00 -07:00
Teddy Reed
25aee56af9 [events] Stabilize INotify event tests 2014-10-03 17:01:32 -07:00
mike@arpaia.co
96986773b3 disabling inotify_tests 2014-10-03 14:21:50 -07:00
Teddy Reed
c553a59745 [events] Use pub/sub diction for events 2014-10-03 11:30:51 -07:00
Teddy Reed
1e36b494b4 [events] Rename MonitorContext to SubscriptionContext 2014-10-03 08:26:41 -07:00
Teddy Reed
b2474b49eb [events] Renamed EventType to EventPublisher 2014-10-03 08:14:36 -07:00
Teddy Reed
e77ae22fe2 [events] Rename EventModule to EventSubscriber 2014-10-03 08:08:06 -07:00
Teddy Reed
69bfb92905 [events] Fleshing out OSX FSEvent framework 2014-10-02 21:30:14 -07:00
Teddy Reed
bf8209ca90 Merge pull request #182 from facebook/events_docs
[events] Added remaining doxy comments
2014-09-30 15:00:08 -07:00
Teddy Reed
ef044c4a72 [events] Added remaining doxy comments 2014-09-30 12:50:14 -07:00
Teddy Reed
6eb9c5fd44 EventFactory, Dispatcher as singletons 2014-09-29 20:47:24 -07:00
Teddy Reed
ed338e8356 [events] Events lifecycle complete, passwd_changes vtable 2014-09-26 12:58:32 -07:00
Teddy Reed
9220da7e3d [events] Registry integration 2014-09-24 12:43:14 -07:00
mike@arpaia.co
5f4108c503 Moving all boost smart pointers to std smart pointers 2014-09-24 10:54:59 -07:00
Teddy Reed
9a2d299424 [events] Events and registry coordination 2014-09-24 10:46:37 -07:00
Teddy Reed
974a53dd98 Merge pull request #155 from facebook/events_modules
Events modules and basic INotifyEventType
2014-09-23 13:01:59 -07:00
Teddy Reed
94953df90e [events] Flesh out inotify eventtype 2014-09-23 13:01:03 -07:00
mike@arpaia.co
4218a4c2ab cmake cleanups 2014-09-22 21:23:16 -07:00
mike@arpaia.co
9e2507409c linking tests against libosquery 2014-09-22 19:54:59 -07:00
Teddy Reed
bb7097a255 [events] EventType threads for each run loop 2014-09-22 18:35:12 -07:00
mike@arpaia.co
ebfc47b399 Edits to https://github.com/facebook/osquery/pull/148/ 2014-09-22 14:35:59 -07:00
Teddy Reed
9b42c060ea [events] Linux inotify event type 2014-09-22 01:47:50 -07:00