Commit Graph

159 Commits

Author SHA1 Message Date
Zachary Wasserman
129ec81853 Add release valve for purging buffered logs in BufferedLogForwarder (#2244) 2016-07-27 15:26:45 -07:00
yying
2fb3797c53 Changes to support building a osquery Windows service. (#2278) 2016-07-22 13:29:37 -07:00
yying
547e8f961c CMake configuration file changes to support Windows (#2258) 2016-07-20 23:48:55 -07:00
Teddy Reed
5be180a8f9 Swap removed and added for logs (#2260) 2016-07-20 12:25:10 -07:00
Teddy Reed
7f304a0934 Various fixups and best practices (#2237) 2016-07-11 09:45:57 -07:00
Teddy Reed
48cb4d555d Add systemLog API (#2229)
This includes a minor SDK refactor as it move quite a few specialized
functions and facilities from core.h into system.h. There was a breaking point
for needing to frequently update core includes.

The new logger systemLog function allows a call site to bypass logging config
and write a line to the OS logger (aka syslog).
2016-07-07 15:16:28 -07:00
Zachary Wasserman
8909602a40 Increase block period in flaky BufferedForwarder test (#2222)
This test was intermittently failing because it relies on the actual thread
scheduling. Our discussion in issue #2218 decided that it was worth keeping the
test around, while trying to mitigate the flakiness. The longer sleeps in this
test ran successfully hundreds of times in local testing.
2016-07-06 14:59:24 -07:00
artemdinaburg
e6fbde820e Windows Daemon/Shell: Make osquery code more Windows-friendly (#2188) 2016-07-01 14:56:07 -07:00
Ryan Holeman
b47f246ab4 Optional randomization of shard ids for AWS Kinesis load balancing (#2157) 2016-06-15 15:29:59 -07:00
Zachary Wasserman
79818c72ab Add newlines in firehose records (#2166) 2016-06-14 16:50:28 -07:00
Nick Anderson
cf30388705 Moved test_utils to it's own directory out of core. Updated references (#2154) 2016-06-09 10:49:26 -07:00
Ryan Holeman
deb9c56633 Changed stream validation from list streams to describe streams (#2141) 2016-06-07 17:17:15 -07:00
Zachary Wasserman
462cd8bdbf Update AWS logger code for AWS SDK 0.12.4 (#2140)
The AWS SDK changed how custom HTTP clients are used, and this commit brings
compatibility with the new initialization style.
2016-06-07 15:06:49 -07:00
Teddy Reed
a5100f7b12 Disable Google Benchmark and AWS SDK from build (#2113)
See: #2112 and #2107
2016-05-19 17:23:07 -07:00
Teddy Reed
77273f6500 Add logEvent API to logger plugins (#2088) 2016-05-13 19:48:40 -07:00
Teddy Reed
ce2ba6d9c8 Refactor logStatus and logger initialization (#2081)
The initialization of a logger plugin was confusing. The 'init' step was
introduced to allow a daemon to buffer status events before a logger plugin
is determined by external/remote configuration. The buffered statuses could
then be transferred via a medium other than Glog (the default). To determine
if Glog should continue to write statuses to the filesystem the 'init' method
returned a Status.

Logger plugins should now use a feature method override to select how status
logs should be handled.
2016-05-11 12:05:09 -07:00
Teddy Reed
0bb79c83e0 Remove the logHealth method from logger plugins (#2079) 2016-05-10 08:34:55 -07:00
Teddy Reed
921221ed46 Add osquery version to status logs (#2067) 2016-05-05 13:19:25 -07:00
Teddy Reed
b6f09a7ecb Add action for snapshot query results (#2061) 2016-05-03 11:16:22 -07:00
Zachary Wasserman
4bb95a3a29 Fix AWS credentials test failure when env vars set (#2062) 2016-05-02 19:17:09 -07:00
Zachary Wasserman
262833c86a Add AWS Kinesis and Firehose logger plugins (#2045)
This commit adds logger plugin implementations for the Amazon
Kinesis (https://aws.amazon.com/kinesis/) and Kinesis
Firehose (https://aws.amazon.com/kinesis/firehose/) services. To support
these plugins there are a number of utility classes and functions for
AWS authentication, configuration and API integration. The logger plugin
implementations take advantage of the BufferedLogForwarder base class
for reliable buffering and batch sending of logs. In their current
implementations, the logger plugins only support sending of result logs
to these AWS services.
2016-04-25 16:19:51 -07:00
Zachary Wasserman
2376fedb73 Introduce BufferedLogForwarder (#2036)
BufferedLogForwarder is a base class for external log forwarding schemes
that require buffering and retries. It generalizes the logic from the
existing TLSLogForwarderRunner.

The existing TLSLoggerPlugin and TLSLogForwarderRunner are refactored to
demonstrate the use of this new base class.
2016-04-15 09:49:35 -07:00
Zachary Wasserman
8cb862e327 Allow TLS to proceed without node key if enrollment disabled 2016-04-07 08:47:03 -07:00
Teddy Reed
2379493721 Introduce decorator queries 2016-03-29 10:03:50 -07:00
Teddy Reed
b9194026db [Fix #1973] Improve logging/extensions relays 2016-03-27 00:43:02 -07:00
Teddy Reed
d2d1431061 Move dispatcher to public API 2016-03-21 15:27:51 -07:00
Teddy Reed
0ba2861cf9 [Fix #1920] Detach thread before joining/clearing (terminate) 2016-03-13 12:15:18 -07:00
Teddy Reed
956ce75601 Remove Glog from third-party build 2016-03-07 12:54:00 -08:00
Teddy Reed
afd17f8134 1. Reorganize RocksDB database handle into a plugin
2. Introduce a SQLite-based database plugin
3. Refactor database usage to include local 'fast-calls'
4. Introduce an 'ephemeral' database plugin for testing (like a mock)
2016-03-06 20:40:16 -08:00
Teddy Reed
3101a32b01 Improve logging tests, add filesystem logger tests 2016-02-21 19:40:16 -08:00
Teddy Reed
14ff03dbf7 [Fix #1861] Add newline to all filesystem logger writes 2016-02-21 14:12:58 -08:00
Teddy Reed
2b304154d2 Merge pull request #1856 from theopolis/cppnetlib_0.12_sni
[Fix #1746] Add SNI options for TLS plugins
2016-02-21 12:29:31 -08:00
Teddy Reed
4d41f7e5ba [Fix #1746] Add SNI options for TLS plugins 2016-02-19 22:39:29 -08:00
Teddy Reed
8dc0fc1c95 Consolidate string/text conversions outside of API 2016-02-12 11:38:15 -08:00
Baraa Hamodi
21c2237eca [osquery] Update copyright headers to new format. 2016-02-11 11:48:58 -08:00
Teddy Reed
71d6107f83 Document logger_mode flag takes decimal values 2016-02-01 11:10:30 -08:00
Teddy Reed
95c4d733cc Add and document TLS debug features 2016-01-22 08:59:07 -08:00
Teddy Reed
e6408e21f9 Add max log size for TLS logger of 1M per line 2016-01-21 10:43:15 -08:00
Teddy Reed
87ea41c6ec Improve TLS logger performance 2016-01-21 10:43:15 -08:00
Teddy Reed
b9117b17a1 [Fix #1788] Use an array for the 'data' key in TLS logs 2016-01-20 11:59:14 -08:00
Teddy Reed
d6e91c81e9 Improve TLS logging memory 2016-01-15 00:22:31 -08:00
Teddy Reed
c5766da6d0 [#1518] Only emit a single line for each logString 2015-12-16 16:42:55 -08:00
Teddy Reed
9ebd292eb6 [#1648] Support multiple loggers 2015-12-06 11:10:10 -08:00
Teddy Reed
ee84f35632 Merge pull request #1645 from stripe/andrew-configure-perms
Allow setting the mode of log files
2015-11-11 13:46:24 -08:00
Andrew Dunham
4ccdcc7864 Allow setting the mode of log files
This also sets the appropriate flags in glog
2015-11-11 11:37:55 -08:00
Jaime
f7ee2437cf Removed result= in the Syslog plugin 2015-11-11 09:16:50 +01:00
Robert C. Seacord
e57828aac3 changes for integer sign problems 2015-10-17 00:18:35 +00:00
Teddy Reed
b7a2d861bf Build Glog with OS X ABI, add SKIP_BENCHMARK 2015-10-11 14:37:49 -07:00
Mike Arpaia
aaa03a1058 Distributed queries client-side 2015-09-08 13:33:48 -07:00
Teddy Reed
0e16f56c8d Add 'hidden' flags to customize TLS plugins 2015-08-28 12:57:53 -07:00
Teddy Reed
2109ae85b7 Allow ConfigParserPlugins to update the ConfigData.
Previously, `ConfigParserPlugin`s could only maintain an internal derived object called `data_`.
Then parts of the code that knew to use the plugin's data would call `getParsedData` and provide the name of the plugin.

Parser plugins can now request a mutable version of the `ConfigData` using `::mutableConfigData`.
This requires a lock on the `ConfigDataInstance` and must be provided to their mutable accessor.

Acess to a mutable config enables parsers to make modifications to internal config structures like options and the query schedule.
2015-07-18 15:08:51 -07:00
Teddy Reed
deecef81c5 Fix broken JOIN predicate passing 2015-07-16 11:29:56 -07:00
Teddy Reed
79de0a5def [#1277] Forward status logs to osqueryd workers
If watcher processes generate warning or error status logs they
will "relay" to the worker processes upon successful sanity check.
2015-07-01 15:26:26 -07:00
Teddy Reed
b24cf6f20d Add syslog plugin 2015-06-18 15:59:40 -04:00
Teddy Reed
da9bd5801b Migrate HTTP remote logger to TLS logger 2015-06-01 10:12:31 -07:00
Mitchell Grenier
418e6495c0 Adding a remote logger for osquery
The first draft of the remote logger for osquery. This should give a rough idea
of how the code will be structured and function. RFC please.

At the advice of @theopolis, I removed the category type and added the
http_logger key. We figure this should be more efficient and doesn't have to
be known at compile time.
2015-05-28 17:14:56 -07:00
Mike Arpaia
fff36af0af Removing trailing whitespace 2015-05-11 23:31:13 -07:00
Teddy Reed
c63bf0451a Various exception hardening 2015-05-03 14:18:20 -07:00
Teddy Reed
e01a73b4f3 Schedule monitoring, doc updates, logger plugin fixes 2015-05-03 11:54:15 -07:00
Teddy Reed
b66a350526 Allow snapshot scheduled items 2015-04-29 15:55:00 -07:00
Teddy Reed
d0bbb0bc4f Towards safer and shuffled unittests 2015-04-29 14:43:27 -07:00
Teddy Reed
be65922569 Fast tests 2015-04-27 09:40:31 -07:00
Teddy Reed
53782c1c7c [Fix #907] Use glog to log to data-identified files 2015-03-30 12:49:57 -07:00
Teddy Reed
14a09cc6f2 Change schedule to a map, splay on config update 2015-03-24 16:28:49 -07:00
Teddy Reed
fe0f369af0 Extension-dependent config/logger plugins 2015-03-13 12:01:30 -07:00
Teddy Reed
6e7f3dbbbd Move logger/config to use Registry getActive 2015-03-08 14:52:13 -07:00
Teddy Reed
99beceaef6 Switch lazy=active concept for registry setup 2015-03-04 18:51:41 -08:00
Teddy Reed
3c02806cd8 Extensions autoloading prequel 2015-03-04 18:51:41 -08:00
Teddy Reed
1f8dacec3c Add flag aliasing, logger/flag tests 2015-02-17 16:26:14 -08:00
Teddy Reed
fc64965c68 Fix ctor logger locking 2015-02-16 16:37:39 -08:00
Teddy Reed
6994361f26 Improved logging control 2015-02-16 14:42:22 -08:00
Teddy Reed
993e2c4577 Changes to flags, extensions now loaded with shell/daemon 2015-02-06 09:40:49 -08:00
Teddy Reed
ed9bae29b7 Organizing headers/build for SDK 2015-02-03 14:59:32 -08:00
Teddy Reed
d39f1fae95 Minor registry documentation, using macros for create/add 2015-02-01 02:20:09 -07:00
Teddy Reed
ab08bc76a8 Towards a new registry 2015-02-01 02:20:09 -07:00
Teddy Reed
9c1faec090 Isolate glog include and depend on libglog for #652 2015-01-21 13:37:06 -08:00
Theodore M. Reed
53d683a3b3 Remove tables dependency from CMake build 2014-12-23 14:37:07 -08:00
mike@arpaia.co
b9f732c31f Updating the license comment to be the correct open source header
As per t5494224, all of the license headers in osquery needed to be updated
to reflect the correct open source header style.
2014-12-18 10:52:55 -08:00
Teddy Reed
7c738c8497 Codemod to improve include search paths 2014-12-03 15:14:02 -08:00
Teddy Reed
24b7be320c Fix #328, add gflags defines for shell-internal flags 2014-11-02 15:40:35 -08:00
Teddy Reed
1554bf3295 Fix #290, add permissions to osqueryd logging 2014-10-30 15:03:05 -07:00
Teddy Reed
6e60612520 Using clang-format 3.5 2014-10-27 17:37:36 -07:00
Teddy Reed
6d50d762ce Changing flag infra, reducing config testing, adding debug macro 2014-10-27 10:30:02 -07:00
Teddy Reed
a82792b3f7 Log results as events 2014-10-24 17:05:17 -07:00
mike@arpaia.co
4218a4c2ab cmake cleanups 2014-09-22 21:23:16 -07:00
mike@arpaia.co
9e2507409c linking tests against libosquery 2014-09-22 19:54:59 -07:00
mike@arpaia.co
627821abc1 Periodic clang-format 2014-09-21 14:29:28 -07:00
mike@arpaia.co
b5ee19f49f Removing the osquery::db namespace 2014-09-21 14:27:09 -07:00
mike@arpaia.co
f06a4ba52e cleaning up the plugin interfaces 2014-09-16 01:34:39 -07:00
mike@arpaia.co
5998dbd1c5 clang-format 2014-09-16 00:36:49 -07:00
mike@arpaia.co
b7f8f5f72a moving logger to the global namespace 2014-09-15 11:14:17 -07:00
mike@arpaia.co
ad9b0bb5c1 Doxyfile, for docs 2014-09-13 15:18:26 -07:00
mike@arpaia.co
e838110e84 Moving header to include 2014-09-12 17:50:03 -07:00
Mike Arpaia
db0f0105dd Revert "Skip tests when making 'fast'" 2014-09-09 21:37:08 -07:00
mike@arpaia.co
c9fafc00d3 using '#pragma once' instead of '#ifndef HEADER'
let's start using #pragma once for our headers. it's less lines of code,
clang supports it, headers become more movable, etc. it's all around a
better plan.
2014-09-09 18:54:53 -07:00
Teddy Reed
2e150ef8a9 Skip tests when making 'fast' 2014-09-09 16:25:22 -07:00
Mike Arpaia
d6699bd0fe Adding header files to CMakeLists.txt so that other build tools can perform better introspection into the codebase. 2014-09-09 10:53:59 -07:00
mike@arpaia.co
c72d069689 vagrant and make deps on linux 2014-09-08 19:24:23 -07:00
Teddy Reed
7e470747b4 Moving sublibs to single libosquery 2014-09-08 01:58:29 -07:00
Teddy Reed
4ffd184eaf Changes for Linux (Ubuntu 14.04) build 2014-09-05 10:58:58 -07:00