Commit Graph

438 Commits

Author SHA1 Message Date
Teddy Reed
0e9733f94c Simplify Registry and plugin concepts (#2887) 2017-01-07 12:21:35 -08:00
Nick Anderson
91ecf22e44 Updating processes table to contain UID and GID (#2889) 2017-01-04 08:41:06 -08:00
Jonathan Lee
383e07e5be [Fix #2734] Remove OpenSSL link dependency for osquery core (#2750) 2016-12-22 00:37:59 -08:00
Teddy Reed
ae19b7797d Allow EINTR within ThriftTransport (#2879) 2016-12-16 18:12:02 -08:00
Phil Christensen
1c21fe4208 C++ conformance fixes (MSVC /permissive-) (#2860) 2016-12-12 00:23:08 -08:00
Teddy Reed
0017de5bf1 Add memory and utilization limit override flags (#2858) 2016-12-11 21:59:32 -08:00
Nick Anderson
b6b4ced56c Fixing python installation path in Chocolatey (#2857) 2016-12-09 17:18:44 -08:00
lambda-conjecture
cd761d1749 Fix deadlock in RocksDB log callback (#2749) 2016-12-02 23:24:08 -08:00
Teddy Reed
bf2efcb8c0 Use syscalls for dropping effective gid/uid on POSIX (#2806) 2016-11-29 09:30:56 -08:00
Teddy Reed
555d59eff6 Remove fsuid permissions drop for OS X and Linux (#2805) 2016-11-28 22:05:08 -08:00
Teddy Reed
59f0bad67c Additional compiler checks, including shadow (#2486) 2016-11-06 01:17:04 -08:00
Teddy Reed
b04736631a Add --extensions_require feature (#2672) 2016-10-24 18:13:44 -07:00
Teddy Reed
5bb5ae1030 Add optional default flagfile /etc/osquery/osquery.flags.default (#2673) 2016-10-22 16:56:32 -07:00
Teddy Reed
93b260025a Allow distributed plugin changes and reduce ifdefs (#2670) 2016-10-22 00:27:04 -07:00
Aditya Srivastava
ef4f8af3b8 Issue #2651 : Changed all NULLs to nullptrs (#2657) 2016-10-21 11:20:28 -07:00
Nick Anderson
81d2794b26 platformGetUid returns 0 for Administrator user (#2643) 2016-10-19 10:25:32 -07:00
Teddy Reed
ab57130178 [Fix #2630] Remove 'definition' TablePlugin action (#2633) 2016-10-18 00:15:38 -07:00
Teddy Reed
0003e72c63 [Fix #2631] osqueryd: Ignore pidfile parsing errors (#2634) 2016-10-18 00:14:48 -07:00
Teddy Reed
02b21d00c3 Add dropToUser method to privileges dropper interface (#2624) 2016-10-14 15:25:54 -07:00
Nick Anderson
208d2324d5 Extending chrome browser extension table to Windows (#2619) 2016-10-14 10:23:37 -07:00
Nick Anderson
616d9f5953 Adding support for Windows platform_info table (#2611) 2016-10-12 09:01:32 -07:00
Teddy Reed
b7c5ee31b2 Return hardware UUIDs without potential trailing NULL bytes (#2616) 2016-10-11 17:55:48 -07:00
Nick Anderson
07a2a3c292 Updating processes table to include memory (#2573) 2016-10-02 22:41:05 -07:00
Nick Anderson
b69981584f Fixing COM memory leak (#2583) 2016-10-02 12:15:17 -07:00
Teddy Reed
4d1451c9b4 Add extensions SDK incompatibility checking (#2527) 2016-09-26 17:32:41 -07:00
Teddy Reed
7aa1762f52 Promote host UUID to version 2 (#2525) 2016-09-26 12:30:05 -07:00
Nick Anderson
3a351ebf43 Adding windows system_info virtual table (#2521) 2016-09-26 11:08:57 -07:00
Teddy Reed
97bc369b6a Attempt to query platform UUID on Linux (#2522) 2016-09-25 17:55:02 -07:00
Teddy Reed
62edfd46fe Toggle --utc to true (#2504) 2016-09-23 10:14:27 -07:00
Teddy Reed
6ac58f17d6 Remove extensions retry and introduce watcher retry (#2498) 2016-09-21 16:17:30 -07:00
Teddy Reed
a6589c49e3 [Fix #2482] Use atomic member in Dispatcher tests (#2494) 2016-09-21 10:52:52 -07:00
yying
a7af70d021 Adding remote config/logging capabilities to Windows build (#2469) 2016-09-20 14:18:58 -07:00
Mitchell Grenier
072a93ccac Accelerated checkins (#2454) 2016-09-12 16:53:42 -07:00
yying
a27d6567e4 Core and Additional Tests (#2441) 2016-09-12 09:46:52 -07:00
yying
84e6a3401a Reducing compiler warnings and fails on warn in VS (#2433) 2016-09-02 15:04:03 -07:00
Nick Anderson
7c90823a0c Upgrade LLVM to 3.8.1 on Linux (#2436) (#2435) 2016-09-02 14:53:04 -07:00
yying
d347c847e1 Support for extensions (#2363) 2016-08-31 16:45:06 -07:00
Teddy Reed
65dd56e113 Introduce table 'attributes' (#2431) 2016-08-31 15:32:20 -07:00
Teddy Reed
d6e20279d8 Use LOG(INFO) and set INFO as default logging mode (#2420)
This is fairly important, it changes the default mode for what status
events are logged to INFO. It had been set to WARNING and INFO was relatively
unused.

This also removes expected support for RocksDB 'in-memory' databases.
If a shell-user requests a database via CLI flags it will now work.
2016-08-31 15:09:01 -07:00
Teddy Reed
132fa3a753 Display flags in alphabetical order (#2407) 2016-08-24 17:44:16 +01:00
Teddy Reed
1b75972181 [Fix #2387] Attempt to create user's osquery homedir (#2395) 2016-08-17 17:27:46 -07:00
Teddy Reed
f3f605e26a Introduce a PLATFORM_MASK and isPlatform (#2334)
Along with the platform defines and platform string defines provided by
CMake to the build, add a PLATFORM_MASK define.

Use this define as a platform-type mask with the PlatformType enum.
2016-08-09 20:27:42 -07:00
yying
50487c6880 Changes to make pidfile work on Windows (#2297) 2016-07-28 16:04:34 -07:00
yying
2fb3797c53 Changes to support building a osquery Windows service. (#2278) 2016-07-22 13:29:37 -07:00
Teddy Reed
c22f6147ea Move OSQUERY_HOME into core and use as filesystem config default (#2275) 2016-07-21 13:28:23 -07:00
Teddy Reed
6df4c8c4d4 The watcher process should apply memory limits to itself (#2263) 2016-07-21 12:33:14 -07:00
Zachary Wasserman
f1f00cec2b Enable DB in osqueryi when --database_path specified (#2268)
Prior to this change, both --disable_database=false and --database_path had to
be specified together. Now, if the user specifies --database_path the database
is enabled automatically.
2016-07-20 17:44:50 -07:00
artemdinaburg
78e1cf7ab4 Transition __attribute__((constructor)) to a more platform independent approach (#2233) 2016-07-14 14:19:33 -07:00
Teddy Reed
45530c0496 Slight performance improvments (#2242) 2016-07-12 19:16:50 -07:00
Teddy Reed
7f304a0934 Various fixups and best practices (#2237) 2016-07-11 09:45:57 -07:00
Teddy Reed
48cb4d555d Add systemLog API (#2229)
This includes a minor SDK refactor as it move quite a few specialized
functions and facilities from core.h into system.h. There was a breaking point
for needing to frequently update core includes.

The new logger systemLog function allows a call site to bypass logging config
and write a line to the OS logger (aka syslog).
2016-07-07 15:16:28 -07:00
Teddy Reed
1f9739eed4 [Fix #2203] Restore extension respawn limits to 20s (#2207) 2016-06-30 17:53:05 -07:00
Teddy Reed
753c023640 Fix milli/micro conversion when waiting for active plugins (#2205) 2016-06-30 13:31:32 -07:00
artemdinaburg
5292100c22 Use vector.data() to get internal vector buffers (#2204) 2016-06-30 12:32:26 -07:00
Teddy Reed
47e1a5ba74 Use noexcept boost::filesystem overloads (#2195) 2016-06-27 16:39:04 -07:00
Teddy Reed
c8c67c455a Simplify watchdog limits configuration (#2173) 2016-06-19 23:27:16 -07:00
Nick Anderson
cf30388705 Moved test_utils to it's own directory out of core. Updated references (#2154) 2016-06-09 10:49:26 -07:00
Teddy Reed
91b34010df Allow table specs to use multiple row indexes (#2146) 2016-06-07 17:13:48 -07:00
Teddy Reed
763f4e9437 Use SQLite 3.14.0 to support LIKE and EQUALS (#2137)
This commit bumps the third-party SQLite to the 3.14.0 pre-release (18:59).
With 3.14.0 the LIKE and EQUALS constraint operators may be mixed within a
query. Previously these would fail to produce a valid set.

As part of the support, each virtual table should choose to bypass rowid-based
deduplication using the new "WITHOUT ROWID" create table epilog. This will
be appended to the schema if the table defines a PRIMARY KEY using index=True.
2016-06-06 09:36:53 -07:00
yying
5f81cd9e2b Filesystem Abstractions - Code and Unit Tests (#2119)
* Implemented filesystem operations abstraction code
* Added filesystem operations abstraction unit tests
* Modified CMake configurations to support the building of the abstraction code and unit tests
2016-05-31 12:20:38 -07:00
Teddy Reed
10664a4945 Add --ephemeral for daemons and disable shell events (#2111)
This changes several initialization steps:
- The daemon (and shell, though not needed) have a new --ephemeral flag.
- Events are now disabled in the shell by default, use --nodisable_events to
  re-enable.
- RocksDB-based backing storage is now disabled in the shell by default.

The --ephemeral flag for the daemon is disabled by default and will allow
skipping configuration and database path sanity, and skipping pidfile checks.
This is intended to be used when debugging or monitoring the daemon process.

To make the RocksDB backing storage feature usage very clear we introduce a new
flag: --disable_database. The shell sets this to true unless overridden in
a flagfile or via command line arguments.
2016-05-20 08:39:04 -07:00
Teddy Reed
c4acfe89e5 Introduce table aliases (#2104) 2016-05-19 09:40:43 -07:00
Teddy Reed
b28c4d8d0f Introduce table options (#2101)
Table options includes a change to the Registry::call API for TablePlugins.
When requesting route information or the 'columns' action, a new 'op' key is included.
2016-05-18 12:23:52 -07:00
yying
26ad131c38 Building osquery unit tests on Windows 10 (#2100)
Integrated process abstraction code into more locations
Defined new macros for abstracting across various platforms
Added GLOG_NO_ABBREVIATED_SEVERITIES for glog to support Windows
Fixed some minor CMake issues involving thrift
Updated gflags package; reflecting change in provision script
Preparing CMake config files for WIN32 support
2016-05-17 12:39:11 -07:00
yying
15d1455957 Process Operations - osquery/core Integration (#2087)
This integrates the process abstraction operations within osquery core.
2016-05-13 19:47:45 -07:00
Teddy Reed
3813f09abb Clean up host identifier UUID access (#2092)
When selecting UUID as the host identifier the client follows:
- If host_identifier is set to hostname, return the hostname
- If UUID check a static UUID container
- If empty request the UUID from the backing storage, RocksDB
- If empty use the Apple hardware UUID or generate a UUID
- If the Apple UUID request fails generate a UUID

When no UUID is available in the backing store save the new UUID.
2016-05-12 08:40:37 -07:00
yying
63c0258aec Abstracted platform specific process operations into a common interface. (#2069)
Added Windows support for process operations.
Added unit tests for process abstraction code for POSIX and Windows.
Modified CMake config files to support building the new code and unit tests.
2016-05-11 14:16:32 -07:00
Teddy Reed
947799d28e Introduce within-query caching (#2077)
This adds a new optimization feature that allows expensive tables to cache
their results between JOINs. Consider JOINing a list of open sockets, for each
process, then requesting to hash each process path. This query may hash
the same path multiple times.

Within-query caching allows the hash table to respond with the previous
result of the hash request as long as the requested computation was the result
of a single query. Subsequent queries will perform subsequent hashing.
2016-05-09 10:32:33 -07:00
Teddy Reed
5a56805970 Introduce --utc flag to convert all calendar UNIX times to UTC (#2064)
Beginning in version 1.8.0 all time uses will converge on an osquery-provided
getUnixTime() API call that returns, by default, UNIX time integers converted
to UTC/GMT. The 'time' table will respond with the parsed time for the
configuration. If the timezone is not UTC then osquery is using localtime.

This configuration option will affect the 'unix_time' response in the 'time'
table. Because of this configurable-effect the table is extended to include
'local_time' which is always the system local UNIX time.
2016-05-03 13:00:31 -07:00
Teddy Reed
2a350afb10 Merge pull request #1974 from theopolis/sane_logging_disable
[Fix #1973] Improve logging/extensions relays
2016-03-27 01:00:16 -07:00
Teddy Reed
b9194026db [Fix #1973] Improve logging/extensions relays 2016-03-27 00:43:02 -07:00
Zachary Wasserman
0eff0f2f4e Collection of doc fixes
- Minor fix in creating tables documentation
- Fix docs for join
- Add note about config paths
- Update events docs
- Fix link to query packs
2016-03-25 14:12:41 -07:00
Teddy Reed
d2d1431061 Move dispatcher to public API 2016-03-21 15:27:51 -07:00
Teddy Reed
27f4754e5e [Fix #1928] Reorder deps installs to build all with clang 2016-03-20 19:15:44 -07:00
Teddy Reed
ebb0ab30ce Fix shell's --profile switch 2016-03-20 16:05:13 -07:00
Teddy Reed
15a998e54f Use the default shutdown flow within extensions 2016-03-20 01:45:49 -07:00
Teddy Reed
c62a0f41b6 Various cleanups 2016-03-18 10:40:07 -07:00
Teddy Reed
0ba2861cf9 [Fix #1920] Detach thread before joining/clearing (terminate) 2016-03-13 12:15:18 -07:00
Teddy Reed
3de52846d0 Remove boost::thread 2016-03-11 11:50:44 -08:00
Teddy Reed
5f8fa32e21 [Fix #1901] Tear down TLS server runner 2016-03-07 09:38:37 -08:00
Teddy Reed
afd17f8134 1. Reorganize RocksDB database handle into a plugin
2. Introduce a SQLite-based database plugin
3. Refactor database usage to include local 'fast-calls'
4. Introduce an 'ephemeral' database plugin for testing (like a mock)
2016-03-06 20:40:16 -08:00
Teddy Reed
08a1d852a2 Add CXXFLAGS to deps build 2016-02-22 22:12:06 -08:00
Teddy Reed
9c42ba51b3 Merge pull request #1858 from theopolis/sqlite_3.11
Bump sqlite to 3.11.0
2016-02-21 23:35:01 -08:00
Teddy Reed
9a54af29ce Bump sqlite to 3.11.0 2016-02-21 22:40:37 -08:00
Teddy Reed
8dc0fc1c95 Consolidate string/text conversions outside of API 2016-02-12 11:38:15 -08:00
Baraa Hamodi
21c2237eca [osquery] Update copyright headers to new format. 2016-02-11 11:48:58 -08:00
Teddy Reed
48a1adf77b Allow extensions autoloading in osqueryi 2016-02-10 10:20:23 -08:00
Teddy Reed
02eb57fc47 Merge pull request #1832 from theopolis/pack_valid
Valid bool in packs for shard/plaform/version checking
2016-02-06 20:29:55 -08:00
Teddy Reed
a2754a01ef Valid bool in packs for shard/plaform/version checking 2016-02-06 17:54:56 -08:00
Teddy Reed
1deee80bf2 Merge pull request #1829 from theopolis/dispatcher_refactor
Refactor dispatcher shutdown logic
2016-02-05 15:17:05 -08:00
Teddy Reed
7f37304c77 Refactor dispatcher shutdown logic 2016-02-05 01:29:42 -08:00
Teddy Reed
b91dc379b3 Handle restricted home directories 2016-02-04 13:43:24 -08:00
Teddy Reed
87ea41c6ec Improve TLS logger performance 2016-01-21 10:43:15 -08:00
Teddy Reed
51fd73c449 Assure dropTo can be used safely consecutively 2015-12-14 21:27:00 -08:00
Teddy Reed
fbc8fb92dc Allow --config_dump with watcher 2015-12-14 16:19:37 -08:00
Teddy Reed
a99b62a31d Preserve atime and mtime by default for readFile 2015-12-11 22:18:45 -08:00
Teddy Reed
59750ec87d Speed up file hashing 2015-12-11 00:36:16 -08:00
Teddy Reed
309944c586 Configuration triggered publisher reconfiguration 2015-12-08 14:03:35 -08:00
Teddy Reed
02c2b37a5d Merge pull request #1709 from theopolis/expire_results
[Fix #1694] Expire results for 'old' scheduled queries
2015-12-07 14:01:44 -08:00
Teddy Reed
12716496aa [Fix #1694] Expire results for 'old' scheduled queries 2015-12-07 12:23:43 -08:00