Commit Graph

5224 Commits

Author SHA1 Message Date
Teddy Reed
0a68a695e5
audit: Allow configuring audit_backlog_wait_time (#5953)
Read the analysis within #5952.

There are multiple sources suggesting an optimal value is 0. Our
implementation previously hardcoded the value to 1. It is now 0 and can
be configured to a different value using the audit_backlog_wait_time
option or command line flag.
2019-10-31 12:05:00 -04:00
Teddy Reed
b0bf653745
powershell/config: Add checks for character_frequencies (#5945)
Co-Authored-By: Stefano Bonicatti <smjert@gmail.com>
2019-10-31 12:03:53 -04:00
Teddy Reed
9a336877fd
cpack: Update build-id and create debug RPMs (#5936) 2019-10-31 11:39:46 -04:00
Stefano Bonicatti
dee0212433 Fix a race condition between Thrift server stop() and listen() (#5937)
If a Thrift server is requested to stop before everything is
properly initialized a race condition could happen.
Specifically if stopping during a listen() a race condition will happen
accessing childInterruptSockWriter_, which is a socket initialized by listen(),
in a non thread-safe way, and used by stop() to notify other threads
that they should shutdown.
2019-10-31 08:38:46 -04:00
Stefano Bonicatti
4768b0997a
Explicitly select Xcode version 10.3 in azure-pipelines.yml (#5973) 2019-10-30 22:46:36 +01:00
packetzero
ceb70dee9e new: aws_session_token support (#5944) 2019-10-30 15:23:53 -04:00
Teddy Reed
04896c85cd
killswitch: Remove killswitch code (#5949)
This was determined to be the wrong approach to adding simple
killswitches. The intent was to quickly flip on/off features. It was
not widely adopted due to the dependencies killswitching has.

A different approach is to implement the same functions with something
simple like filesystem flags.
2019-10-29 19:14:35 -04:00
Teddy Reed
b47e34936c
cpack: Add Debian copyright file and Augeas license (#5951) 2019-10-29 19:03:27 -04:00
howar6hill
b67736d390 improve etc_protocols test (#5957) 2019-10-29 19:01:58 -04:00
Tom Ritter
3c384bf392 Update osqueryfuzz_config_dict.sh to correct tests and packs paths (#5963) 2019-10-29 19:01:12 -04:00
Teddy Reed
d373d04260
logger: Restore TLS logger runtime flags (#5962)
This is a follow up to #5956, which reverts the CLI_FLAG changes.
2019-10-29 00:43:06 -04:00
Teddy Reed
5180dfaa70 tests: Add test for JSON exponent parsing (#5941)
There is a bug in RapidJSON 1.1.0 that results in wild pointer reads
when parsing exponents. We want to prevent regressing.

This requires updating the Buck/pre-built libraries.
2019-10-29 02:16:15 +01:00
Stefano Bonicatti
d2e976168d
Restore correct commit of the RapidJSON submodule (#5961) 2019-10-29 02:15:42 +01:00
packetzero
f0d00b44b2 Advert index not implemented (#5938) 2019-10-28 20:27:06 -04:00
Tom Ritter
abd16c9b1f Update osqueryfuzz_config_corpus.sh to correct the path to the tests directory (#5955) 2019-10-28 20:19:00 -04:00
Teddy Reed
0c06658e45 logger: Expose max_lines and correct flag types (#5956)
Multiple flags are moved from FLAG to CLI_FLAG due to the way they are
implemented/used in code. If they were FLAG(s), meaning if they were
also configurable via configuration at runtime, the new values would
still be ignored.

These are:
  - logger_tls_endpoint
  - logger_tls_period
  - logger_tls_max

The flag logger_tls_max has been renamed to logger_tls_max_linesize and
an alias is added for compatibility.

A new flag is added, logger_tls_max_lines, to expose a previously
hardcoded value of 1024 for the maximum number of log lines to send
every period.
2019-10-28 10:19:50 +01:00
Teddy Reed
8b54ba9b16 http: Minor fixups to Client (#5946) 2019-10-28 10:10:08 +01:00
Teddy Reed
75c6d6545e docs: Add reference to buffered_log_max option (#5954) 2019-10-27 21:20:21 -07:00
Teddy Reed
6978d31277 buck: Detect git version (#5950) 2019-10-27 20:51:16 -07:00
Nick Anderson
b9bdcf40ce
tests: enabling certificates table sanity checks (#5914) 2019-10-27 20:45:14 -07:00
Stefano Bonicatti
62336ee8bb Fix a race condition during the shutdown of the worker process (#5943)
When a thread different from the main requests a shutdown
through Initializer::requestShutdown, it should not call
waitForShutdown; there's no reason to wait, moreover the function
doesn't only wait, but also actually stops other components and then
finally calls exit().

Since the main thread is already inside the waitForShutdown call
waiting on Dispatcher::joinServices or inside the shutdown() callable
on Windows, having a secondary thread do
the same work potentially at the same time is wrong.
Moreover calling exit() from a secondary thread is most of the time
incorrect.

The waitForShutdown function has been renamed to waitThenShutdown
to better represent what it's actually doing.
2019-10-27 17:09:50 -04:00
Teddy Reed
1803972b2f
cpack: Use productbuild shim to override package_id (#5934) 2019-10-27 13:23:01 -04:00
Stefano Bonicatti
33e5fcb3b6
Add elfutils to the CI Dockerfile (#5940)
Needed to create RPM debuginfo packages.

Also update the azure-pipelines.yml to use the new Docker image.
2019-10-27 16:27:11 +01:00
Tom Ritter
2acf0b9177 Add a fuzz target for SQL query (#5923) 2019-10-27 10:20:44 -04:00
Tom Ritter
f637199511 Add scripts to generate a dictionary and seed corpus for the config fuzzing (#5915) 2019-10-27 10:20:15 -04:00
Teddy Reed
198f553036
config: Add check for non-object multi-packs (#5935) 2019-10-26 18:06:48 -04:00
Will Newton
a1e4191109 tests: Increase reliability of interfaces test (#5879)
On macOS it is possible to have interfaces that have no netmask
set, for example:

| utun1 | 100.xx.xx.xx | | | 100.xx.xx.xx     | unknown |

This causes the interfaces test to fail. This change allows the
mask field to be blank and the test to pass on systems with this
type of interface configured.

The sockaddr for this interface has an ifa_netmask with sa_family
set to zero (AF_UNSPEC) which causes getnameinfo(3) to fail.
ifconfig does return a netmask for this interface but it's not
clear if the value is at all meaningful.
2019-10-26 12:36:45 -04:00
uptycs-nishant
bf25d02c86 Removing code repetition and fixing timer bug (#5891) 2019-10-26 12:34:55 -04:00
Chris Broome
4927bf6877 RegistryInterface: add locks to more methods (#5906) 2019-10-26 12:34:03 -04:00
packetzero
08e0b9078b Fix WHERE in(N items) results in table query running N times (#5924) 2019-10-26 10:33:56 -04:00
Alessandro Gario
41fb4efba1
CMake: Migrate macOS and Windows to the OpenSSL formula (#5928) 2019-10-26 14:03:31 +02:00
packetzero
4f5854c93d windows processes: Implement missing pid index (#5919) 2019-10-25 19:32:03 -04:00
howar6hill
ebbc12a94a remove the value_max flag (#5926) 2019-10-25 19:30:43 -04:00
seph
3e1dd14d8d Add platform_mask (#5898)
Right now, there is no way to tell what platform osquery is running on. We have `os_version.platform` and `os_version.platform_like`, but they are highly inconsistent, and require a lot of custom parsing to use. We should expose the underlying platform bitmask.

This adds the `platform_mask` to the `osquery_info` table.

This replaces https://github.com/osquery/osquery/pull/5488
2019-10-23 12:24:26 -04:00
Teddy Reed
43357d43a3
libraries: Update RapidJSON to master 1a825d24 (#5913) 2019-10-21 22:41:49 -04:00
Teddy Reed
c948c403b8
docs: Nitpicks to improve new-user experience (#5912) 2019-10-21 22:41:04 -04:00
Will Newton
df8663516e tests: Add sanity test for nvram (#5903)
Add a simple sanity test for macOS nvram.

Closes: #5019
2019-10-20 22:17:39 -04:00
Teddy Reed
13290ed7ac
leak: Fix memory leak when SQL query fails (#5911)
There is a memory leak when a SQL statement is evaluated and returns an
error while stepping. An example includes returning sqlite3_result_error
from regexStringMatchFunc. The root cause is not finalizing a prepared
statement.
2019-10-20 14:59:07 -04:00
Tom Ritter
1642382ff9 Move fuzzing harness to a new directory and refactor fuzzing constants (#5909) (#5910)
This commit creates a new osquery/main/harnesses directory and moves fuzz-config there.

It removes OSQUERY_FUZZ for a new option OSQUERY_ENABLE_FUZZER_SANITIZERS
It creates a new option OSQUERY_ENABLE_ADDRESS_SANITIZER

The following behaviors are intended:

OSQUERY_BUILD_TESTS=ON will build the fuzzing harness. This configuration is not
intended for fuzzing purposes. OSQUERY_ENABLE_FUZZER_SANITIZERS=ON will also build
the fuzzing harness. However if this variable is true, it also requires
OSQUERY_ENABLE_ADDRESS_SANITIZER=ON and either CMAKE_BUILD_TYPE=Release or
RelWithDebInfo This configuration is actually intended for fuzzing.
2019-10-20 14:09:45 -04:00
Teddy Reed
5ff42eccb6
packaging: Fix up make_linux_package to work for version 4+ (#5825) 2019-10-20 11:54:42 -04:00
Teddy Reed
15d522f447
json: Configure rapidjson to use iterative parsing (#5893) 2019-10-19 15:25:21 -04:00
Zachary Wasserman
0bf2245396 Migrate rapidjson to source build on macOS and Windows (#5899) 2019-10-18 20:05:22 -04:00
kumarak
9346926030 Restore extension SDK support to osquery 4.x (#5851)
CMake utility functions(generateOsqueryExtensionGroup,addOsqueryExtensioni{Ex},
addOsqueryModule) to add extensions.

Updates the boost library submodules needed for extensions build.

Use cache variable(OSQUERY_EXTENSION_GROUP_NAME) for the grouping of extensions
and fix the linkage with osquery interface library.
2019-10-18 18:45:20 -04:00
Stefano Bonicatti
3ad7f65a0d Support source and build directory to be on different filesystems (#5907)
Instead of moving a patched submodule from source to build directory,
we copy it and then hard reset the submodule to its original HEAD.

Minor message change when a library fails to be imported.
2019-10-18 18:25:05 -04:00
Zachary Wasserman
30a7507c95
Fix shallow clone support for libraries in recent git versions (#5897)
Enables the v2 protocol which allows the repositories to be shallow
cloned without error.
2019-10-17 21:19:24 -07:00
Will Newton
599e9d6f3a macOS keychain sanity test (#5885)
When requesting kSecClassIdentity items from the keychain sometimes
invalid items are returned. These cause errSecInvalidItemRef to
be returned from SecKeychainItemCopyAttributesAndData and result
in an empty row in the table. Catch the error and avoid returning
empty rows.
2019-10-17 08:50:17 -04:00
Nick Anderson
3957d8e948
version: fix windows os codename lookup (#5887) 2019-10-16 21:42:06 -07:00
Allan Liu
e1e75aac88 genPCIDevices for linux refactor (#5446) 2019-10-16 16:27:29 -04:00
Zachary Wasserman
c0876408fc Migrate AWS SDK CPP dependency to source build on macOS and Windows (#5889) 2019-10-16 16:20:22 -04:00
William Woodruff
1ca47d3e7a (Windows) New table: default_environment (#5441) 2019-10-16 14:20:52 -04:00