Commit Graph

5310 Commits

Author SHA1 Message Date
Tom Ritter
abd16c9b1f Update osqueryfuzz_config_corpus.sh to correct the path to the tests directory (#5955) 2019-10-28 20:19:00 -04:00
Teddy Reed
0c06658e45 logger: Expose max_lines and correct flag types (#5956)
Multiple flags are moved from FLAG to CLI_FLAG due to the way they are
implemented/used in code. If they were FLAG(s), meaning if they were
also configurable via configuration at runtime, the new values would
still be ignored.

These are:
  - logger_tls_endpoint
  - logger_tls_period
  - logger_tls_max

The flag logger_tls_max has been renamed to logger_tls_max_linesize and
an alias is added for compatibility.

A new flag is added, logger_tls_max_lines, to expose a previously
hardcoded value of 1024 for the maximum number of log lines to send
every period.
2019-10-28 10:19:50 +01:00
Teddy Reed
8b54ba9b16 http: Minor fixups to Client (#5946) 2019-10-28 10:10:08 +01:00
Teddy Reed
75c6d6545e docs: Add reference to buffered_log_max option (#5954) 2019-10-27 21:20:21 -07:00
Teddy Reed
6978d31277 buck: Detect git version (#5950) 2019-10-27 20:51:16 -07:00
Nick Anderson
b9bdcf40ce
tests: enabling certificates table sanity checks (#5914) 2019-10-27 20:45:14 -07:00
Stefano Bonicatti
62336ee8bb Fix a race condition during the shutdown of the worker process (#5943)
When a thread different from the main requests a shutdown
through Initializer::requestShutdown, it should not call
waitForShutdown; there's no reason to wait, moreover the function
doesn't only wait, but also actually stops other components and then
finally calls exit().

Since the main thread is already inside the waitForShutdown call
waiting on Dispatcher::joinServices or inside the shutdown() callable
on Windows, having a secondary thread do
the same work potentially at the same time is wrong.
Moreover calling exit() from a secondary thread is most of the time
incorrect.

The waitForShutdown function has been renamed to waitThenShutdown
to better represent what it's actually doing.
2019-10-27 17:09:50 -04:00
Teddy Reed
1803972b2f
cpack: Use productbuild shim to override package_id (#5934) 2019-10-27 13:23:01 -04:00
Stefano Bonicatti
33e5fcb3b6
Add elfutils to the CI Dockerfile (#5940)
Needed to create RPM debuginfo packages.

Also update the azure-pipelines.yml to use the new Docker image.
2019-10-27 16:27:11 +01:00
Tom Ritter
2acf0b9177 Add a fuzz target for SQL query (#5923) 2019-10-27 10:20:44 -04:00
Tom Ritter
f637199511 Add scripts to generate a dictionary and seed corpus for the config fuzzing (#5915) 2019-10-27 10:20:15 -04:00
Teddy Reed
198f553036
config: Add check for non-object multi-packs (#5935) 2019-10-26 18:06:48 -04:00
Will Newton
a1e4191109 tests: Increase reliability of interfaces test (#5879)
On macOS it is possible to have interfaces that have no netmask
set, for example:

| utun1 | 100.xx.xx.xx | | | 100.xx.xx.xx     | unknown |

This causes the interfaces test to fail. This change allows the
mask field to be blank and the test to pass on systems with this
type of interface configured.

The sockaddr for this interface has an ifa_netmask with sa_family
set to zero (AF_UNSPEC) which causes getnameinfo(3) to fail.
ifconfig does return a netmask for this interface but it's not
clear if the value is at all meaningful.
2019-10-26 12:36:45 -04:00
uptycs-nishant
bf25d02c86 Removing code repetition and fixing timer bug (#5891) 2019-10-26 12:34:55 -04:00
Chris Broome
4927bf6877 RegistryInterface: add locks to more methods (#5906) 2019-10-26 12:34:03 -04:00
packetzero
08e0b9078b Fix WHERE in(N items) results in table query running N times (#5924) 2019-10-26 10:33:56 -04:00
Alessandro Gario
41fb4efba1
CMake: Migrate macOS and Windows to the OpenSSL formula (#5928) 2019-10-26 14:03:31 +02:00
packetzero
4f5854c93d windows processes: Implement missing pid index (#5919) 2019-10-25 19:32:03 -04:00
howar6hill
ebbc12a94a remove the value_max flag (#5926) 2019-10-25 19:30:43 -04:00
seph
3e1dd14d8d Add platform_mask (#5898)
Right now, there is no way to tell what platform osquery is running on. We have `os_version.platform` and `os_version.platform_like`, but they are highly inconsistent, and require a lot of custom parsing to use. We should expose the underlying platform bitmask.

This adds the `platform_mask` to the `osquery_info` table.

This replaces https://github.com/osquery/osquery/pull/5488
2019-10-23 12:24:26 -04:00
Teddy Reed
43357d43a3
libraries: Update RapidJSON to master 1a825d24 (#5913) 2019-10-21 22:41:49 -04:00
Teddy Reed
c948c403b8
docs: Nitpicks to improve new-user experience (#5912) 2019-10-21 22:41:04 -04:00
Will Newton
df8663516e tests: Add sanity test for nvram (#5903)
Add a simple sanity test for macOS nvram.

Closes: #5019
2019-10-20 22:17:39 -04:00
Teddy Reed
13290ed7ac
leak: Fix memory leak when SQL query fails (#5911)
There is a memory leak when a SQL statement is evaluated and returns an
error while stepping. An example includes returning sqlite3_result_error
from regexStringMatchFunc. The root cause is not finalizing a prepared
statement.
2019-10-20 14:59:07 -04:00
Tom Ritter
1642382ff9 Move fuzzing harness to a new directory and refactor fuzzing constants (#5909) (#5910)
This commit creates a new osquery/main/harnesses directory and moves fuzz-config there.

It removes OSQUERY_FUZZ for a new option OSQUERY_ENABLE_FUZZER_SANITIZERS
It creates a new option OSQUERY_ENABLE_ADDRESS_SANITIZER

The following behaviors are intended:

OSQUERY_BUILD_TESTS=ON will build the fuzzing harness. This configuration is not
intended for fuzzing purposes. OSQUERY_ENABLE_FUZZER_SANITIZERS=ON will also build
the fuzzing harness. However if this variable is true, it also requires
OSQUERY_ENABLE_ADDRESS_SANITIZER=ON and either CMAKE_BUILD_TYPE=Release or
RelWithDebInfo This configuration is actually intended for fuzzing.
2019-10-20 14:09:45 -04:00
Teddy Reed
5ff42eccb6
packaging: Fix up make_linux_package to work for version 4+ (#5825) 2019-10-20 11:54:42 -04:00
Teddy Reed
15d522f447
json: Configure rapidjson to use iterative parsing (#5893) 2019-10-19 15:25:21 -04:00
Zachary Wasserman
0bf2245396 Migrate rapidjson to source build on macOS and Windows (#5899) 2019-10-18 20:05:22 -04:00
kumarak
9346926030 Restore extension SDK support to osquery 4.x (#5851)
CMake utility functions(generateOsqueryExtensionGroup,addOsqueryExtensioni{Ex},
addOsqueryModule) to add extensions.

Updates the boost library submodules needed for extensions build.

Use cache variable(OSQUERY_EXTENSION_GROUP_NAME) for the grouping of extensions
and fix the linkage with osquery interface library.
2019-10-18 18:45:20 -04:00
Stefano Bonicatti
3ad7f65a0d Support source and build directory to be on different filesystems (#5907)
Instead of moving a patched submodule from source to build directory,
we copy it and then hard reset the submodule to its original HEAD.

Minor message change when a library fails to be imported.
2019-10-18 18:25:05 -04:00
Zachary Wasserman
30a7507c95
Fix shallow clone support for libraries in recent git versions (#5897)
Enables the v2 protocol which allows the repositories to be shallow
cloned without error.
2019-10-17 21:19:24 -07:00
Will Newton
599e9d6f3a macOS keychain sanity test (#5885)
When requesting kSecClassIdentity items from the keychain sometimes
invalid items are returned. These cause errSecInvalidItemRef to
be returned from SecKeychainItemCopyAttributesAndData and result
in an empty row in the table. Catch the error and avoid returning
empty rows.
2019-10-17 08:50:17 -04:00
Nick Anderson
3957d8e948
version: fix windows os codename lookup (#5887) 2019-10-16 21:42:06 -07:00
Allan Liu
e1e75aac88 genPCIDevices for linux refactor (#5446) 2019-10-16 16:27:29 -04:00
Zachary Wasserman
c0876408fc Migrate AWS SDK CPP dependency to source build on macOS and Windows (#5889) 2019-10-16 16:20:22 -04:00
William Woodruff
1ca47d3e7a (Windows) New table: default_environment (#5441) 2019-10-16 14:20:52 -04:00
Teddy Reed
2b9ea20563
fuzz: Silence config parsing log lines in fuzz-config (#5886) 2019-10-16 10:19:18 -04:00
Zachary Wasserman
419a2c97e8 Use git 2.21.0 for macOS CI (#5892)
This fixes issues with shallow clones of submodules and will enable
#5889 to pass CI.
2019-10-16 10:18:46 -04:00
Alessandro Gario
6ba37014ef
BPF/Map: Add missing return value in move assignment operator (#5874) 2019-10-14 21:45:23 +02:00
Stefano Bonicatti
974bdef901 Various fixes and CI steps for packaging (#5881)
* Fix packaging regression on Windows

Component variables should be set only for Linux and macOS,
otherwise on Windows the internal WiX IDs change and the
WiX fragment patch won't work.

Cleanup the version patch component, since WiX doesn't support
version components that are not digits.

* Add the rpm package to the CI Dockerfile, to test rpm packaging

* Fix TGZ generation on other platforms

* Add dpkg-dev and file packages to the CI Dockerfile

They are needed for creating DEB debug packages.

* Add a packaging steps to the CI

Set TGZ as a default packaging system when one cannot detected.

Fix generatePackageTarget indentation.
2019-10-13 12:53:16 +02:00
widberg
6a20e79424 [Table sanity check] chocolatey_packages (#5871) 2019-10-11 17:26:44 -07:00
howar6hill
2a4acdbd53 [Table sanity check] sudoers (#5869) 2019-10-11 17:25:30 -07:00
Sharvil Shah
bdca64fbcd [shell] support previously supported on|off toggle for osqueryi shell functions (#5876) 2019-10-11 17:17:10 -07:00
Stefano Bonicatti
e1dca49663 Fix regression in calling core only plugins from extensions (#5875)
PR osquery/osquery#5464 makes it impossible to call plugins
which are only registered in the core, as its happening with osquery/osquery#5839.

What happens when this is working is that the filesystem plugin is not present
in the extension registry, so when Registry::call tries to search for a plugin
that provides the "config" feature, it won't find one and the resulting plugin name,
and later item_name, will be empty.

This results in the RegistryInterface::call reroute the call to the core,
through Thrift, which will end up in the plugin correctly answering.
2019-10-11 17:15:19 -07:00
Stefano Bonicatti
7d1d486c1e Fix git command not found after first configure (#5880)
During the first configuration, the git command is searched
when submodules are initialized and it's then saved into a cache variable.
If though we delete the build folder and run configure again,
the submodules won't be initialized again, so the git command
won't be cached and visible to CMake.
Therefore we need to search for it each time.

Cleanup dead/deprecated code.
2019-10-11 23:31:31 +02:00
Stefano Bonicatti
56c69e3cfa Add ccache support also for compiling C (#5872) 2019-10-08 22:39:28 +02:00
Alessandro Gario
225bbaf992
Integrate cppcheck and clang-tidy within the CMake project (#5730) 2019-10-08 18:17:11 +02:00
Teddy Reed
846c392903
cpack: Update package logic to build DEB debuginfo (#5843)
This enables a component install for DEB (only). This is required in
CMake version 3.17.0 (planned) and lower to produce a debuginfo version.

One side-effect is we have to gate the stripping for DEB.
2019-10-07 22:36:18 -07:00
Chris Broome
99bfa342ef Fix thirdparty_*_settings target_link_libraries (#5870)
- Reorder `add_library(osquery_thirdparty_extra_c_settings INTERFACE)`
  for consistency
- Use correct target link library in osquery_thirdparty_extra_*_settings
2019-10-07 21:18:17 +02:00
Stefano Bonicatti
fdd2272148 Provide CI Dockerfile and some fake packages (#5855)
Since normally the tests are run on a more complete VM or physical host,
we create fake empty packages to install so that
the test DebPackages.test_sanity doesn't fail.
2019-10-07 16:26:43 +02:00