Parsing a configuration file as a JSON document
which contains deeply nested elements can lead to a stack overflow
when using the recursive parser of RapidJSON.
Since the configuration isn't changed or parsed frequently,
use the slower iterative parser instead.
Copying the configuration JSON document
that contains deeply nested elements, using the CopyFrom API,
can lead to a stack overflow, due to the recursive nature
of the RapidJSON GenericValue construction.
Detect the depth/nesting level of a config document
and limit it to 32 levels.
Using an iterative parser, while it avoids stack overflows,
can cause memory exhaustion if the config size is too big.
Limit the maximum config size, stripped from its comments, to 1MiB.
Addresses https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20779
Removed the columns "script" and "match" from the test
since they do not belong to the chrome_extensions table,
they belong to chrome_extension_content_scripts.
Added the missing integration test for the table
chrome_extension_content_scripts.
Do not close the http server after 10s if there are requests coming,
since some tests may take more than 10s to run.
Reset the timer each time a request is received by the server instead.
This new toolchain contains a newer LLVM version (9.0.1),
a fix for the scan-build scripts and it keeps the LLVM static libraries,
necessary to implement the new BPF framework and tables.
The "decorators" configuration value must be a JSON object,
otherwise we try to search through its inexistent members
and dereference a null pointer.
Added also a regression test.
Addresses https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=19274
In the past the Windows agent changed the path where Python2
was installed; a special logic was added which should've tested
if the path existed, though it wasn't correct in the case
the powershell script is configured to be aborted at the first error.
Since the old path should not be present anymore,
we simply remove the logic and use the path we expect to exist.
With the increasing size of the build and the respective ccache
and sccache caches, the disk space sometimes is not enough
and the build fails.
This deletes the build folder as the last step since it shouldn't
be necessary anymore.
When the batch script that implements the build step has been
changed to stop the sccache server as the last command,
all build failures started to be ignored because the last command,
always succeeding, was clearing out the exit status.
Batch scripts do not have a global "exit on error" option,
so manually checking the error level and exiting with such error is needed.
* Limit regex_match and regex_split regex size
Add a new HIDDEN_FLAG, regex_max_size, with a default of 256 bytes,
which limits the size of the regex that can be used
with regex_match and regex_split SQL functions.
This is done since it's possible to create a regex
which makes the std::regex destruction go into a stack overflow,
due to too many alternate states (|).
Add a couple of tests to verify that the limit is correctly respected.
Restore the test for regex_split that was originally hanging when using
boost.
The template has a reference to waitThenShutdown
which does not exist in the definitions.
Instead of that, a function, waitForShutdown is defined in the
osquery/system.h which should have been used.
This causes compilation to fail in case of building an extension.
This commit fixes that function call in the template.
Created mdls table which mimics the functionality of the mdls command in macOS. Table allows osquery to retrieve key/value pairs from spotlight metadata.
The windows documentation mentions that ExitThread should not be
used in C++ code. When debugging these exit flows I saw exceptions
when running dtors.
When reviewing the shutdown code for Windows and POSIX there was a
lot of complexity. This change allows any part of code to request
a shutdown and simplifies the response.
We should expect the main thread to call ::waitForShutdown.
The exception is the shell, which waits in a repl.
Implements the Community ID hash that will allow correlating network connections detected by osquery with other tools that support the standard (Zeek, Suricata, etc.).
Updated performance-safety.md to include details about profile.py and it's output. The changes also include information about how profile.py sets the thresholds that appear in the script output.
* Add sccache to the Windows job
Also fix debug symbols on googletest,
so that it doesn't try to create a separate pdb.
* Fix uploading the submodule cache
sccache needs its cache to be uploaded as normal files,
not with a tar, otherwise the cache when extracted
will contain extraneous files and folders (PaxHeader).
The Files mode though by default ignores the .git folder,
so we have to use the .artifactignore file to unignore it,
and we have to copy in the cached path for the cache task to see it.
