When a query triggers multiple xFilter calls
and there's an operation that has to work on the sum of rows
resulting from all those calls, we trigger a use-after-free
when such operation tries to access the rows data.
This happens because each xFilter call we clear the rows
resulting from the previous xFilter call, and because
when returning the values of a text column we don't copy it,
but return a pointer to it.
A contrived example of a query with the issue is:
SELECT path=count(*) FROM file WHERE path = '/' OR path = '1'
This changes the last sqlite3_result_text parameter
from SQLITE_STATIC to SQLITE_TRANSIENT.
Addresses https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20833
* Limit regex_match and regex_split regex size
Add a new HIDDEN_FLAG, regex_max_size, with a default of 256 bytes,
which limits the size of the regex that can be used
with regex_match and regex_split SQL functions.
This is done since it's possible to create a regex
which makes the std::regex destruction go into a stack overflow,
due to too many alternate states (|).
Add a couple of tests to verify that the limit is correctly respected.
Restore the test for regex_split that was originally hanging when using
boost.
Implements the Community ID hash that will allow correlating network connections detected by osquery with other tools that support the standard (Zeek, Suricata, etc.).
sqlite3_result_error() third parameter is the length
of the string of the second parameter, not the error code.
We set that to -1, which means that the length of the string
will be taken using strlen().
Addresses https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18694
We do not need an explicit INDEX column for ADDITIONAL to work.
If the ADDITIONAL option is set the constraint should be passed
into the virtual table context.
Recent changes in INDEX enforcement broke extensions.
This fix adds a compatibility flag that defaults to the old behavior of INDEX on all columns.
This only affects extension virtual tables, all built in tables have index constraints enforced.
This is an API change for the format of logged events.
The top-level log field "logNumericsAsNumbers" was introduced to help
migrate from string-encoded numerics to JSON numbers.
This change updates the field to be "numerics", updates the flag to
conform to flag naming conventions, and documents the expectation.
There is a memory leak when a SQL statement is evaluated and returns an
error while stepping. An example includes returning sqlite3_result_error
from regexStringMatchFunc. The root cause is not finalizing a prepared
statement.
Add a way to compile third-party libraries from source instead of downloading prebuilt ones.
Each library source code is downloaded with git into a submodule at configure time,
in response to the find_package(library_name) CMake call,
except for OpenSSL where the official source archive is used.
Each submodule is attached to a release tag on its own upstream repository.
All the libraries are built using CMake directly, except for OpenSSL which uses a formula system,
which permits to build libraries with a separate build system
when there's no easy way to integrate it directly with CMake.
This new dependency system determines which library is fetched from where using the concept of "layers".
Currently we have three of them: source, formula, facebook,
where the last layer represents the pre-built libraries.
The provided order will be used when looking for libraries.
A system to patch submodule source code has been added and it's currently used with googletest, libudev and util-linux.
Patches should be put under libraries/cmake/source/<library name>/patches/<submodule>,
where <submodule> is often one and is "src", but in other cases, like AWS,
there are multiple with a more specific name.
If for whatever reason the submodule cloning or the patching fails,
the submodule has to be unregistered and its folder should be cleared.
This should be achievable with "git submodule deinit -f <submodule path>"
Following some other changes on existing functionality:
- Changed the CMake variable BUILD_TESTING to OSQUERY_BUILD_TESTS
to avoid enabling tests on third party libraries.
Due to an issue with glog the BUILD_TESTING variable
will be always forced to OFF.
- Moved compiler and linker flags to their own file cmake/flags.cmake
- Moved all the third-party CMakeLists.txt used for pre-built libraries under libraries/cmake/facebook
- Added the --exclude-folders option to tools/format-check.py and tools/git-clang-format.py,
so that it's possible to ignore any third party library source code.
- The format and format_check target use the new --exclude-folders option
to exclude libraries/cmake/source from formatting.
- The test and osquery binaries are properly compiled with PIE (osquery/osquery#5611)
Co-authored-by: Stefano Bonicatti <stefano.bonicatti@gmail.com>
Co-authored-by: Teddy Reed <teddy@casualhacking.io>
Taken from osql-experimental.
- Change CMake code license to the one present in osquery right now
- Package metadata doesn't mention Trail of Bits or osql anymore
- Set specific ACLs for the osqueryd on Windows when packaging
- Remove LLVM_INSTALL_PATH support on macOS, since we are using AppleClang
- Remove OSQUERY_SOURCE_DIR variable need and source in a submodule support
- Add targets format_check and format to check code formatting and
format it with clang-format
- Do not warn about not using Clang on macOS when using AppleClang
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5542
We replace deprecated calls to the Status class with newer ones. This will stop Lint from nagging me every time I open these files. There should be no change to functionality.
Specifically, we use a codemod on `.cpp` files to replace any instance of `return Status();` with `return Status::success();`.
Reviewed By: guliashvili
Differential Revision: D14652922
fbshipit-source-id: 243576195ed201d6b82fc942a785742c6b01cf83
Summary:
Add log_numerics_as_numbers flag.
Internal SQLite query has method that returns QueryDataTyped (instead of QueryData), which are boost::variant<string, double, long long>.
Ints are encoded as such if and only if new log_numerics_as_numbers flag is set to true.
Reviewed By: fmanco
Differential Revision: D13778323
fbshipit-source-id: 7d7bb31781486f63fcc088cd479d3b6f255a5cb4
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5483
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move killswitch
Reviewed By: guliashvili
Differential Revision: D14259760
fbshipit-source-id: deaaa148fac25b3f534a8881a1ea9eda0ec6bdc9
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5487
I got tired of Lint telling me to update the calls to the Status class, so I wrote some codemods to update the vast majority of them:
$ cd ~/fbsource
$ codemod -d xplat/osquery/oss/osquery --extensions cpp,h "return Status\(0\, \".*\"\);" "return Status::success();"
**Blindly accepted all changes**.
FWIW, I tried to do something similar with the failure return values, but the unit tests were failing afterward.
Reviewed By: guliashvili
Differential Revision: D14278739
fbshipit-source-id: b023f6e969be697eeb67d66ea3ad10d16c69ff0c
Summary: To be able to build with different `thrift_impl` modules in extensions (like facebook does with fbthrift) I removed implementation from `extensions` lib and explicitly used for `osqueryd` and for all tests. This allows us to remove `thrift_impl` from sdk dependencies list. If you need it - please use explicitly as I did for `osqueryd`. If you want to use your own implementation of `thrift_impl` - build it and link against it and sdk simultaneously.
Reviewed By: marekcirkos
Differential Revision: D14224206
fbshipit-source-id: 218dc05d87e1ffaca37783185fc672a20684d757
Summary:
This fixes a regression introduced in D13586036 .
The tailer downstream sometimes determines type by trying to parse the string as an int. We want this to consistently fail to do so when the value is in fact a float.
Reviewed By: SAlexandru
Differential Revision: D14161125
fbshipit-source-id: 8e2bb9c59559a5774813a7c7777257742f885c35
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5452
As suggested in another diff, this diff updates the language we use to describe the osquery licensing terms. We are changing all instances of
//This source code is licensed as defined on the LICENSE file found in the root directory of this source tree.//
to
//This source code is licensed in accordance with the terms specified in the LICENSE file found in the root directory of this source tree.//
We accomplish this with a codemod:
$ codemod -md xplat/osquery/oss --extensions cpp,h,in,py,sh,mm,ps1 "(.\s+)This source code is licensed as defined on the LICENSE file found in the(.*)root directory of this source tree\." "\1This source code is licensed in accordance with the terms specified in\2the LICENSE file found in the root directory of this source tree."
Reviewed By: fmanco
Differential Revision: D14131290
fbshipit-source-id: 52c90da342263e2a80f5a678ecd760c19cf7513e
Summary:
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move config and config_parser plugins
Reviewed By: marekcirkos
Differential Revision: D14119102
fbshipit-source-id: 0bc956398b3829c6f1013b38ebba2f0fc1071a93
