The "decorators" configuration value must be a JSON object,
otherwise we try to search through its inexistent members
and dereference a null pointer.
Added also a regression test.
Addresses https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=19274
This is to have better separation between them and the python tests.
BUCK files have been updated accordingly.
Changes to the internal generateCopyFileTarget function were needed.
The function now supports a base folder to be set so that the regex or file path
is appended to that base path, without having it included in the destination.
It will also not use a library target anymore, but a custom one so
that's possible to set properties with custom names.
Use only one copy target to copy test config files instead of having
multiple that overlaps.
It may give issues when copying and tests in the end will need all of
them.
While in the end the osqueryd binary will require almost all libraries
built and linked, having each intermediate library depend on the whole
set of AWS libraries might result in higher compiling time.
Especially since on some platforms not all libraries are used.
This should speed up the compilation speed on Windows.
This is an API change for the format of logged events.
The top-level log field "logNumericsAsNumbers" was introduced to help
migrate from string-encoded numerics to JSON numbers.
This change updates the field to be "numerics", updates the flag to
conform to flag naming conventions, and documents the expectation.
This was determined to be the wrong approach to adding simple
killswitches. The intent was to quickly flip on/off features. It was
not widely adopted due to the dependencies killswitching has.
A different approach is to implement the same functions with something
simple like filesystem flags.
Multiple flags are moved from FLAG to CLI_FLAG due to the way they are
implemented/used in code. If they were FLAG(s), meaning if they were
also configurable via configuration at runtime, the new values would
still be ignored.
These are:
- logger_tls_endpoint
- logger_tls_period
- logger_tls_max
The flag logger_tls_max has been renamed to logger_tls_max_linesize and
an alias is added for compatibility.
A new flag is added, logger_tls_max_lines, to expose a previously
hardcoded value of 1024 for the maximum number of log lines to send
every period.
Add a way to compile third-party libraries from source instead of downloading prebuilt ones.
Each library source code is downloaded with git into a submodule at configure time,
in response to the find_package(library_name) CMake call,
except for OpenSSL where the official source archive is used.
Each submodule is attached to a release tag on its own upstream repository.
All the libraries are built using CMake directly, except for OpenSSL which uses a formula system,
which permits to build libraries with a separate build system
when there's no easy way to integrate it directly with CMake.
This new dependency system determines which library is fetched from where using the concept of "layers".
Currently we have three of them: source, formula, facebook,
where the last layer represents the pre-built libraries.
The provided order will be used when looking for libraries.
A system to patch submodule source code has been added and it's currently used with googletest, libudev and util-linux.
Patches should be put under libraries/cmake/source/<library name>/patches/<submodule>,
where <submodule> is often one and is "src", but in other cases, like AWS,
there are multiple with a more specific name.
If for whatever reason the submodule cloning or the patching fails,
the submodule has to be unregistered and its folder should be cleared.
This should be achievable with "git submodule deinit -f <submodule path>"
Following some other changes on existing functionality:
- Changed the CMake variable BUILD_TESTING to OSQUERY_BUILD_TESTS
to avoid enabling tests on third party libraries.
Due to an issue with glog the BUILD_TESTING variable
will be always forced to OFF.
- Moved compiler and linker flags to their own file cmake/flags.cmake
- Moved all the third-party CMakeLists.txt used for pre-built libraries under libraries/cmake/facebook
- Added the --exclude-folders option to tools/format-check.py and tools/git-clang-format.py,
so that it's possible to ignore any third party library source code.
- The format and format_check target use the new --exclude-folders option
to exclude libraries/cmake/source from formatting.
- The test and osquery binaries are properly compiled with PIE (osquery/osquery#5611)
Co-authored-by: Stefano Bonicatti <stefano.bonicatti@gmail.com>
Co-authored-by: Teddy Reed <teddy@casualhacking.io>
Taken from osql-experimental.
- Change CMake code license to the one present in osquery right now
- Package metadata doesn't mention Trail of Bits or osql anymore
- Set specific ACLs for the osqueryd on Windows when packaging
- Remove LLVM_INSTALL_PATH support on macOS, since we are using AppleClang
- Remove OSQUERY_SOURCE_DIR variable need and source in a submodule support
- Add targets format_check and format to check code formatting and
format it with clang-format
- Do not warn about not using Clang on macOS when using AppleClang
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5524
So let's get rid of the rest usecases of createError with message as an argument in order to remove it completely.
Reviewed By: jessek
Differential Revision: D14437933
fbshipit-source-id: acfb1a2a2a79e03d7cb650b305ee8ad3b6218d2a
Summary:
Add log_numerics_as_numbers flag.
Internal SQLite query has method that returns QueryDataTyped (instead of QueryData), which are boost::variant<string, double, long long>.
Ints are encoded as such if and only if new log_numerics_as_numbers flag is set to true.
Reviewed By: fmanco
Differential Revision: D13778323
fbshipit-source-id: 7d7bb31781486f63fcc088cd479d3b6f255a5cb4
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5485
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually ne
er deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move numeric monitoring
Reviewed By: guliashvili
Differential Revision: D14259758
fbshipit-source-id: 1016fc28a0052417d658b6ce1cb3368e56597a7b
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5483
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move killswitch
Reviewed By: guliashvili
Differential Revision: D14259760
fbshipit-source-id: deaaa148fac25b3f534a8881a1ea9eda0ec6bdc9
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5474
To be able to exclude config:update plugin from osquery plugin sdk.
Reviewed By: guliashvili
Differential Revision: D14241686
fbshipit-source-id: e378718b1e79e3dbe314a676945c18d15d054a3d
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5473
to be able to exclude dependency on enroll:tls plugin from plugins_sdk.
Only plugin registration was moved by now. The actual plugin code will be moved a bit later.
Reviewed By: guliashvili
Differential Revision: D14241687
fbshipit-source-id: 814d52719ff27bd1283bcaecdc912fca3830ff5e
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5472
To get rid of dependency on `plugins/config:tls_config` from `osquery/remote/enroll/tls_enroll`.
To be able to remove dependency on `plugins/config:tls_config` from plugins_sdk.
Reviewed By: guliashvili
Differential Revision: D14241685
fbshipit-source-id: 27cda0be9d0b6dc142374cc3a9e7c40a4fa97b94
Summary: To be able to build with different `thrift_impl` modules in extensions (like facebook does with fbthrift) I removed implementation from `extensions` lib and explicitly used for `osqueryd` and for all tests. This allows us to remove `thrift_impl` from sdk dependencies list. If you need it - please use explicitly as I did for `osqueryd`. If you want to use your own implementation of `thrift_impl` - build it and link against it and sdk simultaneously.
Reviewed By: marekcirkos
Differential Revision: D14224206
fbshipit-source-id: 218dc05d87e1ffaca37783185fc672a20684d757
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5452
As suggested in another diff, this diff updates the language we use to describe the osquery licensing terms. We are changing all instances of
//This source code is licensed as defined on the LICENSE file found in the root directory of this source tree.//
to
//This source code is licensed in accordance with the terms specified in the LICENSE file found in the root directory of this source tree.//
We accomplish this with a codemod:
$ codemod -md xplat/osquery/oss --extensions cpp,h,in,py,sh,mm,ps1 "(.\s+)This source code is licensed as defined on the LICENSE file found in the(.*)root directory of this source tree\." "\1This source code is licensed in accordance with the terms specified in\2the LICENSE file found in the root directory of this source tree."
Reviewed By: fmanco
Differential Revision: D14131290
fbshipit-source-id: 52c90da342263e2a80f5a678ecd760c19cf7513e
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5449
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move killswitch
Move test declaration to it's own BUCK file for consistency with the rest of the project.
Reviewed By: marekcirkos
Differential Revision: D14121618
fbshipit-source-id: 3e30e57befed4387585ed553ec087fdf8db6efc3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5448
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move logger
Reviewed By: marekcirkos
Differential Revision: D14121620
fbshipit-source-id: cef15e7cc354cbe597c6c6878ee63ff09b5fb06d
Summary:
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move distributed
Reviewed By: marekcirkos
Differential Revision: D14121619
fbshipit-source-id: 9ad8a837450874e79a819ab4f11258ae24ec8014
Summary:
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move config and config_parser plugins
Reviewed By: marekcirkos
Differential Revision: D14119102
fbshipit-source-id: 0bc956398b3829c6f1013b38ebba2f0fc1071a93
