When testing on IPv6-only hosts, with an IPv4 lo stack, the resolver may
not return an IPv4 address. This seems to be an issue with the
deprecated overload of async_resolve. Upon inspection, the previous
overload would only return 1 result_type.
Migrating to the recommended overload works as expected.
This was determined to be the wrong approach to adding simple
killswitches. The intent was to quickly flip on/off features. It was
not widely adopted due to the dependencies killswitching has.
A different approach is to implement the same functions with something
simple like filesystem flags.
Add a way to compile third-party libraries from source instead of downloading prebuilt ones.
Each library source code is downloaded with git into a submodule at configure time,
in response to the find_package(library_name) CMake call,
except for OpenSSL where the official source archive is used.
Each submodule is attached to a release tag on its own upstream repository.
All the libraries are built using CMake directly, except for OpenSSL which uses a formula system,
which permits to build libraries with a separate build system
when there's no easy way to integrate it directly with CMake.
This new dependency system determines which library is fetched from where using the concept of "layers".
Currently we have three of them: source, formula, facebook,
where the last layer represents the pre-built libraries.
The provided order will be used when looking for libraries.
A system to patch submodule source code has been added and it's currently used with googletest, libudev and util-linux.
Patches should be put under libraries/cmake/source/<library name>/patches/<submodule>,
where <submodule> is often one and is "src", but in other cases, like AWS,
there are multiple with a more specific name.
If for whatever reason the submodule cloning or the patching fails,
the submodule has to be unregistered and its folder should be cleared.
This should be achievable with "git submodule deinit -f <submodule path>"
Following some other changes on existing functionality:
- Changed the CMake variable BUILD_TESTING to OSQUERY_BUILD_TESTS
to avoid enabling tests on third party libraries.
Due to an issue with glog the BUILD_TESTING variable
will be always forced to OFF.
- Moved compiler and linker flags to their own file cmake/flags.cmake
- Moved all the third-party CMakeLists.txt used for pre-built libraries under libraries/cmake/facebook
- Added the --exclude-folders option to tools/format-check.py and tools/git-clang-format.py,
so that it's possible to ignore any third party library source code.
- The format and format_check target use the new --exclude-folders option
to exclude libraries/cmake/source from formatting.
- The test and osquery binaries are properly compiled with PIE (osquery/osquery#5611)
Co-authored-by: Stefano Bonicatti <stefano.bonicatti@gmail.com>
Co-authored-by: Teddy Reed <teddy@casualhacking.io>
Only define BOOST_ASIO_DISABLE_STD_STRING_VIEW.
We shouldn't define BOOST_ASIO_HAS_STD_STRING_VIEW,
because even if we define BOOST_ASIO_DISABLE_STD_STRING_VIEW
the first define will actually enable parts of code that will use string_view.
This won't work on Windows and in general, string_view should not be
used unless compiling with C++17.
The hack has been also added to a test that was previously missed.
Taken from osql-experimental.
- Change CMake code license to the one present in osquery right now
- Package metadata doesn't mention Trail of Bits or osql anymore
- Set specific ACLs for the osqueryd on Windows when packaging
- Remove LLVM_INSTALL_PATH support on macOS, since we are using AppleClang
- Remove OSQUERY_SOURCE_DIR variable need and source in a submodule support
- Add targets format_check and format to check code formatting and
format it with clang-format
- Do not warn about not using Clang on macOS when using AppleClang
Summary:
Add log_numerics_as_numbers flag.
Internal SQLite query has method that returns QueryDataTyped (instead of QueryData), which are boost::variant<string, double, long long>.
Ints are encoded as such if and only if new log_numerics_as_numbers flag is set to true.
Reviewed By: fmanco
Differential Revision: D13778323
fbshipit-source-id: 7d7bb31781486f63fcc088cd479d3b6f255a5cb4
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5483
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move killswitch
Reviewed By: guliashvili
Differential Revision: D14259760
fbshipit-source-id: deaaa148fac25b3f534a8881a1ea9eda0ec6bdc9
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5487
I got tired of Lint telling me to update the calls to the Status class, so I wrote some codemods to update the vast majority of them:
$ cd ~/fbsource
$ codemod -d xplat/osquery/oss/osquery --extensions cpp,h "return Status\(0\, \".*\"\);" "return Status::success();"
**Blindly accepted all changes**.
FWIW, I tried to do something similar with the failure return values, but the unit tests were failing afterward.
Reviewed By: guliashvili
Differential Revision: D14278739
fbshipit-source-id: b023f6e969be697eeb67d66ea3ad10d16c69ff0c
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5482
We allow people using a debug build of osqueryd to skip verification of the server's TLS certificate. This allow people to connect to a devserver, for example, when testing the Graph API endpoints used by osquery.
This mode broke at some point when we stopped defining the `DEBUG` constant when building osqueryd in debug mode. This diff updates the code so that we use the constant `NDEBUG`, which is actually defined for release builds. For example, for OS X, see the build configs at https://fburl.com/ywr1tyuk and https://fburl.com/mte9ajvr.
We also update the code to ensure the command line option `--tls_allow_unsafe` allows a person to connect to a dev server.
Reviewed By: obelisk
Differential Revision: D14260226
fbshipit-source-id: c0f2f1c1f4c8dd5e3e1eaa6e1a52ab2a4ff765d4
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5473
to be able to exclude dependency on enroll:tls plugin from plugins_sdk.
Only plugin registration was moved by now. The actual plugin code will be moved a bit later.
Reviewed By: guliashvili
Differential Revision: D14241687
fbshipit-source-id: 814d52719ff27bd1283bcaecdc912fca3830ff5e
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5472
To get rid of dependency on `plugins/config:tls_config` from `osquery/remote/enroll/tls_enroll`.
To be able to remove dependency on `plugins/config:tls_config` from plugins_sdk.
Reviewed By: guliashvili
Differential Revision: D14241685
fbshipit-source-id: 27cda0be9d0b6dc142374cc3a9e7c40a4fa97b94
Summary: To be able to build with different `thrift_impl` modules in extensions (like facebook does with fbthrift) I removed implementation from `extensions` lib and explicitly used for `osqueryd` and for all tests. This allows us to remove `thrift_impl` from sdk dependencies list. If you need it - please use explicitly as I did for `osqueryd`. If you want to use your own implementation of `thrift_impl` - build it and link against it and sdk simultaneously.
Reviewed By: marekcirkos
Differential Revision: D14224206
fbshipit-source-id: 218dc05d87e1ffaca37783185fc672a20684d757
Summary: This test is flaky on some systems. Need to investigate and fix.
Reviewed By: guliashvili
Differential Revision: D14149533
fbshipit-source-id: 7df570c84c66cad8bdc67a1286d91a7c7e8e3502
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5458
Use ASSERTs where necessary. ASSERTs are fatal and terminate the test immediately, so should be used when subsequent tests aren't meaningful in case of previous failures. Only use EXPECTED when it makes sense for the tests to continue.
Reviewed By: jessek
Differential Revision: D14138382
fbshipit-source-id: aae0cd8640e602ac467861303457f9cf017bc5e8
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5452
As suggested in another diff, this diff updates the language we use to describe the osquery licensing terms. We are changing all instances of
//This source code is licensed as defined on the LICENSE file found in the root directory of this source tree.//
to
//This source code is licensed in accordance with the terms specified in the LICENSE file found in the root directory of this source tree.//
We accomplish this with a codemod:
$ codemod -md xplat/osquery/oss --extensions cpp,h,in,py,sh,mm,ps1 "(.\s+)This source code is licensed as defined on the LICENSE file found in the(.*)root directory of this source tree\." "\1This source code is licensed in accordance with the terms specified in\2the LICENSE file found in the root directory of this source tree."
Reviewed By: fmanco
Differential Revision: D14131290
fbshipit-source-id: 52c90da342263e2a80f5a678ecd760c19cf7513e
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5449
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move killswitch
Move test declaration to it's own BUCK file for consistency with the rest of the project.
Reviewed By: marekcirkos
Differential Revision: D14121618
fbshipit-source-id: 3e30e57befed4387585ed553ec087fdf8db6efc3
Summary:
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move config and config_parser plugins
Reviewed By: marekcirkos
Differential Revision: D14119102
fbshipit-source-id: 0bc956398b3829c6f1013b38ebba2f0fc1071a93
Summary:
This addresses a slight regression to ensure that we set `set_terminate_threads` on Windows. Without this flag being set, Windows threads will deadlock on exit as the boost managed io service threads never receive termination notifications.
I'm opening this PR up against the old master as I feel we should likely cut a 3.3.3, and I'm happy to re-open this PR against the upstream experimental as well, but we'll want a fix for this released as quickly as possible to Windows deployments.
Pull Request resolved: https://github.com/facebook/osquery/pull/5421
Reviewed By: marekcirkos
Differential Revision: D13972916
Pulled By: muffins
fbshipit-source-id: 55e3b23c80091d5fb51a97d1efc043b52dc48ba3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5375
LICENSE is now defined in a single file on the root of the project, update the
header to contain that information.
**Project LICENSE did not change.**
Reviewed By: akindyakov
Differential Revision: D13750575
fbshipit-source-id: 1e608a81b260b8395f9d008fc67f463160c1fc2b
Summary:
While running `misspell` on a different codebase. I happened to notice that some misspellings in the osquery code base. So, I fixed them
Pull Request resolved: https://github.com/facebook/osquery/pull/5256
Reviewed By: guliashvili
Differential Revision: D13670897
Pulled By: fmanco
fbshipit-source-id: 5d33d858284955c376e8c3980acdf366d4edf3d3
Summary: Some tools does not support virtual headers namespaces and expect header path to match actual file path from project root. This diff will fix few namespaces in utils library
Reviewed By: guliashvili
Differential Revision: D13552878
fbshipit-source-id: 2a06f73550c69777bf73be73abdde297fe580583
Summary: Switch off osquery/remote tests for windows, it takes time to fix it. I'll do it later.
Reviewed By: fmanco
Differential Revision: D13378357
fbshipit-source-id: 21077630864fc4a27ba65078c634e115875f3970
