The test modifies the source code, appending "\n\n" at the end of each .cpp files.
Then runs the format_check target and verifies if the list of files to be formatted,
identified by the script, are the same ones previously modified.
Summary: This adds a few minor updates to the MSI build function. First
it keeps build artifacts located in `build/msi` for ease of access.
Further it updates component referneces to be x64 compatible. Finally it
updates a few path references to reflect the new repo layout, as well as
changes a few references for author and ownership information to be
osquery.
Test Plan: Will update with sample MSI builds for Cmake.
* deployment: Add Mozilla CA certificate store 2019-08-28
This is the cacerts file that was once proviced by the OpenSSL Homebrew
formula. The osquery packages in 3.x and previous provided this store as
the default / fallback.
The hash can be corroborated via:
https://curl.haxx.se/ca/cacert.pem.sha256
38b6230aa4bee062cd34ee0ff6da173250899642b1937fc130896290b6bd91e3
* install certs.pem directly from source
Add a way to compile third-party libraries from source instead of downloading prebuilt ones.
Each library source code is downloaded with git into a submodule at configure time,
in response to the find_package(library_name) CMake call,
except for OpenSSL where the official source archive is used.
Each submodule is attached to a release tag on its own upstream repository.
All the libraries are built using CMake directly, except for OpenSSL which uses a formula system,
which permits to build libraries with a separate build system
when there's no easy way to integrate it directly with CMake.
This new dependency system determines which library is fetched from where using the concept of "layers".
Currently we have three of them: source, formula, facebook,
where the last layer represents the pre-built libraries.
The provided order will be used when looking for libraries.
A system to patch submodule source code has been added and it's currently used with googletest, libudev and util-linux.
Patches should be put under libraries/cmake/source/<library name>/patches/<submodule>,
where <submodule> is often one and is "src", but in other cases, like AWS,
there are multiple with a more specific name.
If for whatever reason the submodule cloning or the patching fails,
the submodule has to be unregistered and its folder should be cleared.
This should be achievable with "git submodule deinit -f <submodule path>"
Following some other changes on existing functionality:
- Changed the CMake variable BUILD_TESTING to OSQUERY_BUILD_TESTS
to avoid enabling tests on third party libraries.
Due to an issue with glog the BUILD_TESTING variable
will be always forced to OFF.
- Moved compiler and linker flags to their own file cmake/flags.cmake
- Moved all the third-party CMakeLists.txt used for pre-built libraries under libraries/cmake/facebook
- Added the --exclude-folders option to tools/format-check.py and tools/git-clang-format.py,
so that it's possible to ignore any third party library source code.
- The format and format_check target use the new --exclude-folders option
to exclude libraries/cmake/source from formatting.
- The test and osquery binaries are properly compiled with PIE (osquery/osquery#5611)
Co-authored-by: Stefano Bonicatti <stefano.bonicatti@gmail.com>
Co-authored-by: Teddy Reed <teddy@casualhacking.io>
The Wix configuration sets the service as _critical_. As I understand it, this means failures to start osquery, are considered startup failures, and will trigger a reboot. As there are occasional windows bugs causing a failure to start, this seems unfortunate. I think hit someone on slack today
This PR changes that to _normal_
Docs are http://wixtoolset.org/documentation/manual/v3/xsd/wix/serviceinstall.html
Taken from osql-experimental.
- Change CMake code license to the one present in osquery right now
- Package metadata doesn't mention Trail of Bits or osql anymore
- Set specific ACLs for the osqueryd on Windows when packaging
- Remove LLVM_INSTALL_PATH support on macOS, since we are using AppleClang
- Remove OSQUERY_SOURCE_DIR variable need and source in a submodule support
- Add targets format_check and format to check code formatting and
format it with clang-format
- Do not warn about not using Clang on macOS when using AppleClang
Summary:
Iterating through a string is no longer allowed, and `range(0, len(_))` and `range(len(_))`
are equivalent. Switch to the shorter, more commonly used form
Reviewrs: #sentinel
Reviewed By: philipjameson
Differential Revision: D14657008
fbshipit-source-id: 1aabcbf168896bd0ee64b0d4eb17a72d6863aab2
Summary: Right now it blocks us, because build on macox doesn't work. Fix will take some time - lots of changes. But on the other hand value of such change in tests is nearly zero. So, let's just mute it.
Reviewed By: guliashvili
Differential Revision: D14597262
fbshipit-source-id: adaacc003f49647e255001bb84cc0e71273cd486
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5528
by using config option `cxx.filepath_length_limited=true`. Because unfortunately there is very low limit for file path length on windows up to win10 (260 chars).
Reviewed By: KapJI
Differential Revision: D14460635
fbshipit-source-id: f63fc564766b49c2d4fb5f1c2bb7015592ab17e1
Summary: Having two configuration files makes it harder to manage system.py so move this to the same file and to the toolchain generation script. This will allow us to automatically determine toolchain path as well in the future.
Reviewed By: marekcirkos
Differential Revision: D14425055
fbshipit-source-id: fdc017f2cc55a2efbb33cdf17df64df620eb11b8
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5490
We use functionality of this libraries, how did it work before?
Reviewed By: guliashvili
Differential Revision: D14280974
fbshipit-source-id: c3b0c2d8d570680460cdc5bbe80efc24467bcb93
Summary: You can now build with `buck build @<mode> osqueryd` for both internal and external build. Also changed NBTD to make use of this.
Reviewed By: marekcirkos
Differential Revision: D14279886
fbshipit-source-id: 1b61bdf254b3d980388e2f23384101c91bf51b20
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5478
This makes it easier to update the osquery version and simplifies cxx.bzl by removing osquery specific preprocessor flags.
This will also make rebuilding osquery after changing versions faster, since the flags are now only defined for the headers which need them.
Reviewed By: akindyakov
Differential Revision: D14183142
fbshipit-source-id: 396d550f5b35a1d294fee802d2364cd9f7ab1d7a
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5469
This way we can specify extra arguments that are going to be added to the library, like exported_preprocessor_flags which is required by some libraries.
Reviewed By: marekcirkos, akindyakov
Differential Revision: D14220787
fbshipit-source-id: 652954e297e49147dfc9f77db8181e2c0e9e123f
Summary:
We need to be able to build `fbcode` projects with dependencies to `fbsource/xplat/osquery/oss/sdk:plugin_sdk`. As far as osquery is a part of `fbsource` now it would be very useful to build against it, make a tests. Which will helps us a lot to develop faster, will unblock us to run tests for every change either to `xplat/osquery` and to `fbcode`, which going to prevent code from bugs and interface breaking (which happens now too often).
`osquery` is very platform dependent project, because it built internally at least for 4 OS: `linux`, `freebsd`, `windows`, `darwin`. `osquery` has its own third-party libraries located in `fbsource/xplat/osquery/third-party`.
Also we have internal osquery extension (`fb_osquery`) in `fbcode` built with strong dependency to `osquery` and with lots of dependencies to `fbcode` projects (e.g. scribe, GK, ODS, configurator, serivicerouter and more).
We could not build `fb_osquery` directly against `osquery` because build system restrictions and third-party dependencies collision.
- Add necessary for `fb_osquery` parts of `fbsource/xpat/osquery` to xplat whitelist.
- Make it possible to use `fbcode` `cpp_library` target definition for `fbsource/xplat/osquery` targets when they are used for `fbcode` build.
- Make a translation platform dependant osquery targets for fbcode platform independent build.
- Use `fbcode/tp` libs instead of `fbsource/xplat/osquery/tp` in case of `fbcode` build.
Differential Revision: D13991062
fbshipit-source-id: 1294825f1c5f991bd465e0e299b8e5ff67bbc543
Summary: This was patched with D13767582 and is already deployed
Reviewed By: mkareta
Differential Revision: D14124516
fbshipit-source-id: 30679472458f4ed9647adc117db4352b940cf1cf
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5452
As suggested in another diff, this diff updates the language we use to describe the osquery licensing terms. We are changing all instances of
//This source code is licensed as defined on the LICENSE file found in the root directory of this source tree.//
to
//This source code is licensed in accordance with the terms specified in the LICENSE file found in the root directory of this source tree.//
We accomplish this with a codemod:
$ codemod -md xplat/osquery/oss --extensions cpp,h,in,py,sh,mm,ps1 "(.\s+)This source code is licensed as defined on the LICENSE file found in the(.*)root directory of this source tree\." "\1This source code is licensed in accordance with the terms specified in\2the LICENSE file found in the root directory of this source tree."
Reviewed By: fmanco
Differential Revision: D14131290
fbshipit-source-id: 52c90da342263e2a80f5a678ecd760c19cf7513e
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5451
This diff adds a Facebook copyright header to the bzl files used in osquery. Ultimately we want to update the files in `tools/build_defs/oss/osquery/`, but those are generated files. This diff updates the source files which we use to generate those files.
Reviewed By: fmanco
Differential Revision: D14131483
fbshipit-source-id: 2230dc382c26530ccd0909882fe6193ee7c674fb
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5445
This diff adds a Facebook copyright header to files in the osquery open source repository which:
* Facebook owns
* Do not currently have a Facebook copyright header
Reviewed By: marekcirkos
Differential Revision: D14122845
fbshipit-source-id: 5a0fea10189ec4ec893f7a036911fd51de0e01ae
Summary: Removing flag which was declared but never used. enable_monitor
Reviewed By: marekcirkos
Differential Revision: D13958265
fbshipit-source-id: 3a812330950b101abdbd83ada4afd5b262cabd26
Summary:
This diff adds Xcode support for osquery.
Part of this diff will be reverted in future after adding prebuilt library and platform deps support to buck.
To use it you need to build osquery in debug mode and then run buck with following flags:
--config osquery.xcode=true --config project.ide=xcode
Reviewed By: SAlexandru
Differential Revision: D13903315
fbshipit-source-id: 4d131964d7a61236f25d917dc060a2f3c3d782bc
Summary: before this diff we were using -O flag, which equals to -O2, and our debug builds were optimized, which make debug much harder
Reviewed By: fmanco
Differential Revision: D13956134
fbshipit-source-id: b358d8fd68c8f5d51ae6d4c2033e7ec3afdd50d2
Summary:
Not every environment requires all tables, this diff introduce flag that allows you mark table as foreign. New option should be used in conjunction with target filer.
Example:
> buck build ... --config osquery.target_ignore_list="smart" --config osquery.spec_ignore_list="smart/smart_drive_info.table" -- -S
Reviewed By: fmanco
Differential Revision: D13942107
fbshipit-source-id: fb34d6b7a296f69f6b95bf17bfd19cee31b34dec
Summary:
Not every environment require all osquery feature, with this diff you can specify targets that you want to ignore, together with all sub tree of deps. To use this you need to specify new osquery config like:
[osquery]
target_ignore_list="kafka_producer"
Or from command line:
--config osquery.target_ignore_list="kafka_producer"
This also includes killswitch that force buck to build all targets. This is needed when you have local buckcofig with ignore list and want to build all without modifying config.
--config osquery.force_build_all=true
Reviewed By: fmanco
Differential Revision: D13941689
fbshipit-source-id: 3c4e1b4cda4d74f33fb914ba2c3a17df4710d5d3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5405
this should find where VS is installed and set the buck flags properly.
Have tested on my VM and the paths are ok. This only works for 2017 and newer (hopefully)
I'm not sure how future proof this is, Microsoft usually changes directory structures randomly.
Reviewed By: muffins
Differential Revision: D13762391
fbshipit-source-id: 894e6a6d5888e13ab646ca9cb4a0d604bcf53ee5
