Commit Graph

1163 Commits

Author SHA1 Message Date
Alessandro Gario
4581ec42ce Remove deprecated build tools 2019-08-04 22:21:38 +02:00
Stefano Bonicatti
9cf33c84f8 Restore AugeasTests test
PR: osquery/osquery#5629
2019-07-19 22:36:40 +02:00
seph
54f967a528
Change MSI Service Error handling (#5467)
The Wix configuration sets the service as _critical_. As I understand it, this means failures to start osquery, are considered startup failures, and will trigger a reboot. As there are occasional windows bugs causing a failure to start, this seems unfortunate. I think hit someone on slack today 

This PR changes that to _normal_

Docs are http://wixtoolset.org/documentation/manual/v3/xsd/wix/serviceinstall.html
2019-07-19 09:51:41 -07:00
Nick Anderson
4eede7b00b
buck: fixing up buck config generation script for use on Windows (#5623) 2019-07-05 15:49:47 -07:00
Gustavo Grieco
b4bbcd274a Improve the downloader.py script to use urllib2 (#5612)
* Improve the downloader.py script to use urllib2

This will allow to automatically handle URL re-directions (e.g. 302)

* Better exception handling
2019-06-29 02:28:12 +02:00
Stefano Bonicatti
942878854b Add CMake support
Taken from osql-experimental.

- Change CMake code license to the one present in osquery right now

- Package metadata doesn't mention Trail of Bits or osql anymore

- Set specific ACLs for the osqueryd on Windows when packaging

- Remove LLVM_INSTALL_PATH support on macOS, since we are using AppleClang

- Remove OSQUERY_SOURCE_DIR variable need and source in a submodule support

- Add targets format_check and format to check code formatting and
  format it with clang-format

- Do not warn about not using Clang on macOS when using AppleClang
2019-06-26 21:49:06 -04:00
Alessandro Gario
33fbbecb5f Initial commit
Taken from osql-experimental.

Initial support for Linux and macOS.
2019-06-26 21:49:06 -04:00
Teddy Reed
597a0c613d buck: Remove quotes from project/buck_out config 2019-06-25 08:49:11 -04:00
Teddy Reed
c2451f3383
release: prepping for osquery 3.4.0 release on Windows
Please see #5590
2019-06-21 17:25:34 -04:00
Nick Anderson
214302bdeb
windows: updating more references to osquery installation path (#5580) 2019-06-05 12:45:40 -07:00
Nick Anderson
46b9c8f800
deploy: updating MSI package to install to Program Files (#5579) 2019-06-05 09:46:41 -07:00
Nick Anderson
c10914f199
deployment: updating osquery installation path to reside in Program Files (#5568) 2019-05-17 10:03:37 -07:00
Michael Lee
65c9c9b9ac Switch to the more common form
Summary:
Iterating through a string is no longer allowed, and `range(0, len(_))` and `range(len(_))`
are equivalent. Switch to the shorter, more commonly used form

Reviewrs: #sentinel

Reviewed By: philipjameson

Differential Revision: D14657008

fbshipit-source-id: 1aabcbf168896bd0ee64b0d4eb17a72d6863aab2
2019-03-28 19:39:30 -07:00
Alexander Kindyakov
76cc3eaf66 Mute signed-unsigned comparison warning for tests on macosx
Summary: Right now it blocks us, because build on macox doesn't work. Fix will take some time - lots of changes. But on the other hand value of such change in tests is nearly zero. So, let's just mute it.

Reviewed By: guliashvili

Differential Revision: D14597262

fbshipit-source-id: adaacc003f49647e255001bb84cc0e71273cd486
2019-03-25 09:28:27 -07:00
Alexander Kindyakov
7e8f5536e6 Force buck to produce short file paths on windows (#5528)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5528

by using config option `cxx.filepath_length_limited=true`. Because unfortunately there is very low limit for file path length on windows up to win10 (260 chars).

Reviewed By: KapJI

Differential Revision: D14460635

fbshipit-source-id: f63fc564766b49c2d4fb5f1c2bb7015592ab17e1
2019-03-14 10:55:41 -07:00
Filipe Manco
0166d04805 Move python configuration to the toolchain bcfg for Windows
Summary: Having two configuration files makes it harder to manage system.py so move this to the same file and to the toolchain generation script. This will allow us to automatically determine toolchain path as well in the future.

Reviewed By: marekcirkos

Differential Revision: D14425055

fbshipit-source-id: fdc017f2cc55a2efbb33cdf17df64df620eb11b8
2019-03-13 05:39:11 -07:00
Alexander Kindyakov
8a3c498f33 Add windows system OLE library to osquery windows buckconfig (#5490)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5490

We use functionality of this libraries, how did it work before?

Reviewed By: guliashvili

Differential Revision: D14280974

fbshipit-source-id: c3b0c2d8d570680460cdc5bbe80efc24467bcb93
2019-03-01 10:18:29 -08:00
Filipe Manco
aaa94770b0 Add osqueryd buck alias
Summary: You can now build with `buck build @<mode> osqueryd` for both internal and external build. Also changed NBTD to make use of this.

Reviewed By: marekcirkos

Differential Revision: D14279886

fbshipit-source-id: 1b61bdf254b3d980388e2f23384101c91bf51b20
2019-03-01 08:50:48 -08:00
Filipe Manco
bc35638678 Define osquery version on base.cfg (#5478)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5478

This makes it easier to update the osquery version and simplifies cxx.bzl by removing osquery specific preprocessor flags.

This will also make rebuilding osquery after changing versions faster, since the flags are now only defined for the headers which need them.

Reviewed By: akindyakov

Differential Revision: D14183142

fbshipit-source-id: 396d550f5b35a1d294fee802d2364cd9f7ab1d7a
2019-02-27 11:23:03 -08:00
Filipe Manco
53626785a2 Pass kwargs to osquery_tp_prebuilt_cxx_library (#5469)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5469

This way we can specify extra arguments that are going to be added to the library, like exported_preprocessor_flags which is required by some libraries.

Reviewed By: marekcirkos, akindyakov

Differential Revision: D14220787

fbshipit-source-id: 652954e297e49147dfc9f77db8181e2c0e9e123f
2019-02-26 07:59:14 -08:00
Alexander Kindyakov
c44067f788 Frank approach to build fbcode/fb_osquery with dependency to xplat/osquery
Summary:
We need to be able to build `fbcode` projects with dependencies to `fbsource/xplat/osquery/oss/sdk:plugin_sdk`. As far as osquery is a part of `fbsource` now it would be very useful to build against it, make a tests. Which will helps us a lot to develop faster, will unblock us to run tests for every change either to `xplat/osquery` and to `fbcode`, which going to prevent code from bugs and interface breaking (which happens now too often).

`osquery` is very platform dependent project, because it built internally at least for 4 OS: `linux`, `freebsd`, `windows`, `darwin`. `osquery` has its own third-party libraries located in `fbsource/xplat/osquery/third-party`.

Also we have internal osquery extension (`fb_osquery`) in `fbcode` built with strong dependency to `osquery` and with lots of dependencies to `fbcode` projects (e.g. scribe, GK, ODS, configurator, serivicerouter and more).

We could not build `fb_osquery` directly against `osquery` because build system restrictions and third-party dependencies collision.

  - Add necessary for `fb_osquery` parts of `fbsource/xpat/osquery` to xplat whitelist.
  - Make it possible to use `fbcode` `cpp_library` target definition for `fbsource/xplat/osquery` targets when they are used for `fbcode` build.
  - Make a translation platform dependant osquery targets for fbcode platform independent build.
  - Use `fbcode/tp` libs instead of `fbsource/xplat/osquery/tp` in case of `fbcode` build.

Differential Revision: D13991062

fbshipit-source-id: 1294825f1c5f991bd465e0e299b8e5ff67bbc543
2019-02-22 05:46:01 -08:00
Marek Cirkos
a1246af406 Revert part of D13903315
Summary: This was patched with D13767582 and is already deployed

Reviewed By: mkareta

Differential Revision: D14124516

fbshipit-source-id: 30679472458f4ed9647adc117db4352b940cf1cf
2019-02-20 01:59:23 -08:00
Jesse Kornblum
c7355b19aa Update osquery licensing wording (#5452)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5452

As suggested in another diff, this diff updates the language we use to describe the osquery licensing terms. We are changing all instances of

//This source code is licensed as defined on the LICENSE file found in the root directory of this source tree.//

to

//This source code is licensed in accordance with the terms specified in the LICENSE file found in the root directory of this source tree.//

We accomplish this with a codemod:

  $ codemod -md xplat/osquery/oss --extensions cpp,h,in,py,sh,mm,ps1 "(.\s+)This source code is licensed as defined on the LICENSE file found in the(.*)root directory of this source tree\." "\1This source code is licensed in accordance with the terms specified in\2the LICENSE file found in the root directory of this source tree."

Reviewed By: fmanco

Differential Revision: D14131290

fbshipit-source-id: 52c90da342263e2a80f5a678ecd760c19cf7513e
2019-02-19 10:59:48 -08:00
Jesse Kornblum
56d552d2a4 Add Facebook copyright header to bzl files (#5451)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5451

This diff adds a Facebook copyright header to the bzl files used in osquery. Ultimately we want to update the files in `tools/build_defs/oss/osquery/`, but those are generated files. This diff updates the source files which we use to generate those files.

Reviewed By: fmanco

Differential Revision: D14131483

fbshipit-source-id: 2230dc382c26530ccd0909882fe6193ee7c674fb
2019-02-19 09:29:31 -08:00
Marek Cirkos
02eeb2a323 Remove duplicated osquery_read_config definitions
Summary: Pull Request resolved: https://github.com/facebook/osquery/pull/5447

Reviewed By: akindyakov

Differential Revision: D14124515

fbshipit-source-id: ab5255c35a4a957ba45dc4e87851ee5c635a55d6
2019-02-19 02:28:20 -08:00
Jesse Kornblum
910e46b28f Add Facebook Copyright message to Facebook owned, open-source osquery files (#5445)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5445

This diff adds a Facebook copyright header to files in the osquery open source repository which:
* Facebook owns
* Do not currently have a Facebook copyright header

Reviewed By: marekcirkos

Differential Revision: D14122845

fbshipit-source-id: 5a0fea10189ec4ec893f7a036911fd51de0e01ae
2019-02-18 13:51:04 -08:00
Filipe Manco
464fa46788 Bump version to 3.3.2 (#5440)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5440

We're currently on 3.3.2.

Reviewed By: SAlexandru

Differential Revision: D14082297

fbshipit-source-id: 960e4f19ebd5cd0e8b9cdf31fbbd2e64331035d8
2019-02-14 08:22:48 -08:00
Alexander Kindyakov
6ee0fd18ea Do not mess with namaspace which comes from thirdparty library (rapidjson) (#5424)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5424

Let's use `SizeType` definition from rapidjson itself

Reviewed By: marekcirkos, fmanco

Differential Revision: D13986840

fbshipit-source-id: 53b3bf4dcb41c27e2cf18fee3aed97b9e2e42202
2019-02-11 02:33:19 -08:00
George Guliashvili
758706331f Remove unused flag enable_monitor
Summary: Removing flag which was declared but never used. enable_monitor

Reviewed By: marekcirkos

Differential Revision: D13958265

fbshipit-source-id: 3a812330950b101abdbd83ada4afd5b262cabd26
2019-02-07 08:16:00 -08:00
Max Kareta
a2a37fd6a4 added comments for xcode support
Reviewed By: marekcirkos

Differential Revision: D13957835

fbshipit-source-id: 2ffa94728aaaa3a8a9e5a40d36bf6e7c11393e2c
2019-02-05 08:48:47 -08:00
Max Kareta
40742aa582 temp solution for xcode support
Summary:
This diff adds Xcode support for osquery.
Part of this diff will be reverted in future after adding prebuilt library and platform deps support to buck.

To use it you need to build osquery in debug mode and then run buck with following flags:
  --config osquery.xcode=true --config project.ide=xcode

Reviewed By: SAlexandru

Differential Revision: D13903315

fbshipit-source-id: 4d131964d7a61236f25d917dc060a2f3c3d782bc
2019-02-05 07:20:00 -08:00
Max Kareta
13dbc5a887 Do not optimize debug builds
Summary: before this diff we were using -O flag, which equals to -O2, and our debug builds were optimized, which make debug much harder

Reviewed By: fmanco

Differential Revision: D13956134

fbshipit-source-id: b358d8fd68c8f5d51ae6d4c2033e7ec3afdd50d2
2019-02-05 04:35:29 -08:00
Max Kareta
e79d0ae5ce add feature that allow to mark tables foreign at build time
Summary:
Not every environment requires all tables, this diff introduce flag that allows you mark table as foreign. New option should be used in conjunction with target filer.

Example:
> buck build ... --config osquery.target_ignore_list="smart" --config osquery.spec_ignore_list="smart/smart_drive_info.table" -- -S

Reviewed By: fmanco

Differential Revision: D13942107

fbshipit-source-id: fb34d6b7a296f69f6b95bf17bfd19cee31b34dec
2019-02-05 04:03:50 -08:00
Max Kareta
0ee22f3172 added feature that allow ignore specific targets during build
Summary:
Not every environment require all osquery feature, with this diff you can specify targets that you want to ignore, together with all sub tree of deps. To use this you need to specify new osquery config like:
  [osquery]
  target_ignore_list="kafka_producer"

Or from command line:
   --config osquery.target_ignore_list="kafka_producer"

This also includes killswitch that force buck to build all targets. This is needed when you have local buckcofig with ignore list and want to build all without modifying config.
  --config osquery.force_build_all=true

Reviewed By: fmanco

Differential Revision: D13941689

fbshipit-source-id: 3c4e1b4cda4d74f33fb914ba2c3a17df4710d5d3
2019-02-05 04:03:50 -08:00
Alexandru Stefanica
a61f527554 automatically generate buck flags for windows (#5405)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5405

this should find where VS is installed and set the buck flags properly.
Have tested on my VM and the paths are ok.  This only works for 2017 and newer (hopefully)
I'm not sure how future proof this is, Microsoft usually changes directory structures randomly.

Reviewed By: muffins

Differential Revision: D13762391

fbshipit-source-id: 894e6a6d5888e13ab646ca9cb4a0d604bcf53ee5
2019-01-31 02:06:46 -08:00
Zachary Wasserman
04f632f16e Fix typo in provision script
Summary: Pull Request resolved: https://github.com/facebook/osquery/pull/5392

Reviewed By: mkareta

Differential Revision: D13838392

Pulled By: guliashvili

fbshipit-source-id: 0ad56503cd9bb0120e4ef669dea02e759131c823
2019-01-28 04:31:02 -08:00
George Guliashvili
08c032bbf5 Typed Row fix left shift overflow (#5385)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5385

Left shift with >= 31 steps was done to integer type. Using unisgned long long(1ULL) instead of the int (1).

Reviewed By: fmanco

Differential Revision: D13751355

fbshipit-source-id: 4564b33e2d26a0cb459ee86d180c0af492fa1f43
2019-01-22 09:52:03 -08:00
Marek Cirkos
78832379bc Use apple_binary to generate binary for MacOS (#5383)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5383

It is better supported and also allows us to generate Xcode project

Reviewed By: akindyakov

Differential Revision: D13761638

fbshipit-source-id: 4a1cec6106f5e427e23a85ccee9760579ec4d597
2019-01-22 06:33:44 -08:00
Filipe Manco
e00a30ef87 Fix files with no newline at end of file
Summary: Pull Request resolved: https://github.com/facebook/osquery/pull/5377

Reviewed By: jessek

Differential Revision: D13750577

fbshipit-source-id: a646ff61dcbf1559a9d6f8379fa0295bd3be0451
2019-01-21 11:51:54 -08:00
Filipe Manco
a67525fae1 Fix LICENSE information on file headers (#5375)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5375

LICENSE is now defined in a single file on the root of the project, update the
header to contain that information.

**Project LICENSE did not change.**

Reviewed By: akindyakov

Differential Revision: D13750575

fbshipit-source-id: 1e608a81b260b8395f9d008fc67f463160c1fc2b
2019-01-21 11:51:54 -08:00
Marek Cirkos
85438c100e Mirror/bridge cxx.bzl (#5365)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5365

Rather than having two copies of the same implementation it would be better to just bridge it's implementation

Reviewed By: akindyakov

Differential Revision: D13684438

fbshipit-source-id: 3faf5ddfcc302b6e1e59613169905497d6e98504
2019-01-18 02:45:55 -08:00
Marek Cirkos
70c90c1e54 Mirror/bridge third_party.bzl (#5366)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5366

Rather than having two copies of the same implementation it would be better to just bridge it's implementation

Reviewed By: akindyakov, fmanco

Differential Revision: D13684437

fbshipit-source-id: 95693317c7219ea1d0e0b94f604bd61c4e3a444f
2019-01-18 02:45:55 -08:00
Marek Cirkos
27f8583b73 Mirror/bridge native.bzl (#5364)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5364

Rather than having two copies of the same implementation it would be better to just bridge it's implementation

Reviewed By: akindyakov, fmanco

Differential Revision: D13671592

fbshipit-source-id: e8f9ebbaee587e4f28f63bef3561a84559c278ab
2019-01-17 09:53:14 -08:00
Marek Cirkos
6c868ca3d4 Mirror/bridge python.bzl (#5363)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5363

Rather than having two copies of the same implementation it would be better to just bridge it's implementation

Reviewed By: akindyakov, fmanco

Differential Revision: D13671460

fbshipit-source-id: d1b1b1097ede178d0d645a8ef886f8cecb9e302a
2019-01-17 07:23:25 -08:00
George Guliashvili
d498bcbd65 unique target names for xcode (#5328)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5328

xcode needs unique target names to properly parse buck generated project

Reviewed By: marekcirkos

Differential Revision: D13487400

fbshipit-source-id: cf0c76145344d0873a0973e226d007597a06d17d
2019-01-16 12:28:04 -08:00
seph
637eb104b8 Spelling (#5256)
Summary:
While running `misspell` on a different codebase. I happened to notice that some misspellings in the osquery code base. So, I fixed them
Pull Request resolved: https://github.com/facebook/osquery/pull/5256

Reviewed By: guliashvili

Differential Revision: D13670897

Pulled By: fmanco

fbshipit-source-id: 5d33d858284955c376e8c3980acdf366d4edf3d3
2019-01-16 08:17:07 -08:00
Alexander Kindyakov
3d5309b615 One place to define version of api (#5346)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5346

Let's define win32 api version only inside of buck files, but not in cpp header

Reviewed By: guliashvili

Differential Revision: D13635704

fbshipit-source-id: cd978661ed6f733950363c2ac261811045263ed2
2019-01-14 09:46:36 -08:00
Jonathan Keljo
068efc9d67 Generate strongly-typed TableRow subclasses for each table
Summary:
Continuing to march toward low-overhead, type-safe table rows, this commit
introduces the code generation for said rows. Nothing uses it yet; see the
next commit for that.

(Adapted from https://github.com/facebook/osquery/pull/5199)

Reviewed By: guliashvili

Differential Revision: D13438017

fbshipit-source-id: 959a6e092aee38d33e1c6539cbe14b85172c0135
2019-01-09 13:50:15 -08:00
Jonathan Keljo
1870fd86d8 Introduce TableRow interface
Summary:
Continuing to march toward low-overhead, type-safe table rows, this commit
changes `TableRow` to be an interface rather than simply an alias for `Row`.
Accordingly, `DynamicTableRow` becomes an implementation of that interface
backed by a `Row`. The few remaining pieces of code that treated `TableRow`s as
`Row`s now call methods on the `TableRow` interface. Subsequent commits will
add code generation for strongly-typed table-specific implementations of
`TableRow`.

(Adapted from https://github.com/facebook/osquery/pull/5198)

Reviewed By: guliashvili

Differential Revision: D13438015

fbshipit-source-id: 61d5547e878e519c9706f94f844aab9d3e553410
2019-01-09 13:50:15 -08:00
Jonathan Keljo
b9f1e94fc8 Prefactor: QueryData -> TableRows for virtual tables
Summary:
Continuing to march toward low-overhead, type-safe table rows, this commit introduces
a distinction between rows being returned from a table (`TableRows`) and as the
result of a query (`QueryData`). Right now the two are simply aliases for each other;
that will change shortly.

(Adapted from https://github.com/facebook/osquery/pull/5198)

Reviewed By: guliashvili

Differential Revision: D13438019

fbshipit-source-id: 6563fc8c372d9d6c4b05705943ddf39b42260feb
2019-01-09 13:50:15 -08:00