Summary:
I didn't find a way to refer file in fbcode thrift_library from xplat. So, let's copy it.
There is something wrong with a namespaces in python - I'll fix it later. This is not a problem for now - file is not used.
Reviewed By: marekcirkos
Differential Revision: D14124542
fbshipit-source-id: 1060e23675a8086af494aa93037e1c613f250c71
Summary:
As far as this file is only for internal build, to be able to build osquery sdk against fbthrift I moved it to fbcode from oss.
Later if we want to osqeruy work with fbthrift we can copy/move it again
Reviewed By: marekcirkos
Differential Revision: D14124448
fbshipit-source-id: aaeadb40a6fac19621423b225cf6d522607d1313
Summary:
We need to be able to build `fbcode` projects with dependencies to `fbsource/xplat/osquery/oss/sdk:plugin_sdk`. As far as osquery is a part of `fbsource` now it would be very useful to build against it, make a tests. Which will helps us a lot to develop faster, will unblock us to run tests for every change either to `xplat/osquery` and to `fbcode`, which going to prevent code from bugs and interface breaking (which happens now too often).
`osquery` is very platform dependent project, because it built internally at least for 4 OS: `linux`, `freebsd`, `windows`, `darwin`. `osquery` has its own third-party libraries located in `fbsource/xplat/osquery/third-party`.
Also we have internal osquery extension (`fb_osquery`) in `fbcode` built with strong dependency to `osquery` and with lots of dependencies to `fbcode` projects (e.g. scribe, GK, ODS, configurator, serivicerouter and more).
We could not build `fb_osquery` directly against `osquery` because build system restrictions and third-party dependencies collision.
- Add necessary for `fb_osquery` parts of `fbsource/xpat/osquery` to xplat whitelist.
- Make it possible to use `fbcode` `cpp_library` target definition for `fbsource/xplat/osquery` targets when they are used for `fbcode` build.
- Make a translation platform dependant osquery targets for fbcode platform independent build.
- Use `fbcode/tp` libs instead of `fbsource/xplat/osquery/tp` in case of `fbcode` build.
Differential Revision: D13991062
fbshipit-source-id: 1294825f1c5f991bd465e0e299b8e5ff67bbc543
Summary:
Hi! This PR adds a new column called `product_version` to the file table, which is only
populated when queries are done on Windows. It is a very minimal PR that uses an existing helper function (`windowsGetFileVersion`) to populate the column.
The column is not named `file_version`, despite the name of the helper function because the underlying data retrieved by that helper function is actually the `dwProductVersion*` fields of the `VS_FIXEDFILEINFO` struct. In the future, if we want to add a column that _actually_ contains the results of the `dwFileVersion*` fields, we can add a new column called `file_version` without modifying existing functionality.
Pull Request resolved: https://github.com/facebook/osquery/pull/5431
Differential Revision: D14169773
Pulled By: fmanco
fbshipit-source-id: 6fa7c92425fc92aa2e83a37383b1a8c796b17644
Summary: It cause problems with compiling against tp2 because in tp2 there is no such target. As far as I can see there is no necessity to depend on that target separatelly from gtest. So, the simplest solution is to remove it
Reviewed By: SAlexandru
Differential Revision: D14149801
fbshipit-source-id: fe0fcd0593bded7c8f2c6fd247b1d52bd29bac9f
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5459
This diff updates the osquery Code of Conduct policy to reference the Facebook Open Source Code of Conduct, https://code.fb.com/codeofconduct/.
Reviewed By: marekcirkos
Differential Revision: D14149368
fbshipit-source-id: 4e6ae7acf9367053268c5d8fb6289b5e94703604
Summary: This test is flaky on some systems. Need to investigate and fix.
Reviewed By: guliashvili
Differential Revision: D14149533
fbshipit-source-id: 7df570c84c66cad8bdc67a1286d91a7c7e8e3502
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5458
Use ASSERTs where necessary. ASSERTs are fatal and terminate the test immediately, so should be used when subsequent tests aren't meaningful in case of previous failures. Only use EXPECTED when it makes sense for the tests to continue.
Reviewed By: jessek
Differential Revision: D14138382
fbshipit-source-id: aae0cd8640e602ac467861303457f9cf017bc5e8
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5455
We recently added a Facebook copyright header to many files in the osquery repository. Unfortunately we missed this one in the first diff.
Reviewed By: fmanco
Differential Revision: D14140835
fbshipit-source-id: 370da0af4e67723f8c5002158e643f1f20fd48ba
Summary: This was patched with D13767582 and is already deployed
Reviewed By: mkareta
Differential Revision: D14124516
fbshipit-source-id: 30679472458f4ed9647adc117db4352b940cf1cf
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5452
As suggested in another diff, this diff updates the language we use to describe the osquery licensing terms. We are changing all instances of
//This source code is licensed as defined on the LICENSE file found in the root directory of this source tree.//
to
//This source code is licensed in accordance with the terms specified in the LICENSE file found in the root directory of this source tree.//
We accomplish this with a codemod:
$ codemod -md xplat/osquery/oss --extensions cpp,h,in,py,sh,mm,ps1 "(.\s+)This source code is licensed as defined on the LICENSE file found in the(.*)root directory of this source tree\." "\1This source code is licensed in accordance with the terms specified in\2the LICENSE file found in the root directory of this source tree."
Reviewed By: fmanco
Differential Revision: D14131290
fbshipit-source-id: 52c90da342263e2a80f5a678ecd760c19cf7513e
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5451
This diff adds a Facebook copyright header to the bzl files used in osquery. Ultimately we want to update the files in `tools/build_defs/oss/osquery/`, but those are generated files. This diff updates the source files which we use to generate those files.
Reviewed By: fmanco
Differential Revision: D14131483
fbshipit-source-id: 2230dc382c26530ccd0909882fe6193ee7c674fb
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5449
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move killswitch
Move test declaration to it's own BUCK file for consistency with the rest of the project.
Reviewed By: marekcirkos
Differential Revision: D14121618
fbshipit-source-id: 3e30e57befed4387585ed553ec087fdf8db6efc3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5448
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move logger
Reviewed By: marekcirkos
Differential Revision: D14121620
fbshipit-source-id: cef15e7cc354cbe597c6c6878ee63ff09b5fb06d
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5450
This file was originally written by wxsBSD in 2015. He has since joined Facebook and has graciously agreed to re-license this file to Facebook. This diff formalizes the relicensing by changing the copyright notice on the file. Note that wxsBSD still retains a copyright to all previous versions of the file.
Reviewed By: wxsBSD
Differential Revision: D14131447
fbshipit-source-id: 3148eafc0162a23b86e064a9784ea01b685164ef
Summary:
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move distributed
Reviewed By: marekcirkos
Differential Revision: D14121619
fbshipit-source-id: 9ad8a837450874e79a819ab4f11258ae24ec8014
Summary:
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move config and config_parser plugins
Reviewed By: marekcirkos
Differential Revision: D14119102
fbshipit-source-id: 0bc956398b3829c6f1013b38ebba2f0fc1071a93
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5445
This diff adds a Facebook copyright header to files in the osquery open source repository which:
* Facebook owns
* Do not currently have a Facebook copyright header
Reviewed By: marekcirkos
Differential Revision: D14122845
fbshipit-source-id: 5a0fea10189ec4ec893f7a036911fd51de0e01ae
Summary: It fails on some platforms because of permissions, let's just check if file path is not empty and is absolute
Reviewed By: marekcirkos
Differential Revision: D14086996
fbshipit-source-id: 98068e4b93e6be12a2392345fa74b547d26a2d43
Summary: Please never reuse tmp directory with the same name :(
Reviewed By: guliashvili
Differential Revision: D14066968
fbshipit-source-id: 164d0b9e6f34b102759bc5919dadc37197ff0798
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5438
currently the magic table is broken. libmagic which is used to generate this information needs a database/configuration file that it usually auto-finds.
Our libmagic library tries to open the following file ```open("/usr/local/osquery/Cellar/libmagic/5.32_200/share/misc/magic.mgc", O_RDONLY) = -1 ENOENT (No such file or directory)``` (you can generate this by using strace like ```trace -q -e trace=open ./buck-out/debug/gen/xplat/osquery/oss/osquery/osqueryd#gcc-5-glibc-2.23-clang -verbose -S "select * from magic where path = '/etc/passwd'"```).
How it auto-finds it I don't know 100%, but I guess it has something to with how the libmagic.so is actually build and installed. Basically this never works unless you are a developer on mac and used our previous build system.
I've updated the table to be able to specify the path to magic database file. If you don't specify it, I tried to check if one of the default files (files that should be present under /usr/share/ exists and use the first found). If all fail, I try the default one, but that most likely will fail.
Reviewed By: guliashvili
Differential Revision: D14066467
fbshipit-source-id: d9d2aca4829b2275e6792f974de1f2a7808dc321
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5428
This is a final diff to be able to track syscalls by using eBPF + kernel events. Basically that one and previous are about to join high level initialisation routine in one place.
Part of a linux tracing system, blueprint: [#5218](https://github.com/facebook/osquery/issues/5218)
Reviewed By: SAlexandru
Differential Revision: D13801093
fbshipit-source-id: db8503b0d42127281a975ff517600872e9ed4302
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5437
- test should not assume that fs::temp_directory_path() is always the same
- test should clean everithin up in TearDown() method
- tests should no depend on the order (test_decompression previously was depending on test_compression)
Reviewed By: mkareta
Differential Revision: D14064645
fbshipit-source-id: 653e2061c3de8e3fc30a4f0fc553831f22e62fb7
Summary:
This addresses a slight regression to ensure that we set `set_terminate_threads` on Windows. Without this flag being set, Windows threads will deadlock on exit as the boost managed io service threads never receive termination notifications.
I'm opening this PR up against the old master as I feel we should likely cut a 3.3.3, and I'm happy to re-open this PR against the upstream experimental as well, but we'll want a fix for this released as quickly as possible to Windows deployments.
Pull Request resolved: https://github.com/facebook/osquery/pull/5421
Reviewed By: marekcirkos
Differential Revision: D13972916
Pulled By: muffins
fbshipit-source-id: 55e3b23c80091d5fb51a97d1efc043b52dc48ba3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5432
This diff makes some minor changes to the CONTRIBUTING file. Specifically:
* Clean up the grammar in a few places for clarity.
* Spell out some acronyms the first time they are used.
There shouldn't be any changes to the actual ideas presented.
Reviewed By: guliashvili
Differential Revision: D14030423
fbshipit-source-id: 9d7e4d7c6cf4853b7f5695919a675b5716d90f19
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5419
to load new programs, enable native events and attach program to them.
Reviewed By: SAlexandru
Differential Revision: D13787783
fbshipit-source-id: cfc001da15b343e5c80fd0ab6a276f263aa0ef7a
Summary:
Build osquery extensions/plugins against all osquery is an apparently bad idea.
Because plugins usually are not require full set of thirdparty libs of osquery. Most of osquery code is also not necesarry for plugins. And the main thing, clear definition of public interface of osquery plugins will force us to keep it clear and short, change it carefully and consciously, crate a tests for it.
Reviewed By: fmanco
Differential Revision: D13990668
fbshipit-source-id: ed6ed3f6f75178d829fc6bcbd1af20ef2e268fa8
Summary:
As we're expecting to be auto-generating the buck VS toolchain files, we should add this file to the .gitignore, as it'll potentially be system specific.
Pull Request resolved: https://github.com/facebook/osquery/pull/5423
Reviewed By: marekcirkos
Differential Revision: D13973094
Pulled By: muffins
fbshipit-source-id: 98325a4dbe444915d066cd259d9312b09347b080
Summary: Removing flag which was declared but never used. enable_monitor
Reviewed By: marekcirkos
Differential Revision: D13958265
fbshipit-source-id: 3a812330950b101abdbd83ada4afd5b262cabd26
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5422
We were just de-prioritizing type of queries not constraining required columns. However, when the query is just useless without specific constraint, sqlite suggestion is to return SQLITE_CONSTRAINT status.
Reviewed By: marekcirkos
Differential Revision: D13964562
fbshipit-source-id: ee0e5f8baf9abbf83c34f7a39d2b5bd705cbac6d
Summary:
Fixed crash in virtual tables that occurs after following steps:
1. sqlite opens VT with xCreate
2. during query executions invokes xFilter with multiple tables
3. Few tables accumulated in affectedTables
4. xDestroy called before finishing query (last step of query execution)
5. query execution finished, SQL instance try to cleanup affected tables, but they were already destroyed by xDestory
This is only hotfix for this crash and this code base require full memory management review in future
Reviewed By: SAlexandru
Differential Revision: D13917015
fbshipit-source-id: 15396e47e4c4e592cf30608a783bc80d560c776f
Summary:
This diff adds Xcode support for osquery.
Part of this diff will be reverted in future after adding prebuilt library and platform deps support to buck.
To use it you need to build osquery in debug mode and then run buck with following flags:
--config osquery.xcode=true --config project.ide=xcode
Reviewed By: SAlexandru
Differential Revision: D13903315
fbshipit-source-id: 4d131964d7a61236f25d917dc060a2f3c3d782bc
Summary: before this diff we were using -O flag, which equals to -O2, and our debug builds were optimized, which make debug much harder
Reviewed By: fmanco
Differential Revision: D13956134
fbshipit-source-id: b358d8fd68c8f5d51ae6d4c2033e7ec3afdd50d2
Summary:
Not every environment requires all tables, this diff introduce flag that allows you mark table as foreign. New option should be used in conjunction with target filer.
Example:
> buck build ... --config osquery.target_ignore_list="smart" --config osquery.spec_ignore_list="smart/smart_drive_info.table" -- -S
Reviewed By: fmanco
Differential Revision: D13942107
fbshipit-source-id: fb34d6b7a296f69f6b95bf17bfd19cee31b34dec
Summary:
Not every environment require all osquery feature, with this diff you can specify targets that you want to ignore, together with all sub tree of deps. To use this you need to specify new osquery config like:
[osquery]
target_ignore_list="kafka_producer"
Or from command line:
--config osquery.target_ignore_list="kafka_producer"
This also includes killswitch that force buck to build all targets. This is needed when you have local buckcofig with ignore list and want to build all without modifying config.
--config osquery.force_build_all=true
Reviewed By: fmanco
Differential Revision: D13941689
fbshipit-source-id: 3c4e1b4cda4d74f33fb914ba2c3a17df4710d5d3
Summary: We fixed the meaning of `start_time` in previous PR/diffs, let's check it in the integration test.
Reviewed By: SAlexandru
Differential Revision: D13918628
fbshipit-source-id: 60f4ff74eadfbe286dfb325d713389f01142a0f8
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5412
Now on different platforms column `start_time` in `processes` table means different things. On Linux it is seconds since system boot, but it works correct only for some platforms, because the number of clock ticks per second was hardcoded. On windows it was abs unix time in seconds since Epoch. On macos it is a time in milliseconds (may be?) since system boot. On freeBSD as far as I can see it an abs time since boot, but also I'm not sure.
In order to make it consistent for all OS we changed to more convenient format - absolute time since Epoch. This commit is about macos.
Reviewed By: marekcirkos
Differential Revision: D13918625
fbshipit-source-id: eacb297358b36ce72cb0d5a7d9171553688ab2a3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5414
Now on different platforms column `start_time` in `processes` table means different things. On Linux it is seconds since system boot, but it works correct only for some platforms, because the number of clock ticks per second was hardcoded. On windows it was abs unix time in seconds since Epoch. On macos it is a time in milliseconds (may be?) since system boot. On freeBSD as far as I can see it an abs time since boot, but also I'm not sure.
In order to make it consistent for all OS we changed to more convenient format - absolute time since Epoch. This commit is about Linux. Next diffs going to be about Darwin and freeBSD.
Reviewed By: guliashvili
Differential Revision: D13918626
fbshipit-source-id: a9cf0570dc6ac9fa125bc8233e9965c4e01566a6
