Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5452
As suggested in another diff, this diff updates the language we use to describe the osquery licensing terms. We are changing all instances of
//This source code is licensed as defined on the LICENSE file found in the root directory of this source tree.//
to
//This source code is licensed in accordance with the terms specified in the LICENSE file found in the root directory of this source tree.//
We accomplish this with a codemod:
$ codemod -md xplat/osquery/oss --extensions cpp,h,in,py,sh,mm,ps1 "(.\s+)This source code is licensed as defined on the LICENSE file found in the(.*)root directory of this source tree\." "\1This source code is licensed in accordance with the terms specified in\2the LICENSE file found in the root directory of this source tree."
Reviewed By: fmanco
Differential Revision: D14131290
fbshipit-source-id: 52c90da342263e2a80f5a678ecd760c19cf7513e
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5451
This diff adds a Facebook copyright header to the bzl files used in osquery. Ultimately we want to update the files in `tools/build_defs/oss/osquery/`, but those are generated files. This diff updates the source files which we use to generate those files.
Reviewed By: fmanco
Differential Revision: D14131483
fbshipit-source-id: 2230dc382c26530ccd0909882fe6193ee7c674fb
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5449
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move killswitch
Move test declaration to it's own BUCK file for consistency with the rest of the project.
Reviewed By: marekcirkos
Differential Revision: D14121618
fbshipit-source-id: 3e30e57befed4387585ed553ec087fdf8db6efc3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5448
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move logger
Reviewed By: marekcirkos
Differential Revision: D14121620
fbshipit-source-id: cef15e7cc354cbe597c6c6878ee63ff09b5fb06d
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5450
This file was originally written by wxsBSD in 2015. He has since joined Facebook and has graciously agreed to re-license this file to Facebook. This diff formalizes the relicensing by changing the copyright notice on the file. Note that wxsBSD still retains a copyright to all previous versions of the file.
Reviewed By: wxsBSD
Differential Revision: D14131447
fbshipit-source-id: 3148eafc0162a23b86e064a9784ea01b685164ef
Summary:
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move distributed
Reviewed By: marekcirkos
Differential Revision: D14121619
fbshipit-source-id: 9ad8a837450874e79a819ab4f11258ae24ec8014
Summary:
Initial steps to separate plugins from the rest of osquery. On the long run separating plugins will provide more build flexibility such that we can have configurable builds that include only the bits and pieces we actually need per deployment. Reducing the attack surface, possibility of supply chain attacks, binary size, etc.
Move config and config_parser plugins
Reviewed By: marekcirkos
Differential Revision: D14119102
fbshipit-source-id: 0bc956398b3829c6f1013b38ebba2f0fc1071a93
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5445
This diff adds a Facebook copyright header to files in the osquery open source repository which:
* Facebook owns
* Do not currently have a Facebook copyright header
Reviewed By: marekcirkos
Differential Revision: D14122845
fbshipit-source-id: 5a0fea10189ec4ec893f7a036911fd51de0e01ae
Summary: It fails on some platforms because of permissions, let's just check if file path is not empty and is absolute
Reviewed By: marekcirkos
Differential Revision: D14086996
fbshipit-source-id: 98068e4b93e6be12a2392345fa74b547d26a2d43
Summary: Please never reuse tmp directory with the same name :(
Reviewed By: guliashvili
Differential Revision: D14066968
fbshipit-source-id: 164d0b9e6f34b102759bc5919dadc37197ff0798
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5438
currently the magic table is broken. libmagic which is used to generate this information needs a database/configuration file that it usually auto-finds.
Our libmagic library tries to open the following file ```open("/usr/local/osquery/Cellar/libmagic/5.32_200/share/misc/magic.mgc", O_RDONLY) = -1 ENOENT (No such file or directory)``` (you can generate this by using strace like ```trace -q -e trace=open ./buck-out/debug/gen/xplat/osquery/oss/osquery/osqueryd#gcc-5-glibc-2.23-clang -verbose -S "select * from magic where path = '/etc/passwd'"```).
How it auto-finds it I don't know 100%, but I guess it has something to with how the libmagic.so is actually build and installed. Basically this never works unless you are a developer on mac and used our previous build system.
I've updated the table to be able to specify the path to magic database file. If you don't specify it, I tried to check if one of the default files (files that should be present under /usr/share/ exists and use the first found). If all fail, I try the default one, but that most likely will fail.
Reviewed By: guliashvili
Differential Revision: D14066467
fbshipit-source-id: d9d2aca4829b2275e6792f974de1f2a7808dc321
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5428
This is a final diff to be able to track syscalls by using eBPF + kernel events. Basically that one and previous are about to join high level initialisation routine in one place.
Part of a linux tracing system, blueprint: [#5218](https://github.com/facebook/osquery/issues/5218)
Reviewed By: SAlexandru
Differential Revision: D13801093
fbshipit-source-id: db8503b0d42127281a975ff517600872e9ed4302
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5437
- test should not assume that fs::temp_directory_path() is always the same
- test should clean everithin up in TearDown() method
- tests should no depend on the order (test_decompression previously was depending on test_compression)
Reviewed By: mkareta
Differential Revision: D14064645
fbshipit-source-id: 653e2061c3de8e3fc30a4f0fc553831f22e62fb7
Summary:
This addresses a slight regression to ensure that we set `set_terminate_threads` on Windows. Without this flag being set, Windows threads will deadlock on exit as the boost managed io service threads never receive termination notifications.
I'm opening this PR up against the old master as I feel we should likely cut a 3.3.3, and I'm happy to re-open this PR against the upstream experimental as well, but we'll want a fix for this released as quickly as possible to Windows deployments.
Pull Request resolved: https://github.com/facebook/osquery/pull/5421
Reviewed By: marekcirkos
Differential Revision: D13972916
Pulled By: muffins
fbshipit-source-id: 55e3b23c80091d5fb51a97d1efc043b52dc48ba3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5432
This diff makes some minor changes to the CONTRIBUTING file. Specifically:
* Clean up the grammar in a few places for clarity.
* Spell out some acronyms the first time they are used.
There shouldn't be any changes to the actual ideas presented.
Reviewed By: guliashvili
Differential Revision: D14030423
fbshipit-source-id: 9d7e4d7c6cf4853b7f5695919a675b5716d90f19
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5419
to load new programs, enable native events and attach program to them.
Reviewed By: SAlexandru
Differential Revision: D13787783
fbshipit-source-id: cfc001da15b343e5c80fd0ab6a276f263aa0ef7a
Summary:
Build osquery extensions/plugins against all osquery is an apparently bad idea.
Because plugins usually are not require full set of thirdparty libs of osquery. Most of osquery code is also not necesarry for plugins. And the main thing, clear definition of public interface of osquery plugins will force us to keep it clear and short, change it carefully and consciously, crate a tests for it.
Reviewed By: fmanco
Differential Revision: D13990668
fbshipit-source-id: ed6ed3f6f75178d829fc6bcbd1af20ef2e268fa8
Summary:
As we're expecting to be auto-generating the buck VS toolchain files, we should add this file to the .gitignore, as it'll potentially be system specific.
Pull Request resolved: https://github.com/facebook/osquery/pull/5423
Reviewed By: marekcirkos
Differential Revision: D13973094
Pulled By: muffins
fbshipit-source-id: 98325a4dbe444915d066cd259d9312b09347b080
Summary: Removing flag which was declared but never used. enable_monitor
Reviewed By: marekcirkos
Differential Revision: D13958265
fbshipit-source-id: 3a812330950b101abdbd83ada4afd5b262cabd26
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5422
We were just de-prioritizing type of queries not constraining required columns. However, when the query is just useless without specific constraint, sqlite suggestion is to return SQLITE_CONSTRAINT status.
Reviewed By: marekcirkos
Differential Revision: D13964562
fbshipit-source-id: ee0e5f8baf9abbf83c34f7a39d2b5bd705cbac6d
Summary:
Fixed crash in virtual tables that occurs after following steps:
1. sqlite opens VT with xCreate
2. during query executions invokes xFilter with multiple tables
3. Few tables accumulated in affectedTables
4. xDestroy called before finishing query (last step of query execution)
5. query execution finished, SQL instance try to cleanup affected tables, but they were already destroyed by xDestory
This is only hotfix for this crash and this code base require full memory management review in future
Reviewed By: SAlexandru
Differential Revision: D13917015
fbshipit-source-id: 15396e47e4c4e592cf30608a783bc80d560c776f
Summary:
This diff adds Xcode support for osquery.
Part of this diff will be reverted in future after adding prebuilt library and platform deps support to buck.
To use it you need to build osquery in debug mode and then run buck with following flags:
--config osquery.xcode=true --config project.ide=xcode
Reviewed By: SAlexandru
Differential Revision: D13903315
fbshipit-source-id: 4d131964d7a61236f25d917dc060a2f3c3d782bc
Summary: before this diff we were using -O flag, which equals to -O2, and our debug builds were optimized, which make debug much harder
Reviewed By: fmanco
Differential Revision: D13956134
fbshipit-source-id: b358d8fd68c8f5d51ae6d4c2033e7ec3afdd50d2
Summary:
Not every environment requires all tables, this diff introduce flag that allows you mark table as foreign. New option should be used in conjunction with target filer.
Example:
> buck build ... --config osquery.target_ignore_list="smart" --config osquery.spec_ignore_list="smart/smart_drive_info.table" -- -S
Reviewed By: fmanco
Differential Revision: D13942107
fbshipit-source-id: fb34d6b7a296f69f6b95bf17bfd19cee31b34dec
Summary:
Not every environment require all osquery feature, with this diff you can specify targets that you want to ignore, together with all sub tree of deps. To use this you need to specify new osquery config like:
[osquery]
target_ignore_list="kafka_producer"
Or from command line:
--config osquery.target_ignore_list="kafka_producer"
This also includes killswitch that force buck to build all targets. This is needed when you have local buckcofig with ignore list and want to build all without modifying config.
--config osquery.force_build_all=true
Reviewed By: fmanco
Differential Revision: D13941689
fbshipit-source-id: 3c4e1b4cda4d74f33fb914ba2c3a17df4710d5d3
Summary: We fixed the meaning of `start_time` in previous PR/diffs, let's check it in the integration test.
Reviewed By: SAlexandru
Differential Revision: D13918628
fbshipit-source-id: 60f4ff74eadfbe286dfb325d713389f01142a0f8
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5412
Now on different platforms column `start_time` in `processes` table means different things. On Linux it is seconds since system boot, but it works correct only for some platforms, because the number of clock ticks per second was hardcoded. On windows it was abs unix time in seconds since Epoch. On macos it is a time in milliseconds (may be?) since system boot. On freeBSD as far as I can see it an abs time since boot, but also I'm not sure.
In order to make it consistent for all OS we changed to more convenient format - absolute time since Epoch. This commit is about macos.
Reviewed By: marekcirkos
Differential Revision: D13918625
fbshipit-source-id: eacb297358b36ce72cb0d5a7d9171553688ab2a3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5414
Now on different platforms column `start_time` in `processes` table means different things. On Linux it is seconds since system boot, but it works correct only for some platforms, because the number of clock ticks per second was hardcoded. On windows it was abs unix time in seconds since Epoch. On macos it is a time in milliseconds (may be?) since system boot. On freeBSD as far as I can see it an abs time since boot, but also I'm not sure.
In order to make it consistent for all OS we changed to more convenient format - absolute time since Epoch. This commit is about Linux. Next diffs going to be about Darwin and freeBSD.
Reviewed By: guliashvili
Differential Revision: D13918626
fbshipit-source-id: a9cf0570dc6ac9fa125bc8233e9965c4e01566a6
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5417
Hash multimap based joiner with ability to perform clean up old unpaired events from time to time.
Part of a linux tracing system, blueprint: [#5218](https://github.com/facebook/osquery/issues/5218)
Reviewed By: SAlexandru
Differential Revision: D13761675
fbshipit-source-id: f4b17cbeed495b2a9e6616a005f001963849875e
Summary: For some monitoring, we need to include machine identity. Two example, I found is hostname or similar hostnames schema name.
Reviewed By: SAlexandru
Differential Revision: D13880705
fbshipit-source-id: e1d0238f4981adad1554d73f0ef6e5ef65a98c33
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5408
Move out c++ enums to std::string conversion function from
osquery/utils/error.h to separate module. To be able to use it somewhere else.
Reviewed By: guliashvili
Differential Revision: D13896772
fbshipit-source-id: 0a9f6327d5b2f115ce688446a67677879411eb1f
Summary: before this diff osqueryd was relying on linker order to use right main function, since gtest also contains main function
Reviewed By: guliashvili
Differential Revision: D13897622
fbshipit-source-id: d260b7496f513c7052f4db87c8e7ff9300493671
