Summary:
This addresses a slight regression to ensure that we set `set_terminate_threads` on Windows. Without this flag being set, Windows threads will deadlock on exit as the boost managed io service threads never receive termination notifications.
I'm opening this PR up against the old master as I feel we should likely cut a 3.3.3, and I'm happy to re-open this PR against the upstream experimental as well, but we'll want a fix for this released as quickly as possible to Windows deployments.
Pull Request resolved: https://github.com/facebook/osquery/pull/5421
Reviewed By: marekcirkos
Differential Revision: D13972916
Pulled By: muffins
fbshipit-source-id: 55e3b23c80091d5fb51a97d1efc043b52dc48ba3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5432
This diff makes some minor changes to the CONTRIBUTING file. Specifically:
* Clean up the grammar in a few places for clarity.
* Spell out some acronyms the first time they are used.
There shouldn't be any changes to the actual ideas presented.
Reviewed By: guliashvili
Differential Revision: D14030423
fbshipit-source-id: 9d7e4d7c6cf4853b7f5695919a675b5716d90f19
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5419
to load new programs, enable native events and attach program to them.
Reviewed By: SAlexandru
Differential Revision: D13787783
fbshipit-source-id: cfc001da15b343e5c80fd0ab6a276f263aa0ef7a
Summary:
Build osquery extensions/plugins against all osquery is an apparently bad idea.
Because plugins usually are not require full set of thirdparty libs of osquery. Most of osquery code is also not necesarry for plugins. And the main thing, clear definition of public interface of osquery plugins will force us to keep it clear and short, change it carefully and consciously, crate a tests for it.
Reviewed By: fmanco
Differential Revision: D13990668
fbshipit-source-id: ed6ed3f6f75178d829fc6bcbd1af20ef2e268fa8
Summary:
As we're expecting to be auto-generating the buck VS toolchain files, we should add this file to the .gitignore, as it'll potentially be system specific.
Pull Request resolved: https://github.com/facebook/osquery/pull/5423
Reviewed By: marekcirkos
Differential Revision: D13973094
Pulled By: muffins
fbshipit-source-id: 98325a4dbe444915d066cd259d9312b09347b080
Summary: Removing flag which was declared but never used. enable_monitor
Reviewed By: marekcirkos
Differential Revision: D13958265
fbshipit-source-id: 3a812330950b101abdbd83ada4afd5b262cabd26
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5422
We were just de-prioritizing type of queries not constraining required columns. However, when the query is just useless without specific constraint, sqlite suggestion is to return SQLITE_CONSTRAINT status.
Reviewed By: marekcirkos
Differential Revision: D13964562
fbshipit-source-id: ee0e5f8baf9abbf83c34f7a39d2b5bd705cbac6d
Summary:
Fixed crash in virtual tables that occurs after following steps:
1. sqlite opens VT with xCreate
2. during query executions invokes xFilter with multiple tables
3. Few tables accumulated in affectedTables
4. xDestroy called before finishing query (last step of query execution)
5. query execution finished, SQL instance try to cleanup affected tables, but they were already destroyed by xDestory
This is only hotfix for this crash and this code base require full memory management review in future
Reviewed By: SAlexandru
Differential Revision: D13917015
fbshipit-source-id: 15396e47e4c4e592cf30608a783bc80d560c776f
Summary:
This diff adds Xcode support for osquery.
Part of this diff will be reverted in future after adding prebuilt library and platform deps support to buck.
To use it you need to build osquery in debug mode and then run buck with following flags:
--config osquery.xcode=true --config project.ide=xcode
Reviewed By: SAlexandru
Differential Revision: D13903315
fbshipit-source-id: 4d131964d7a61236f25d917dc060a2f3c3d782bc
Summary: before this diff we were using -O flag, which equals to -O2, and our debug builds were optimized, which make debug much harder
Reviewed By: fmanco
Differential Revision: D13956134
fbshipit-source-id: b358d8fd68c8f5d51ae6d4c2033e7ec3afdd50d2
Summary:
Not every environment requires all tables, this diff introduce flag that allows you mark table as foreign. New option should be used in conjunction with target filer.
Example:
> buck build ... --config osquery.target_ignore_list="smart" --config osquery.spec_ignore_list="smart/smart_drive_info.table" -- -S
Reviewed By: fmanco
Differential Revision: D13942107
fbshipit-source-id: fb34d6b7a296f69f6b95bf17bfd19cee31b34dec
Summary:
Not every environment require all osquery feature, with this diff you can specify targets that you want to ignore, together with all sub tree of deps. To use this you need to specify new osquery config like:
[osquery]
target_ignore_list="kafka_producer"
Or from command line:
--config osquery.target_ignore_list="kafka_producer"
This also includes killswitch that force buck to build all targets. This is needed when you have local buckcofig with ignore list and want to build all without modifying config.
--config osquery.force_build_all=true
Reviewed By: fmanco
Differential Revision: D13941689
fbshipit-source-id: 3c4e1b4cda4d74f33fb914ba2c3a17df4710d5d3
Summary: We fixed the meaning of `start_time` in previous PR/diffs, let's check it in the integration test.
Reviewed By: SAlexandru
Differential Revision: D13918628
fbshipit-source-id: 60f4ff74eadfbe286dfb325d713389f01142a0f8
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5412
Now on different platforms column `start_time` in `processes` table means different things. On Linux it is seconds since system boot, but it works correct only for some platforms, because the number of clock ticks per second was hardcoded. On windows it was abs unix time in seconds since Epoch. On macos it is a time in milliseconds (may be?) since system boot. On freeBSD as far as I can see it an abs time since boot, but also I'm not sure.
In order to make it consistent for all OS we changed to more convenient format - absolute time since Epoch. This commit is about macos.
Reviewed By: marekcirkos
Differential Revision: D13918625
fbshipit-source-id: eacb297358b36ce72cb0d5a7d9171553688ab2a3
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5414
Now on different platforms column `start_time` in `processes` table means different things. On Linux it is seconds since system boot, but it works correct only for some platforms, because the number of clock ticks per second was hardcoded. On windows it was abs unix time in seconds since Epoch. On macos it is a time in milliseconds (may be?) since system boot. On freeBSD as far as I can see it an abs time since boot, but also I'm not sure.
In order to make it consistent for all OS we changed to more convenient format - absolute time since Epoch. This commit is about Linux. Next diffs going to be about Darwin and freeBSD.
Reviewed By: guliashvili
Differential Revision: D13918626
fbshipit-source-id: a9cf0570dc6ac9fa125bc8233e9965c4e01566a6
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5417
Hash multimap based joiner with ability to perform clean up old unpaired events from time to time.
Part of a linux tracing system, blueprint: [#5218](https://github.com/facebook/osquery/issues/5218)
Reviewed By: SAlexandru
Differential Revision: D13761675
fbshipit-source-id: f4b17cbeed495b2a9e6616a005f001963849875e
Summary: For some monitoring, we need to include machine identity. Two example, I found is hostname or similar hostnames schema name.
Reviewed By: SAlexandru
Differential Revision: D13880705
fbshipit-source-id: e1d0238f4981adad1554d73f0ef6e5ef65a98c33
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5408
Move out c++ enums to std::string conversion function from
osquery/utils/error.h to separate module. To be able to use it somewhere else.
Reviewed By: guliashvili
Differential Revision: D13896772
fbshipit-source-id: 0a9f6327d5b2f115ce688446a67677879411eb1f
Summary: before this diff osqueryd was relying on linker order to use right main function, since gtest also contains main function
Reviewed By: guliashvili
Differential Revision: D13897622
fbshipit-source-id: d260b7496f513c7052f4db87c8e7ff9300493671
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5406
use new flag file for windows toolchain configuration
Reviewed By: mkareta
Differential Revision: D13859827
fbshipit-source-id: 7560c989e2590663d511b5b644ba6bc3c6b5e5f4
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5405
this should find where VS is installed and set the buck flags properly.
Have tested on my VM and the paths are ok. This only works for 2017 and newer (hopefully)
I'm not sure how future proof this is, Microsoft usually changes directory structures randomly.
Reviewed By: muffins
Differential Revision: D13762391
fbshipit-source-id: 894e6a6d5888e13ab646ca9cb4a0d604bcf53ee5
Summary:
This PR is the result of the discussion in a previous PR (#5348) after we determined account_policy_data was the wrong place for the column.
Add `is_hidden` column to the users and groups tables in macOS. `is_hidden` is populated by looking for the `dsAttrTypeNative:IsHidden` attribute in the OpenDirectory record for the user/group if the value is `1`, `True`, or `Yes` is_hidden is 1. If the value is anything else it's set to 0. Invalid values have the same affect as the attribute not existing at all.
The `dsAttrTypeNative:IsHidden` attribute controls whether a user account is is visible in the preferences panel similar to having a uid < 500.
One test failed when running buck test:
```
====STANDARD OUT====
tests/integration/tables/helper.cpp:159: Failure
Value of: boost::get<CustomCheckerType>(validator)(value)
Actual: false
Expected: true
Custom validator of the column "mask" with value "" failed
```
This also fails when I ran the test on the current experimental branch as well.
Important to note I had to remove the optimization on both the user and group tables that just called `getpwnam` if the query specified the `uid` or `gid` since the struct returned doesn't contain the `IsHidden` attribute. I'm not sure if or how much this will affect performance since I wasn't able to get the profiling to work with the new version (very likely I'm just doing it incorrectly).
Pull Request resolved: https://github.com/facebook/osquery/pull/5368
Differential Revision: D13862375
Pulled By: akindyakov
fbshipit-source-id: 1fec88a6ba71884f7e611e1d96ea00630c5be655
Summary: profiler file contained the CodeProfiler class, which is a bit odd. Thanks alecx
Reviewed By: akindyakov
Differential Revision: D13861320
fbshipit-source-id: 11d711fd1d15830d4cce0be084cc6da3b1c6589c
Summary:
Split query name and pack name with the delimiter(dot) which should not accrue inside their names(instead of _ or -).
Also log things per pack name
Reviewed By: SAlexandru
Differential Revision: D13854471
fbshipit-source-id: 181e7e19fcfb5d57a779cea6a2804eda09dc5a91
Summary:
Split query name and pack name with the delimiter(dot) which should not accrue inside their names(instead of _ or -).
Also log things per pack name
Reviewed By: mkareta
Differential Revision: D13854389
fbshipit-source-id: 35ba80813d98371dd36a83ec32aad677f24aa6eb
Summary: ScheduledQuery was declared both in query.h and in scheduled_query.h. Let's keep scheduled_query.h only.
Reviewed By: mkareta
Differential Revision: D13859371
fbshipit-source-id: 241b948d21fe619be360037a4e2e52a833a23b2b
Summary: In the interest of making changes smaller, I'm replicating stuff around QueryData to QueryDataTyped in intermediate commits. I'll come back and remove stuff that's no longer used when subsequent changes eliminate use of them.
Reviewed By: guliashvili
Differential Revision: D13741994
fbshipit-source-id: de0b00ddb0ad4b344d68ce799fe9fac759bb6199
Summary: Also it causes debug failure on unchecked Expected when status is not ok
Reviewed By: mkareta
Differential Revision: D13859377
fbshipit-source-id: 596410350cb91d469dc0a19f0e8eec558e8627bd
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5398
The file moved to a different location.
Reviewed By: marekcirkos
Differential Revision: D13817331
fbshipit-source-id: 6bd3947894daa712edae84b71502af997947b9b0
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5396
Buckify new version of the sqlite.
Reviewed By: marekcirkos
Differential Revision: D13801135
fbshipit-source-id: 0acb5c90c962b114dc2a699cfe84f9264efae336
