Commit Graph

2886 Commits

Author SHA1 Message Date
Steven Peters
41b785b86c Fix build with boost 1.69: add missing boost/noncopyable.hpp includes (#5325)
Summary:
boost 1.69 is in a pull request in homebrew-core at https://github.com/Homebrew/homebrew-core/pull/35030 and `osquery` is failing to compile due to using `boost::noncopyable` in a couple header files without including `boost/noncopyable.hpp`:

* [jenkins log of build failure](https://jenkins.brew.sh/job/Homebrew%20Core%20Pull%20Requests/34941/version=high_sierra/testReport/junit/brew-test-bot/high_sierra/install_osquery/)

The build still fails on homebrew due to #5284.
Pull Request resolved: https://github.com/facebook/osquery/pull/5325

Differential Revision: D13464741

Pulled By: fmanco

fbshipit-source-id: bdaf573b180f8cdfd08dc719db4829911249caaf
2019-01-10 08:48:37 -08:00
Jeremy Calvert
a1058b7372 s/processesRow/ProcessesRow/ on osquery/oss/osquery/tables/system/tests/darwin/processes_tests.cpp
Reviewed By: jkeljo

Differential Revision: D13618914

fbshipit-source-id: 48c0d86107723446497638e6a2367d92b3876586
2019-01-09 18:44:01 -08:00
Jonathan Keljo
f7ec202778 Optimize process command line computation
Summary:
Previously we computed the args and environment, then discarded the environment and joined the args together to make the command line. By cutting out all that extra work, we get a ~3% CPU win for a typical `processes` query.

Before:
```
 D:0  C:0  M:2  F:0  U:0  pack_processes_resources (1/5): duration: 0.592410087585 cpu_time: 0.053588293 memory: 12603392 fds: 4 utilization: 5.05
 D:0  C:0  M:1  F:0  U:0  pack_processes_resources (2/5): duration: 0.52742600441 cpu_time: 0.05227561 memory: 12578816 fds: 4 utilization: 4.95
 D:0  C:0  M:2  F:0  U:0  pack_processes_resources (3/5): duration: 0.524594068527 cpu_time: 0.052265788 memory: 12582912 fds: 4 utilization: 4.95
 D:0  C:0  M:2  F:0  U:0  pack_processes_resources (4/5): duration: 0.519255876541 cpu_time: 0.052351296 memory: 12697600 fds: 4 utilization: 5.0
 D:0  C:0  M:1  F:0  U:0  pack_processes_resources (5/5): duration: 0.525310993195 cpu_time: 0.05189243 memory: 12570624 fds: 4 utilization: 4.9
 D:0  C:0  M:2  F:0  U:0  pack_processes_resources   avg: duration: 0.537799406052 cpu_time: 0.0524746834 memory: 12606668.8 fds: 4.0 utilization: 4.97
```

After:
```
 D:0  C:0  M:1  F:0  U:0  pack_processes_resources (1/5): duration: 0.577349901199 cpu_time: 0.052007643 memory: 12525568 fds: 4 utilization: 4.9
 D:0  C:0  M:1  F:0  U:0  pack_processes_resources (2/5): duration: 0.516617059708 cpu_time: 0.050582555 memory: 12480512 fds: 4 utilization: 4.8
 D:0  C:0  M:1  F:0  U:0  pack_processes_resources (3/5): duration: 0.519332170486 cpu_time: 0.049710162 memory: 12492800 fds: 4 utilization: 4.7
 D:0  C:0  M:1  F:0  U:0  pack_processes_resources (4/5): duration: 0.523589849472 cpu_time: 0.051366226 memory: 12578816 fds: 4 utilization: 4.9
 D:0  C:0  M:1  F:0  U:0  pack_processes_resources (5/5): duration: 0.528687000275 cpu_time: 0.049640225 memory: 12529664 fds: 4 utilization: 4.7
 D:0  C:0  M:1  F:0  U:0  pack_processes_resources   avg: duration: 0.533115196228 cpu_time: 0.0506613622 memory: 12521472.0 fds: 4.0 utilization: 4.8
```

(Adapted from https://github.com/facebook/osquery/pull/5200)

Reviewed By: guliashvili

Differential Revision: D13458612

fbshipit-source-id: 5642225a673f8ce954a60ec01fb7ddfcc79c0608
2019-01-09 13:50:15 -08:00
Jonathan Keljo
5bb1ec75a0 Migrate processes to strongly-typed TableRows on macOS
Summary:
Continuing to march toward low-overhead, type-safe table rows, this commit
converts the Darwin `processes` table to using the generated row types.

My march concludes here, but there's a lot of work yet to be done with
migrating other tables.

(Adapted from https://github.com/facebook/osquery/pull/5199)

Reviewed By: guliashvili

Differential Revision: D13438014

fbshipit-source-id: 0e8365f26fe95131fe53ba3491cf07899600e997
2019-01-09 13:50:15 -08:00
Jonathan Keljo
068efc9d67 Generate strongly-typed TableRow subclasses for each table
Summary:
Continuing to march toward low-overhead, type-safe table rows, this commit
introduces the code generation for said rows. Nothing uses it yet; see the
next commit for that.

(Adapted from https://github.com/facebook/osquery/pull/5199)

Reviewed By: guliashvili

Differential Revision: D13438017

fbshipit-source-id: 959a6e092aee38d33e1c6539cbe14b85172c0135
2019-01-09 13:50:15 -08:00
Jonathan Keljo
1870fd86d8 Introduce TableRow interface
Summary:
Continuing to march toward low-overhead, type-safe table rows, this commit
changes `TableRow` to be an interface rather than simply an alias for `Row`.
Accordingly, `DynamicTableRow` becomes an implementation of that interface
backed by a `Row`. The few remaining pieces of code that treated `TableRow`s as
`Row`s now call methods on the `TableRow` interface. Subsequent commits will
add code generation for strongly-typed table-specific implementations of
`TableRow`.

(Adapted from https://github.com/facebook/osquery/pull/5198)

Reviewed By: guliashvili

Differential Revision: D13438015

fbshipit-source-id: 61d5547e878e519c9706f94f844aab9d3e553410
2019-01-09 13:50:15 -08:00
Jonathan Keljo
78a6960dcd Make TableRows more abstract
Summary:
Continuing to march toward low-overhead, type-safe table rows, this commit changes
much of the code that uses `TableRow`s to stop assuming that they're just `vector`s
by another name. (`TableRow` is on the way to becoming an interface with multiple
implementations.) They're now held in `unique_ptr`s (`TableRowHolder`). For cases
where we really want a `vector`-backed `TableRow` (mostly test code and extension
support), we have a factory function (`make_table_row`) and a helper class
(`DynamicTableRowHolder`) to make that smoother.

(Adapted from https://github.com/facebook/osquery/pull/5198)

Reviewed By: mkareta

Differential Revision: D13438016

fbshipit-source-id: 2de9ce46a64c0a067b5d3299c59bbe3ccacd4abe
2019-01-09 13:50:15 -08:00
Jonathan Keljo
b9f1e94fc8 Prefactor: QueryData -> TableRows for virtual tables
Summary:
Continuing to march toward low-overhead, type-safe table rows, this commit introduces
a distinction between rows being returned from a table (`TableRows`) and as the
result of a query (`QueryData`). Right now the two are simply aliases for each other;
that will change shortly.

(Adapted from https://github.com/facebook/osquery/pull/5198)

Reviewed By: guliashvili

Differential Revision: D13438019

fbshipit-source-id: 6563fc8c372d9d6c4b05705943ddf39b42260feb
2019-01-09 13:50:15 -08:00
Jeremy Calvert
5461a3b79d Get typed data from sqlite
Summary:
Step towards completion of https://github.com/facebook/osquery/pull/4904
- Add boost variant (typed) counterparts for RowData, Row, and QueryData.
- Add a queryInternal to sqlite_util that returns QueryDataTyped.
- Replace the queryInternal that returns QueryData with one that's a simple wrapper around the new typed implementation.

Reviewed By: guliashvili

Differential Revision: D13586036

fbshipit-source-id: d1d9b56470fbfcfb5802de422f87d9b9d6da7fb7
2019-01-09 09:06:57 -08:00
George Guliashvili
75dade56b3 fix null dereference (#5332)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5332

malloc might return null and memcpy will crash in that case

Reviewed By: akindyakov

Differential Revision: D13517060

fbshipit-source-id: 261f493823974dcc40a40139094eb4a98cd24dec
2019-01-08 08:01:14 -08:00
Alexander Kindyakov
081d5053ee Create OSQUERY_NODISCARD to prevent non returning errors (#5331)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5331

Let's use attributes to prevent such mistakes (see previous diff in the stack)

Reviewed By: guliashvili

Differential Revision: D13504146

fbshipit-source-id: 7cab2dd345599b036fa2a27bf682f2a159fd1c08
2018-12-19 03:58:50 -08:00
Alexander Kindyakov
a724147c22 created error should be returned (#5330)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5330

small bug in killswitch and rocksdb code

Reviewed By: guliashvili

Differential Revision: D13504145

fbshipit-source-id: be9ae605a8c5588c5613889eb9a1af408935bfc8
2018-12-19 03:58:50 -08:00
George Guliashvili
ae2486ff16 fix xcode duplicate target names (#5323)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5323

xcode needs unique target names not to fail on buck generated project

Reviewed By: marekcirkos, akindyakov

Differential Revision: D13449869

fbshipit-source-id: 73511aeaa3245586adc293fbe0311f51f71ea12f
2018-12-13 16:13:44 -08:00
George Guliashvili
4967cf3de1 osquery windows oss build fix
Summary: osquery windows oss build fix

Reviewed By: fmanco

Differential Revision: D13431552

fbshipit-source-id: c13f1edac9c08d49901c5db3f58fc5c558ad8410
2018-12-13 05:21:47 -08:00
Jonathan Keljo
a88a010e30 Plumb columns used as a bitfield
Summary:
Hand port of https://github.com/facebook/osquery/pull/5154 to the new build system

I'm moving toward generating constants for each column so that we
can do used column lookups without having to do a string set lookup,
but first I need to have the used columns information plumbed through
as a bitfield.

Once the code generation is in and all cases have been migrated to it,
I'll remove the string variants of `isColumnUsed`/`isAnyColumnUsed`.

Reviewed By: guliashvili

Differential Revision: D13423615

fbshipit-source-id: 6a2afe7dad42942610dfe0f6f55bcee4603a25af
2018-12-12 10:40:58 -08:00
Jonathan Keljo
5f81138eaf Four new "add" wrapper JSON methods
Summary:
Manual port of https://github.com/facebook/osquery/pull/5153 to the new build system.

New methods to allow adding larger numeric types.

Reviewed By: guliashvili

Differential Revision: D13422612

fbshipit-source-id: 6b503553f05139351f803ea6fcc5c825e62c35ea
2018-12-12 10:40:58 -08:00
Jonathan Keljo
898ed37dfb Table for OSX Running and Active Applications
Summary:
This is a manual port of https://github.com/facebook/osquery/pull/5216 to the new build system.

This table allows you to fetch a list of running applications on OSX. Contains PID, Bundle ID and whether or not the application is in focus. (The latter can be used to figure out what application was being used at a given moment.)

Reviewed By: guliashvili

Differential Revision: D13422150

fbshipit-source-id: 236b28d9140a9a9062fb913815d0c9f1da21c8b5
2018-12-12 09:17:19 -08:00
Filipe Manco
22da13d863 Remove unused and most likely broken operator== overload from error (#5318)
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5318

This is most likely broken and is causing problems with some toolchains (e.g. clang on Ubuntu xenial).

Reviewed By: guliashvili

Differential Revision: D13415457

fbshipit-source-id: b4aa686913d3cfe6bb26089b0525144fae7cc928
2018-12-11 16:10:38 -08:00
George Guliashvili
c80bd28e5f inotify_tests buckify
Summary: inotify_tests buckify

Reviewed By: fmanco

Differential Revision: D13399806

fbshipit-source-id: 2ea33d61c9a3f48cf3914ed2f8e60b5012e5808c
2018-12-11 08:29:15 -08:00
George Guliashvili
059645a078 buckify fsevents test
Summary: buckify fsevents test

Reviewed By: akindyakov

Differential Revision: D13399058

fbshipit-source-id: d8ee81aa203cc8ffc30cf53922e4a76426604c4b
2018-12-11 08:29:15 -08:00
Filipe Manco
6ebabe6ce0 Add missing includes to posix/filepath.cpp
Summary: These missing includes make the build fail with some toolchains.

Reviewed By: guliashvili

Differential Revision: D13414926

fbshipit-source-id: b9a7992e37751da81ec1f5a9055703bc1e3cc4bc
2018-12-11 05:00:18 -08:00
Alexander Kindyakov
3b992f03bd events_tests should not depend on yara utils
Reviewed By: guliashvili

Differential Revision: D13398243

fbshipit-source-id: 1cb956df89843eb647dd0ba65f8ff46e73faaf5b
2018-12-11 03:31:18 -08:00
George Guliashvili
3775b82cdb auditprocess_file_events_tests buckify
Summary: auditprocess_file_events_tests buckify

Reviewed By: fmanco

Differential Revision: D13377453

fbshipit-source-id: 819d56b42eb55aefcb403003f0ad6fc46b481bf0
2018-12-11 03:31:17 -08:00
George Guliashvili
4704892aca fix operator=
Summary: operator= should be returning value.

Reviewed By: fmanco

Differential Revision: D13377348

fbshipit-source-id: e3a5e0adf9a79bf6e76795423c9b88cd425c02f3
2018-12-11 03:31:17 -08:00
George Guliashvili
e28335998a events_database_tests buckify
Summary: events_database_tests buckify

Reviewed By: akindyakov

Differential Revision: D13377334

fbshipit-source-id: c347f05e66af2c0777857352b591b8dad83e7317
2018-12-11 03:31:17 -08:00
Alexander Kindyakov
3a0683f049 Switch off osquery/remote tests for windows
Summary: Switch off osquery/remote tests for windows, it takes time to fix it. I'll do it later.

Reviewed By: fmanco

Differential Revision: D13378357

fbshipit-source-id: 21077630864fc4a27ba65078c634e115875f3970
2018-12-11 03:31:17 -08:00
Alexander Kindyakov
dcf27a5956 Fix up config/tests:test_config_update for windows
Summary: It turns out we already fixed that old issue

Reviewed By: guliashvili

Differential Revision: D13376876

fbshipit-source-id: a7822a1c79aa180e40f7ce54faa7c811b0d1b24b
2018-12-11 03:31:17 -08:00
Alexander Kindyakov
f60d2100de fix up extenstions tests on windows
Summary: Windows named pipe requires a bit different name format in compare to posix platforms.

Reviewed By: guliashvili

Differential Revision: D13376805

fbshipit-source-id: 443c8f963863adbacd1edb76945919f00a1a2e4e
2018-12-11 03:31:17 -08:00
George Guliashvili
971bee4441 Move build system to BUCK
fbshipit-source-id: 8ffef5e6a393ac67ce56dcb74845402e43d964a0
2018-12-07 16:12:35 +00:00
Alexander Kindyakov
29df1efe00 CarbonBlack table have to assume any key in sensor settings file could be missing
Fix up failing tests with exception 'No such node (CB.SensorBackendServer)' thrown in the test body
2018-12-07 16:02:09 +00:00
Alexander Kindyakov
e3037331d4 Fix up bug in NonNegativeInt and NonNegativeOrErrorInt table column data validator 2018-12-07 16:01:38 +00:00
George Guliashvili
a31d7582f4 Fix rocksdb crash
Configuration of OptimizeForSmallDb was crashing osquery. To be more specific ColumnFamilyOptions::OptimizeForSmallDb part was doing it.
2018-12-07 16:00:46 +00:00
George Guliashvili
a32ed7c45d Fix random port problem
random port was not really random. Seed was never set so it was generating the same port.
2018-12-07 16:00:46 +00:00
Marek Cirkos
6a64e353e9 Refactor battery table and return some information even if advanced information is missing 2018-12-07 16:00:46 +00:00
Filipe Manco
bf4c0a3117 Move iptables querying to a C based proxy
The iptables uapi kernel headers are incompatible with C++. To avoid
depending on modified kernel headers query iptables from a C based
proxy. This is a temporary solution until the problem is fixed on
upstream Linux.
2018-12-07 16:00:36 +00:00
packetzero
5188ce5288 update aws-sdk-cpp 1.4.55 on windows (#5255) 2018-10-29 21:24:29 +00:00
Nick Anderson
0314871908
bug: explicitly set safe permissions on osquery dbs (#5229) 2018-09-18 22:51:29 -07:00
Michael McGrew
e6302379fd Add per process performance data to windows processes table (#5224) 2018-09-18 20:59:35 -07:00
James Jerger
b8d7243aa9 Add InstallDate to os_version for Windows (#5226) 2018-09-14 16:07:37 -07:00
Giorgi Guliashvili
7bac8cb7f6
remove old boost bug supprot (#5221) 2018-09-14 23:51:44 +01:00
Nick Anderson
bf93fcf2f9
database: changing default path of the database for pathing uniformity (#4832) 2018-09-14 11:36:24 -07:00
James Jerger
e10b243ecf Add NTDomain Table for Windows (#5152) 2018-09-14 11:25:49 -07:00
Max Kareta
61d415c6bc
added database migration class (#5112) 2018-09-14 20:58:03 +03:00
Adrien Schildknecht
bdf504f6b6 tables: netmask should be an integer (#5217) 2018-09-14 16:13:37 +01:00
Alessandro Gario
52805dca44 chrome_extensions: Use the translation dictionary with case insensitive keys (#5215) 2018-09-14 16:06:31 +01:00
Giorgi Guliashvili
53a48d8fb4
posix profiling (#5187) 2018-09-14 16:04:28 +01:00
Giorgi Guliashvili
70806ae4fc
numeric_monitoring recursive sync record (#5204) 2018-09-14 12:17:24 +01:00
Alexander
dcd653ce33
Create helper functions to get CPU cores info on linux (#5209)
Helper functions to get CPU cores info exposed via sysfs on linux.
I need this information to use kprobes in osquery on linux.
2018-09-14 11:32:25 +01:00
Alexander
8ab08ed6c0
Prepare the filesystem directories in initialisation routine (#5207)
Create directory for the database only if database is switched on and do not fail on unsuccessful creation, just write down a message to log.
2018-09-14 11:19:07 +01:00
Alexander
9b3e14789b
Non throwing directory creation function. (#5206)
Directory creation function to osquery/filesystem.h

Status was used as a return value, as far as all function in this file operate with Status not Expected. Let's move from Status to Expected in another PR.
2018-09-12 11:48:11 +01:00