[Added] python packages which are not installable via pip on lucid
[Moved] installing latest bison and openssl right before thrift building for lucid
[Fixed] package bison installation for not lucid
[Added] OpenSSL dependency for lucid
[Changed] prefix to /usr/local
[Fixed] wrong file for checking if dependency is intalled
1. Table implementations (spec files) can mark the table as 'cachable'.
2. Cached results depend on the shortest/quickest interval of scheduled
queries that act on results of the table.
3. The table API generator blocks caching on index/additional/required
table column options.
If the option of remembering known Wi-Fi networks is enabled on a system,
they are persisted to disk as a preferences property list file.
This table is populated by parsing that file.
There was a bug in the `osquery::Schedule` container object such that,
when the iteration through the schedule occured, pack objects were being
passed by value (copied) instead of passed by reference. Thus, the
discovery query would be executed, the object's cache would be updated,
and then the object would go out of scope and be destructed, thus
leaving the original object without ever having ran the discovery query.
This caused discovery queries to thrash. Bad times.
I added a new test so that we don't regress here as well as const'd a
few functions that should have been const in `osquery::Pack`.
This commit contains the features specified in #1390 as well as a
refactoring of the general osquery configuration code.
The API for the config plugins hasn't changed, although now there's a
`genPack` method that config plugins can implement. If a plugin doesn't
implement `genPack`, then the map<string, string> format cannot be used.
The default config plugin, the filesystem plugin, now implements
`genPack`, so existing query packs code will continue to work as it
always has.
Now many other config plugins can implement custom pack handling for
what makes sense in their context. `genPacks` is not a pure virtual, so
it doesn't have to be implemented in your plugin if you don't want to
use it. Also, more importantly, all config plugins can use the standard
inline pack format if they want to use query packs. Which is awesome.
For more information, refer to #1390, the documentation and the doxygen
comments included with this pull requests, as well as the following
example config which is now supported, regardless of what config plugin
you're using:
```json
{
"options": {
"enable_monitor": "true"
},
"packs": {
"core_os_monitoring": {
"version": "1.4.5",
"discovery": [
"select pid from processes where name like '%osqueryd%';"
],
"queries": {
"kernel_modules": {
"query": "SELECT name, size FROM kernel_modules;",
"interval": 600
},
"system_controls": {
"query": "SELECT * FROM system_controls;",
"interval": 600,
"snapshot": true,
},
"usb_devices": {
"query": "SELECT * FROM usb_devices;",
"interval": 600
}
}
},
"osquery_internal_info": {
"version": "1.4.5",
"discovery": [
"select pid from processes where name like '%osqueryd%';"
],
"queries": {
"info": {
"query": "select i.*, p.resident_size, p.user_time, p.system_time, time.minutes as counter from osquery_info i, processes p, time where p.pid = i.pid;",
"interval": 60,
"snapshot": true
},
"registry": {
"query": "SELECT * FROM osquery_registry;",
"interval": 600,
"snapshot": true
},
"schedule": {
"query": "select name, interval, executions, output_size, wall_time, (user_time/executions) as avg_user_time, (system_time/executions) as avg_system_time, average_memory from osquery_schedule;",
"interval": 60,
"snapshot": true
}
}
}
}
}
```
The `osquery_packs` table was modified to remove the superfluous
columns which could already have been found in `osquery_schedule`. Two
more columns were added in their place, representing stats about pack's
discovery query execution history.
Notably, the internal API for the `osquery::Config` class has changed
rather dramatically as apart of the refactoring. We think this is an
improvement. While strictly adhering to the osquery config plugin
interface will have avoided any compatibility errors, advanced users may
notice compilation errors if they access config data directly. All
internal users of the config have obviously been updated. Yet another
reason to merge your code into mainline; we update it for you when we
refactor!
* It wasn't straightforward to get OpenSSL building
without avx/vxoprs optimizations on 10.10
* libressl is essentially a modern/lean-ish drop-in replacement for OpenSSL
and can build without avx optimizations to support older Macs
This change:
* Installs libressl (builds a bottle) using homebrew
* And statically links `libcrypto.a` and `libssl.a` unless
`BUILD_LINK_SHARED` is requested.
Fixes#1329
OS X 10.9 should not build/test a kernel extension yet. The MAC policy framework is slightly different and the APIs/version dependencies need to be tested.
Having `-mtune=i386` is causing compilation failure for gflags on ubuntu.
This change removes the `mtune` compile flag.
`-march` flag is already set to `x86-64` and according to gcc doc,
Specifying `-march=cpu-type` implies `-mtune=cpu-type.`
Fixes#1428
Boost property trees are level delimited using '.' characters.
An Apple property list may contain keys with '.' characters, so the plist conversion must use iterators and raw node appends.
POSIX-globbing will allow event publishers/subscribers to post-check
results against glob-syntax, fnpath matching, and POSIX C-regex.
These checks are anecdotally speedy.
We found that not installing the headers for snappy caused RocksDB's
snappy detection to not find that snappy was installed:
https://goo.gl/YOWJl0
The snippet there requires that the headers are installed, not just the
library. By installing the headers, we can ensure that snappy is linked.
OR, alternatively, we could just leave it and not link snappy. It's
uncertain what the specific benefits of including snappy are for our
use-case. (CC @igorcanadi)
1. Example queries will run with an (optional) integration test.
2. Fix bad accesses with OS X package BOMs
3. Move spec files from ./osquery/tables/specs to ./specs
4. Remove server parsers (netlib) from client builds.
`make tests` fails with another osquery process running.
The backing-store check happens after a config plugin is setUp and
the initial load occures. This may involve calls to cached keys, the
check should occur pre-config initialize.
1. added docs for vagrant-aws support in Vagrantfile
2. removed aws target that have local vagrant support. inline-string'd aws.user_data
3. support building rhel6/7 in aws
4. correct aws-rhel6.6 name. it should be rhel6.5
Prior to this commit, you'd get this error when running `make deps` with a version of autoconf that belongs in a museum (e.g. the one that's in CentOS 6 Base):
```
[+] autoconf is already installed. skipping provision.
/home/brandt/osquery/tools/provision/lib.sh: line 163: force_provision: command not found
make: *** [deps] Error 127
```
I couldn't find a `force_provision` in the codebase. Plain old `provision` should do the job.
Looks like a simple typo: `automake` for `libtool`
Before this commit, this is the error you'd get when you ran `make deps`:
```
[+] libtool is not installed/provisioned. installing...
[+] libtool has not been downloaded. downloading...
--2015-05-11 06:01:38-- https://osquery-packages.s3.amazonaws.com/deps/automake-2.4.5.tar.gz
Resolving osquery-packages.s3.amazonaws.com... failed: Temporary failure in name resolution.
wget: unable to resolve host address “osquery-packages.s3.amazonaws.com”
[+] libtool has not been extracted. extracting...
tar (child): libtool-2.4.5.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
/vagrant/tools/provision/lib.sh: line 196: pushd: libtool-2.4.5: No such file or directory
make: *** [deps] Error 1
```
The extensions tests were previously using the same extensions socket
path. This may lead to races during the tests. There might be more
stability work needed to relax time/speed assumptions.
The RHN subscription manager has been failing on RHEL when enabling
repos. We can turn this fail into a warning and attempt to recover.
moved disk_ecryption table spec to crossplatform
link libcryptsetup
implemented get cipher type and cipher_mode:
more idiomatic c++11
no need to explicitly call std::string constructor to convert char * to std::string
update cryptsetup sources for centos
add function prototype for older libcryptsetup which is in centos6
ifdef check for centos6 which uses older libcryptsetup
remove forward declared functions defined in libcryptsetup, stylistic changes
Mostly minor documentation/wiki/guide fixes.
The breaks down the "using osqueryd" page into more of a summary
of what the daemon does from a schedule/logging perspective.
The bulk of the "using osqueryd" page now exists in the configuration
deployment page and the new "logging" deployment page.
There are a couple places where this was an issue.
The first place was in the filesystem plugin where it was only checked that it
existed, and not that it was an actual file.
The second was a lack of try and catch on the parse call in config.cpp.
Both of those issues are addressed in this diff.
