Commit Graph

786 Commits

Author SHA1 Message Date
Teddy Reed
5aa225d4c3 Merge pull request #1619 from sharvilshah/wifi
Implement wifi_networks tables for OS X
2015-11-02 16:11:21 -08:00
Teddy Reed
15215cdbc0 Add persistent splays 2015-11-02 14:10:04 -08:00
Teddy Reed
5233d7dcf8 Add start time to osquery_info, remove md5/path 2015-11-02 10:57:01 -08:00
Teddy Reed
75bfcddc31 Merge pull request #1622 from theopolis/faster_sockets
Faster socket_events on Linux
2015-11-02 10:56:37 -08:00
Teddy Reed
a1a9131174 Optimize socket_events and Linux users 2015-11-02 10:37:56 -08:00
Teddy Reed
50550e607a Build and provision edits for FreeBSD CI 2015-11-02 01:47:09 -08:00
Sharvil Shah
9a6d6d1293 Implement wifi_networks tables for OS X
If the option of remembering known Wi-Fi networks is enabled on a system,
they are persisted to disk as a preferences property list file.
This table is populated by parsing that file.
2015-11-01 16:53:51 -08:00
Teddy Reed
d27a7ecc4c Fix clang warnings, promote warnings to errors 2015-11-01 02:12:07 -08:00
Teddy Reed
97a6521445 Merge pull request #1614 from theopolis/drop_around_files
Expand the scope of permissions dropping
2015-10-30 17:07:04 -07:00
Teddy Reed
09e4e3c42e Expand the scope of permissions dropping 2015-10-30 09:56:33 -07:00
Michael George
fb545bb85e added sh_history 2015-10-29 10:53:04 -07:00
Teddy Reed
db58ec5e44 Only emit process events for 0-status execve 2015-10-27 17:12:23 -07:00
Teddy Reed
b81b6de6ae This refactors a bit of config/packs and adds a socket_events table to Linux.
The refactor of config/packs was initiated because event subscribers needed
a method for toggling `::init` based on some configurable option. In the case
of auditd, turning on the support with `--disable_audit=false` used to start
auditing the EXECVE syscall. It was understandable that this would cause
latency based on the number of processes executing per measure of time.

A new `socket_events` table will do the same but for `bind` and `connect`. These
are less-obvious and for now, require a scan of /proc for socket tuples. In the
future this file descriptor to socket tuple will be faster.
2015-10-27 15:13:02 -07:00
Teddy Reed
654830cf11 Merge pull request #1594 from rcseacord/additional-sign-fixes
eliminated some warnings from Clang 3.7 analyze mode
2015-10-23 13:03:54 -03:00
Robert C. Seacord
09481d0381 Fixed some type problems, casting away const, integer types, old style casts, etc. 2015-10-21 20:56:58 +00:00
Robert C. Seacord
1d9695ac31 eliminated some warnings from Clang 3.7 analyze mode 2015-10-21 06:02:58 +00:00
Teddy Reed
7ba87a88bb Merge pull request #1585 from rcseacord/additional-sign-fixes
Additional sign fixes
2015-10-19 11:25:18 -07:00
Teddy Reed
8214dd1309 Merge pull request #1584 from theopolis/fix_1580
[Fix #1580] Handle exceptions in linux process_memory_map
2015-10-19 09:28:16 -07:00
Teddy Reed
f891503cd9 Merge pull request #1577 from nemith/dpkg
Support for newer versions of libdpkg
2015-10-19 09:24:37 -07:00
Teddy Reed
bc50c053fb Remove boolean type-columns from file in favor of 'type' 2015-10-17 12:16:54 -07:00
Robert C. Seacord
e57828aac3 changes for integer sign problems 2015-10-17 00:18:35 +00:00
Teddy Reed
3cc7984cc2 [Fix #1580] Handle exceptions in linux process_memory_map 2015-10-16 16:59:23 -07:00
Robert C. Seacord
acb2f6f628 eliminating diagnostics, mostly for comparisons between signed and unsigned operations 2015-10-16 16:10:37 +00:00
Robert C. Seacord
37b8e83a9e fixes for problems related to unsigned to signed integer comparisons 2015-10-16 16:10:36 +00:00
Teddy Reed
3f8cb14fbb Merge pull request #1579 from nemith/segv
Fix segfault on interfaces tables
2015-10-15 17:58:04 -07:00
Brandon Bennett
f683871653 Fix segfault on interfaces tables
getifaddrs(3) states that ifa_addr can be null. Check to make sure they are not null before accessing them
2015-10-15 16:53:14 -06:00
Brandon Bennett
65738a73c1 Support for newer versions of libdpkg
Libdpkg has some breaking changes in newer versions which prevented
compiling the deb_packages table on Ubuntu 15.04.  This change looks for
the libpkg version user pkg-config and adds some preprocessor magic to
support the newer versions.
2015-10-15 16:43:14 -06:00
Teddy Reed
3be0994933 [Fix #1570] Check for invalid apt sources
This fixes a crash identified by @endrazine.
When apt sources data in /etc/apt/sources.list or /etc/apt/sources.list.d/{*}.list contain invalid data/lines the cache_file.GetPkgCache(); call will fail and cache will be nullptr. Subsequent usage results in a SIGSEV.

To reproduce the fault try:

$ zzuf -I /etc/ -r 0.01:0.1 -s 0:1000 -v \
 ./build/trusty/osquery/osqueryi --registry_exceptions=true --verbose \
 "select count(*) from apt_sources"

Signed-off-by: Jonathan Brossard
2015-10-15 15:20:26 -07:00
Teddy Reed
34717fda29 Merge pull request #1554 from mlw/fix-lingering-socket-fds
Close socket fds when finished with them
2015-10-12 13:32:52 -07:00
Matthew White
2446b22a5f Close socket fds when finished with them 2015-10-12 09:59:09 -07:00
Teddy Reed
6b16720039 Fix kernel_info on OS X, remove md5 2015-10-11 11:43:42 -07:00
Teddy Reed
31b7966088 [Fix #1531] Use libarchive finish for safari_extension parsing 2015-09-28 17:33:42 -07:00
Teddy Reed
bbac2cf07f [#1529] Allow DB Readonly with RocksDB lite 2015-09-28 01:50:32 -07:00
Teddy Reed
64c18a70a9 Merge pull request #1525 from theopolis/process_adds
Add state, group, and nice to processes
2015-09-24 14:43:17 -07:00
Teddy Reed
5890901c00 Add state, group, and nice to processes 2015-09-24 13:11:46 -07:00
Mike Arpaia
327a9bcdb1 Merge pull request #1522 from marpaia/startup_items
Include system startup items
2015-09-22 16:06:20 -07:00
Mike Arpaia
b09031adda Include system startup items
We were not parsing system startup items.
2015-09-22 15:50:55 -07:00
Mike Arpaia
4021a742df Merge pull request #1507 from jacknagz/os_version_rhel
RHEL os_version fix
2015-09-21 18:03:03 -07:00
Teddy Reed
d042967f43 Fix YARA sigfile caching 2015-09-20 00:06:57 -07:00
Jack Naglieri
9c1e114728 Fix os_version table regex for REDHAT_BASED systems. Updating centos6/7 and freebsd10 Vagrant boxes. 2015-09-18 14:47:08 -07:00
Teddy Reed
333f2ce8c8 [#1506] Silent kext loading messages from syslog 2015-09-16 13:13:56 -07:00
Teddy Reed
65162e7239 Merge pull request #1501 from sharvilshah/sysinfo_updates
Update system_info table to include CPU type, CPU cores and total memory
2015-09-14 20:02:56 -04:00
Teddy Reed
944e3de206 Merge pull request #1496 from theopolis/events_table
[#1487] Add osquery_events table to track pubsub stats
2015-09-14 15:27:35 -04:00
Sharvil Shah
28143f64f0 Update system_info table: adds CPU type, CPU cores and total memory.
This change adds following columns to `system_info` table:

    cpu_type, cpu_subtype, cpu_brand, cpu_physical_cores,
    cpu_logical_cores, physical_memory, hardware_model

Here's an example output of those columns:

```
              cpu_type = x86_64h
           cpu_subtype = Intel x86-64h Haswell
             cpu_brand = Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
    cpu_physical_cores = 4
     cpu_logical_cores = 8
       physical_memory = 17179869184
        hardware_model = MacBookPro11,3
```
2015-09-10 14:44:48 -07:00
Matthew White
28d456a2f1 Fix build for Ubuntu Lucid 2015-09-10 13:55:59 -04:00
Scott Piper
5e7d0d6a37 Added system_info table 2015-09-09 10:26:16 -07:00
Mike Arpaia
de58353131 Config MD5 a bit more deterministic
```
$ ./build/darwin/osquery/osqueryi --config_path=/asdfasdfadfs
E0903 11:45:02.050308 1990836992 init.cpp:370] Error reading config: config file does not exist
Using a virtual database. Need help, type '.help'
osquery> .mode line
osquery> .all osquery_info
           pid = 33700
       version = 1.5.2-43-gb06fa92
    config_md5 =
  config_valid = 0
   config_path = /asdfasdfadfs
    extensions = active
build_platform = darwin
  build_distro = 10.10
osquery> .exit

$ ./build/darwin/osquery/osqueryi
osquery> .mode line
osquery> .all osquery_info
           pid = 33781
       version = 1.5.2-43-gb06fa92
    config_md5 = 8a432ac93d3de080c62d77ba99b89783
  config_valid = 1
   config_path = /var/osquery/osquery.conf
    extensions = active
build_platform = darwin
  build_distro = 10.10
osquery> .exit
```
2015-09-03 22:03:40 -07:00
Teddy Reed
b57040db60 Add osquery_events table to track pubsub stats 2015-09-03 15:10:53 -07:00
Teddy Reed
2813d3ab87 Add a Linux audit event publisher 2015-09-03 08:45:02 -07:00
Teddy Reed
ba7cef3f78 Merge pull request #1493 from theopolis/fix_1492
[Fix #1492] Fix firefox key counting and spec typo
2015-09-02 23:49:55 -07:00
Teddy Reed
7a15d25796 [Fix #1492] Fix firefox key counting and spec typo 2015-09-02 19:50:36 -07:00
Teddy Reed
bb2b5f594b Static analysis cleanups, static libmagic 2015-09-02 16:55:20 -07:00
Mike Arpaia
a140333441 [fix #1390] query pack re-org
This commit contains the features specified in #1390 as well as a
refactoring of the general osquery configuration code.

The API for the config plugins hasn't changed, although now there's a
`genPack` method that config plugins can implement. If a plugin doesn't
implement `genPack`, then the map<string, string> format cannot be used.
The default config plugin, the filesystem plugin, now implements
`genPack`, so existing query packs code will continue to work as it
always has.

Now many other config plugins can implement custom pack handling for
what makes sense in their context. `genPacks` is not a pure virtual, so
it doesn't have to be implemented in your plugin if you don't want to
use it. Also, more importantly, all config plugins can use the standard
inline pack format if they want to use query packs. Which is awesome.

For more information, refer to #1390, the documentation and the doxygen
comments included with this pull requests, as well as the following
example config which is now supported, regardless of what config plugin
you're using:

```json
{
  "options": {
    "enable_monitor": "true"
  },
  "packs": {
    "core_os_monitoring": {
        "version": "1.4.5",
        "discovery": [
          "select pid from processes where name like '%osqueryd%';"
        ],
        "queries": {
          "kernel_modules": {
              "query": "SELECT name, size FROM kernel_modules;",
              "interval": 600
          },
          "system_controls": {
              "query": "SELECT * FROM system_controls;",
              "interval": 600,
              "snapshot": true,
          },
          "usb_devices": {
              "query": "SELECT * FROM usb_devices;",
              "interval": 600
          }
        }
    },
    "osquery_internal_info": {
        "version": "1.4.5",
        "discovery": [
          "select pid from processes where name like '%osqueryd%';"
        ],
        "queries": {
          "info": {
              "query": "select i.*, p.resident_size, p.user_time, p.system_time, time.minutes as counter from osquery_info i, processes p, time where p.pid = i.pid;",
              "interval": 60,
              "snapshot": true
          },
          "registry": {
              "query": "SELECT * FROM osquery_registry;",
              "interval": 600,
              "snapshot": true
          },
          "schedule": {
              "query": "select name, interval, executions, output_size, wall_time, (user_time/executions) as avg_user_time, (system_time/executions) as avg_system_time, average_memory from osquery_schedule;",
              "interval": 60,
              "snapshot": true
          }
        }
    }
  }
}
```

The `osquery_packs` table was modified to remove the superfluous
columns which could already have been found in `osquery_schedule`. Two
more columns were added in their place, representing stats about pack's
discovery query execution history.

Notably, the internal API for the `osquery::Config` class has changed
rather dramatically as apart of the refactoring. We think this is an
improvement. While strictly adhering to the osquery config plugin
interface will have avoided any compatibility errors, advanced users may
notice compilation errors if they access config data directly. All
internal users of the config have obviously been updated. Yet another
reason to merge your code into mainline; we update it for you when we
refactor!
2015-09-02 10:56:26 -07:00
Teddy Reed
776de9c4d1 Merge pull request #1477 from theopolis/table_xp_meta
XProtect meta virtual table and safari_extensions column additions
2015-08-30 21:31:35 -07:00
Teddy Reed
906d19927f [#1418] Use libarchive to parse Safari extension bundles 2015-08-29 23:59:41 -07:00
Javier Marcos
74be3d1da0 Removing dots at the end of log entries 2015-08-28 16:50:44 -07:00
Javier Marcos
086ab40f83 Merge pull request #1473 from javuto/this_is_real_magic
Adding magic table to check for libmagic data
2015-08-28 14:03:10 -07:00
Javier Marcos
1a50977a23 Adding magic table to check for libmagic data 2015-08-28 12:49:46 -07:00
Teddy Reed
88c7ad35a2 Merge pull request #1471 from theopolis/process_start_fix
[Fix #1453] Use second precision for process start times
2015-08-28 11:48:25 -07:00
Teddy Reed
2433d9e06c [#1418] Include XProtect's meta list of plugin versions, and blacklisted extensions 2015-08-28 11:46:21 -07:00
Teddy Reed
014e504fba [Fix #1432] Improve OS X USB device reporting 2015-08-27 16:36:54 -07:00
Teddy Reed
3c114c3439 [Fix #1453] Use second precision for process start times 2015-08-27 15:47:06 -07:00
Mathieu Kooiman
b151ecedc2 Refs https://github.com/facebook/osquery/issues/320
Add provisioning scripts to build osquery on Debian Wheezy and Debian Jessie.
2015-08-20 20:57:22 +02:00
Teddy Reed
bdadc9753b Additional OS X table performance improvements 2015-08-18 01:35:10 -07:00
Teddy Reed
5bf30a779d RocksDB usage speedups 2015-08-15 20:43:53 -07:00
Teddy Reed
43cf5f1a0a Merge pull request #1448 from theopolis/strol-speedup
Speedup type conversions, yara, and 10.10 symbols at runtime
2015-08-14 11:01:46 -07:00
Teddy Reed
68d7a6e0be Speedup type conversions, yara, and 10.10 symbols at runtime 2015-08-13 18:04:03 -07:00
Teddy Reed
634dfe7da1 Merge pull request #1438 from sharvilshah/fix_homebrew_version
[Fix #1434] version reporting for homewbrew_packages
2015-08-12 11:30:21 -07:00
Sharvil Shah
b190f5f99a Fix #1433, os_version reporting for 10.11 2015-08-11 14:03:27 -07:00
Sharvil Shah
369040e69b Fix version reporting for homewbrew_packages. Fixes #1434 2015-08-11 01:50:40 -07:00
Sharvil Shah
64588be88b Fix build on OS X 10.11
enum `SecItemClass` changed in 10.11 headers,
so don't instantiate with rvalue of int.

Update `SecKeychainSearchCreateFromAttributes` to match the stricter definition.

Fixes #1423
2015-08-05 18:29:29 -07:00
Michael O'Farrell
5d0e4be6a1 Merge pull request #1335 from mofarrell/kernel-file-events
Added kernel file access events.
2015-07-31 15:22:11 -07:00
Michael O'Farrell
9f2b318778 Added kernel file access events. 2015-07-31 15:06:46 -07:00
Mike Arpaia
a45c794f52 building on 10.9 2015-07-31 11:57:39 -07:00
osquery
ae8305e00e Revert "Remove OS X 10.9 code path since we no longer support it"
This reverts commit 05bbe2ce06.
2015-07-31 11:44:34 -07:00
Michael O'Farrell
b0289adcf5 Merge pull request #1414 from theopolis/env_limits
Add optional environment variable whitelist to process_events
2015-07-30 18:17:31 -07:00
Teddy Reed
dc82ffa636 Add optional environment variable whitelist to process_events 2015-07-30 16:05:11 -07:00
Chris Down
260df0d6d0 linux users table: Do not drop users with duplicate UIDs
See Github issue #1301. FreeBSD (which also uses this table) by default has two
users which are UID 0 -- both `toor` and `root`. 19a2d64959 made it so that we
would only get the first one from `getpwent`, but this feature is undesirable
in cases where two different users share the same UID.
2015-07-29 09:00:47 -07:00
Michael O'Farrell
93a65eaf04 Merge pull request #1400 from mofarrell/process-events-env-arg
Adding environment variables and arguments for process events.
2015-07-27 17:54:06 -07:00
Michael O'Farrell
3f87d5832f Adding environment variables and arguments for process events. 2015-07-27 15:48:47 -07:00
Wesley Shields
698e226b80 Add tags and strings columns to YARA tables.
When strings match they will be populated into the "strings" column of
the table. The format is identifier:offset.

When a matching rule has tags defined the tags will be put into the
"tags" column of the table in a comma separated list.
2015-07-27 08:20:24 -04:00
Teddy Reed
d2effc539c [Fix #1374] Allow subscription subclassing 2015-07-26 01:48:27 -07:00
Teddy Reed
cce8a6aab3 Merge pull request #1384 from theopolis/table_cleanups
Remove some non-warning/error log lines from tables
2015-07-24 00:32:11 -07:00
Teddy Reed
2d7ce9341a Remove some non-warning/error log lines from tables 2015-07-24 00:09:06 -07:00
Teddy Reed
ad94eaf0b8 [Fix #1367] Disable user-controlled FIFO reads 2015-07-22 10:15:39 -07:00
Teddy Reed
fc24682816 Fix profile platform bug in leaks checking 2015-07-20 02:06:52 -07:00
Teddy Reed
e8cb919f03 Merge pull request #1364 from theopolis/harden_applications
[Fix #1357] Use OS X LS API for app listing
2015-07-20 01:14:07 -07:00
Mike Arpaia
5ccfe886ba Merge pull request #1363 from theopolis/less_rows
[Fix #1303] Only emit rows when appropriate for processes/users.
2015-07-19 20:36:26 -07:00
Teddy Reed
dd7990b719 [Fix #1357] Use OS X LS API for app listing
Attempt to use OS X's LaunchServices to get a list of applications.
Fall back to basic directory traversal of well-known application paths.
2015-07-19 20:22:48 -07:00
Teddy Reed
5249e74146 [Fix #1303] Only emit rows when appropriate for processes/users.
When optimizing a table using query constraints an implementation should not add unneeded rows.
A user experience bug exists when selecting with an explicit non-existing pid/uid.
2015-07-19 20:20:04 -07:00
Teddy Reed
bcdbb40f0c [Fix #1356] Tokenize process environ by '\0' on Linux 2015-07-19 14:34:49 -07:00
Teddy Reed
6104aaebfe Add optional TLS config plugin refresh 2015-07-17 14:59:08 -07:00
Teddy Reed
c36fbda274 Merge pull request #1349 from theopolis/centos_version
[Fix #1319] CentOS version reporting and file read error
2015-07-17 09:07:29 -07:00
Teddy Reed
f06820f578 [Fix #1319] CentOS version reporting and file read error
1. Redhat-based distributions were not reporting their version correct.
2. The file read API assumed stat would return an accurate file size.
This has been replaced with an attempt to seek to the end of the file.
2015-07-16 14:16:51 -07:00
Mike Arpaia
9eeb224ce7 clang-format authorizations files 2015-07-16 11:09:16 -07:00
Tom Burgin
e8d3e45cea Added authorization_mechanisms and authorizations tables 2015-07-15 14:25:19 -04:00
Teddy Reed
c269bbeaf3 Rollup of build changes 2015-07-14 13:45:53 -07:00
Michael O'Farrell
276891ad00 Merge pull request #1330 from mofarrell/kernel
Kernel!!!
2015-07-13 17:29:08 -07:00
Michael O'Farrell
58ec6415d3 Created a basic publisher system for kernel events in the kernel extension. 2015-07-13 16:42:55 -07:00
Teddy Reed
3bd6b64b8b Silence OS X OpenSSL-related deprecations 2015-07-13 10:14:47 -07:00