valitydev/osquery-1
14
0
Fork 0
mirror of https://github.com/valitydev/osquery-1.git synced 2024-11-06 09:35:20 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
121f7e2589
osquery-1/tests
History
puffyCid 121f7e2589
Add Shellbags table (#6949) 
Hello, this largish PR adds shellbags support to osquery.
Shellbags is a complex (imo) windows Registry artifact that primarily keeps track of directories a user has browsed to (specifically directories accessed using Windows Explorer).  By parsing shellbags its possible to recreate what directories a user accessed
Shellbags are composed of shellitems, this PR also adds support to parsing several shellitems, finally shellbags also contain FAT timestamps that show when a directory was created, modified, accessed, FAT timestamp parsing is also included in this PR

Example query of what shellbags looks like
```
osquery> select * from shellbags;
+-----------------------------------------------+--------------+-------------------------------------------------------------------------------------+---------------+--------------+---------------+-----------+--------------+
| sid                                           | source       | path                                                                                | modified_time | created_time | accessed_time | mft_entry | mft_sequence |
+-----------------------------------------------+--------------+-------------------------------------------------------------------------------------+---------------+--------------+---------------+-----------+--------------+
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob                                                                | 0             | 0            | 0             | 0         | 0            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Downloads                                                      | 1571635108    | 1571620406   | 1571635108    | 3074      | 5            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects                                                       | 0             | 0            | 0             | 0         | 0            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects\osquery                                               | 0             | 0            | 0             | 0         | 0            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects\osquery\build                                         | 0             | 0            | 0             | 0         | 0            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects\osquery\build\osquery                                 | 0             | 0            | 0             | 0         | 0            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects\osquery\build\osquery\RelWithDebInfo                  | 0             | 0            | 0             | 0         | 0            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects\osquery\osquery                                       | 1578192498    | 1571701478   | 1578192498    | 495902    | 4            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects\osquery\osquery\killswitch                            | 1578192406    | 1575859554   | 1578192406    | 707032    | 2            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects\clamav-osquery                                        | 1572045050    | 1572045050   | 1572045050    | 221518    | 14           |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects\clamav-osquery\extension_clamav                       | 1572045050    | 1572045050   | 1572045050    | 432733    | 8            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\Projects\clamav-osquery\extension_clamav\src                   | 1572045050    | 1572045050   | 1572045050    | 432736    | 11           |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\.osquery                                                       | 1571706262    | 1571706212   | 1571706262    | 575462    | 4            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\AppData                                                        | 1571623328    | 1571623318   | 1571623328    | 206482    | 7            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\AppData\Local                                                  | 1571701908    | 1571623318   | 1571701908    | 206502    | 8            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\AppData\Local\Microsoft                                        | 1593297370    | 1571623318   | 1593297370    | 206504    | 8            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\AppData\Local\Microsoft\Office                                 | 1593297370    | 1593295160   | 1593297370    | 52684     | 60           |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\AppData\Local\Microsoft\Office\16.0                            | 1593297388    | 1593295160   | 1593297388    | 81742     | 11           |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\AppData\Local\autopsy                                          | 1612935328    | 1612935328   | 1612935328    | 37041     | 7            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\AppData\Local\autopsy\Cache                                    | 1612935328    | 1612935328   | 1612935328    | 37104     | 8            |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\super secret sensitive stuff                                   | 1613198066    | 1613198066   | 1613198066    | 101729    | 123          |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\super secret sensitive stuff\secret data                       | 1613198092    | 1613198092   | 1613198092    | 383733    | 38           |
| S-1-5-21-1079689790-2336414676-942872339-1001 | usrclass.dat | This PC\C:\Users\bob\super secret sensitive stuff\secret data\dont look secret data | 1613198108    | 1613198108   | 1613198108    | 383736    | 15           |
+-----------------------------------------------+--------------+-------------------------------------------------------------------------------------+---------------+--------------+---------------+-----------+--------------+
```
Due to the complexity of shellbags currently this PR does not support (or only has partial support) for the following shellitems:
* optical disc
* variable (partial support)
* mtp (partial support)
* user property view data (partial support)

I was not able to generate shellbag data (or only some data) for the above shellitems in my windows vms (tested on two different Windows 10 systems, Windows 8.1, and Windows Server 2019), if osquery encounters any unknown shellbag data it will log a warning and mark the shellitem as "[UNKNOWN SHELL FORMAT]" when building directory paths.  
The main value of shellbags is reconstructing directories accessed as shown above, but this PR does include additional shellbag support such as FTP servers connected to via Windows Explorer, ZIP files opened, MTP devices (partial), and network shares browsed to via Windows Explorer
This PR is kind of large, let me know if there are any questions, suggestions for improvements, or issues, thanks!

Shellbags references:
[Shellitems](https://github.com/libyal/libfwsi/blob/main/documentation/Windows%20Shell%20Item%20format.asciidoc)
[Shellbags](https://www.magnetforensics.com/blog/forensic-analysis-of-windows-shellbags/)
[Property Stores](https://github.com/libyal/libfwps/blob/main/documentation/Windows%20Property%20Store%20format.asciidoc)
2021-02-26 15:47:37 -08:00
..
cppcheck Update cmake_minum_required to 3.17.5 and pin version in CI (#6770) 2020-11-30 15:02:19 +01:00
integration/tables Add Shellbags table (#6949) 2021-02-26 15:47:37 -08:00
CMakeLists.txt Reduce linkage requirements for tests (#6715) 2020-10-18 10:47:00 -04:00
test_util.cpp refactor: Move ephemeral database plugin into core and simplify tests (#6648) 2020-09-21 18:25:08 -04:00
test_util.h Eventing framework refactor (#6610) 2020-12-17 22:21:53 -05:00