Commit Graph

5310 Commits

Author SHA1 Message Date
Teddy Reed
62ba77cbe6
cpack: Add RPM and DEB postinstall script (#6097)
Co-Authored-By: Stefano Bonicatti <smjert@gmail.com>
2019-12-11 15:44:44 -05:00
Zachary Wasserman
039d7353cf
Use cert bundle in repo for make_windows_package script (#6114)
Instead of using the path to an openssl provided cert bundle (which no
longer exists in recent installs of openssl from Chocolatey), use the
cert bundle already stored in the repo.

Fixes #6109
2019-12-10 15:09:20 -08:00
Alessandro Gario
ec2593e143 rpm_packages: Add the install_time column 2019-12-10 22:41:45 +01:00
Teddy Reed
2933614301
tables: Remove INDEX requirement for ADDITIONAL option (#6104)
We do not need an explicit INDEX column for ADDITIONAL to work.
If the ADDITIONAL option is set the constraint should be passed
into the virtual table context.
2019-12-10 15:43:35 -05:00
Teddy Reed
8c891ab0a8
docs: Fix custom package markdown (#6110)
Co-Authored-By: Nick Anderson <nanderson7@gmail.com>
2019-12-10 00:32:27 -05:00
Stefano Bonicatti
7922cd78aa Fix possible xxhash collision between zstd and rdkafka (#6101)
Both libraries use the xxhash library,
compiling its source files directly.
The version they use though it's different so to avoid the linker
resolve the collision by removing one of the two implementations,
we prefix the functions with the respective library name.

Moreover we make the xxhash API private, by declaring the static
so that any unused function is not included anymore in the binary
and only the used one are present.
2019-12-07 20:52:34 -05:00
Stefano Bonicatti
0e0d7b719f Update building doc with Python packages needed by tests 2019-12-07 20:51:09 -05:00
Stefano Bonicatti
e7cef1a906 Prepare CMake and Python framework for Python tests
test_base.py received several fixes and now supports and requires
a custom folder where to find the config files that the tests need.
It has been also updated to use Python 3.

Fixed the gen_api function in genapi.py missing the path to the spec
file when evaluating blacklisted tables.

Co-Authored-By: SS <sahil.suneja@trailofbits.com>
2019-12-07 20:51:09 -05:00
Stefano Bonicatti
a5afb35175 Update azure-pipelines.yml to install needed Python packages
Add an init process so that reaping works in the CI Docker image

Update Docker image name.
2019-12-07 20:51:09 -05:00
Stefano Bonicatti
8ab9a798eb Change and add some python packages in the CI Dockerfile
- Add the bdist_wheel package to have pip optimize packages.
- Update Python 2 packages to Python 3
- Use pexpect==3.3
2019-12-07 20:51:09 -05:00
Stefano Bonicatti
515704954c Move the test config files in their own directory
This is to have better separation between them and the python tests.
BUCK files have been updated accordingly.

Changes to the internal generateCopyFileTarget function were needed.
The function now supports a base folder to be set so that the regex or file path
is appended to that base path, without having it included in the destination.
It will also not use a library target anymore, but a custom one so
that's possible to set properties with custom names.

Use only one copy target to copy test config files instead of having
multiple that overlaps.
It may give issues when copying and tests in the end will need all of
them.
2019-12-07 20:51:09 -05:00
Teddy Reed
6cf169c6d8
tables: Fix mdfind table by requiring query (#6103) 2019-12-06 10:11:51 -05:00
Teddy Reed
f3b3cd1c7d
linux: Update systemd service to use KillMode=control-group (#6096) 2019-12-05 20:42:22 -05:00
Teddy Reed
4a7c64b030
cpack: Revert renaming com.facebook.plist to com.osquery.plist (#6093) 2019-12-03 23:22:17 -05:00
Garret Reece
5c3a4636ec macOS: Add a bsd_flags column to the file table (#5981) 2019-12-03 18:40:42 -08:00
Teddy Reed
00f43e9915 config: Add more checks for pack JSON (#6071) 2019-12-03 18:22:54 -08:00
Nick Anderson
80cbefbae7 tests: adding platform conversion strings tests (#5908)
* Summary:

This adds tests for the windows and darwin conversions strings helper
classes. Darwin currently has tests, but they are not hooked up and
being run. Windows does not have tests, this adds two simple unit tests
against the logic for our string conversions.

Test Plan:
Ran the tests.
2019-12-02 14:32:05 -05:00
Stefano Bonicatti
a30c67da30 Update steps to install prerequisites on all platforms
- Remove Python 2 as a prerequisite

- Add missing instructions when install Git for Windows

- Correct and move the paragraph about symbolic links permission earlier.
  The "Enable symbolic link" check available during the Git for Windows
  installing procedure doesn't enable the policy, but instead asks git
  to create real symlinks when cloning, which is still something
  we want.
2019-11-28 13:24:28 +01:00
mark m
706874c3d3 Implement azure metadata and tags tables (#5434) 2019-11-26 21:12:27 -05:00
Brendan Shaklovitz
b18f4315a4 Fix python3 deprecation warning (#5977) 2019-11-26 16:57:27 -05:00
Stefano Bonicatti
501b92465b CMake: Change Python version required to be at least 3.5 (#6081)
Removed the restriction for the Python version to be >= 3.6 but < 3.8.
Now it has to be >=3.5, lower versions are EOL.

Removed the search for Python2 too since nothing uses it anymore.

Fixes osquery/osquery#6079
2019-11-26 13:50:21 -05:00
Stefano Bonicatti
8e6fefb47a CMake: explicitly enable ASM_MASM language on Windows (#6065)
This is needed by some third party libraries and
when using Ninja on Windows.
CMake complains that it cannot find the internal variable
for CMAKE_ASM_MASM_COMPILE_OBJECT if it's not active for the whole
project.
2019-11-23 00:14:56 -05:00
thehellajeff
e3b809b4de Added return code to requestShutdown call (#6018) 2019-11-21 10:27:00 -08:00
Stefano Bonicatti
45cf390c2a Force git to create symlinks on the CI (#6064)
git on Windows supports cloning a repository with symlinks
without actually creating them; it will substitute them
with text files containing the target path.
This can potentially break the build of some of osquery
third party libraries.

Moreover this is the default on the CI,
which is different compared to the setup on local machines,
which means that it can hide other kind of issues.
2019-11-20 21:36:23 -05:00
Stefano Bonicatti
93ceef7e5f CMake: Prefer Python 3.7 then 3.6 (#6057)
We are not currently ready for Python 3.8,
so we prefer either the newest 3.7 or 3.6.
2019-11-19 12:54:40 -05:00
Teddy Reed
4147ec8c75
changelog: Add 4.1.1 changelog (#6058) 2019-11-19 12:53:36 -05:00
Stefano Bonicatti
5315d4be8e CMake: Workaround Thrift symlink issue on Windows (#6060)
CMake doesn't support reading and therefore copying Windows symlinks.
A recent update to Thrift creates a symlink in its tutorial folder,
which makes the copy after patch process fail.
As a workaround we avoid copying the tutorial folder,
since we don't care about it anyway.
2019-11-19 12:52:23 -05:00
Daniel Zabari
0fa5751c60 added warning and turned off shallow for git version less than 2.14 (#6055)
added warning and turned off shallow for git version less than 2.14
2019-11-19 11:24:47 -05:00
Teddy Reed
b170a517b5
tables: Fix apt_sources and refactor tests (#6047) 2019-11-19 08:56:18 -05:00
Teddy Reed
66700b9251
fuzzing: Minify logic required for new harnesses (#5942) 2019-11-19 00:25:29 -05:00
Teddy Reed
d3959d578d
tables: Re-enable required-column warning message (#6038) 2019-11-19 00:24:53 -05:00
Teddy Reed
19d4e3fc78
docs: Remove references to old osquery kernel extension (#6051) 2019-11-19 00:23:47 -05:00
Teddy Reed
d52786d08c
parsers: Always apply the options parser first (#6050) 2019-11-19 00:23:15 -05:00
Teddy Reed
fa8ac48a87
http_client: Handle relative redirects (#6049) 2019-11-19 00:22:41 -05:00
packetzero
0f9b15b05b new windows_optional_features table (#5991) 2019-11-18 20:48:11 -08:00
Teddy Reed
e4e95e9e65 docs: Add common code-review checks to PR template (#6048) 2019-11-18 20:44:34 -08:00
Teddy Reed
2d4308b284 tables: Improve integration test debugging and fix deb_packages (#6045) 2019-11-18 20:43:18 -08:00
Teddy Reed
07afc4e2a7 tables: Change atom_packages to use user constraints (#6052) 2019-11-18 20:40:45 -08:00
Teddy Reed
3e437d555d tables: Improve nvram to use input variable names (#6053) 2019-11-18 20:39:37 -08:00
Stefano Bonicatti
8bab38e30d Fix use after move in YARAEventSubscriber::configure (#6054) 2019-11-17 17:04:36 -05:00
Teddy Reed
e9b224b53b
extensions: Skip shell socket when extensions are disabled (#6046) 2019-11-17 16:04:35 -05:00
Teddy Reed
c8cd366c03
docs: Recommend macOS 10.11 deployment target (#6000) 2019-11-15 15:20:34 -05:00
Ted Reed
59dc17266b libraries: Add rocksdb to source_migration 2019-11-15 15:19:48 -05:00
Ted Reed
0dccb51087 libraries: Add ROCKSDB_LITE to public definitions 2019-11-15 15:19:48 -05:00
Ted Reed
c61c56f159 libraries: Reduce rocksdb sources
Several source files are not exporting any symbols.
2019-11-15 15:19:48 -05:00
packetzero
b2e48695ec yara: avoid scanning special files that could hang (#5971) 2019-11-12 18:51:55 -05:00
Teddy Reed
bf9a9ec567
config: Code review for ATC parser (#6029) 2019-11-12 18:46:18 -05:00
packetzero
de54b643e5 extensions compatibility : default columns to INDEX (#6006)
Recent changes in INDEX enforcement broke extensions.

This fix adds a compatibility flag that defaults to the old behavior of INDEX on all columns.
This only affects extension virtual tables, all built in tables have index constraints enforced.
2019-11-12 13:40:48 -05:00
Alessandro Gario
2e2e9032b8
Update SQLite from version 3.29.0-3 to 3.30.1-1 (#6020) 2019-11-12 18:33:01 +01:00
Ted Reed
76720bea2f libraries: Add thrift to source_migration
This commit adds scope_exit and typeof boost subprojects.
These are dependencies of Window's thrift library.
2019-11-12 08:15:53 -05:00