Change the message that is logged when a required extension is not
loaded or found from "Extension not autoloaded: <extension id>" to
"Required extension not found or not loaded: <extension id>".
Change the message that is logged when the extension manager has an
error happening during startup from "Cannot start extension manager:
<error>" to "An error occurred during extension manager startup: <error>"
Add the possibility of startExtensionManager returning with an error
if the Dispatcher failed to properly add the ExtensionManagerWatcher
or ExtensionManagerRunner.
Fallback to logging the startExtensionManager error message with a
severity of INFO only if extensions are required to be disabled.
Add tests for starting the extension manager with a nonexistent
socket path and a nonexistent required extension.
Issue osquery/osquery#5679
This defines a new CMake option: OSQUERY_FUZZ, which appends compiler
and linker flags for libFuzzer to be effective.
There is one example harness that will fuzz config parsing.
Add a way to compile third-party libraries from source instead of downloading prebuilt ones.
Each library source code is downloaded with git into a submodule at configure time,
in response to the find_package(library_name) CMake call,
except for OpenSSL where the official source archive is used.
Each submodule is attached to a release tag on its own upstream repository.
All the libraries are built using CMake directly, except for OpenSSL which uses a formula system,
which permits to build libraries with a separate build system
when there's no easy way to integrate it directly with CMake.
This new dependency system determines which library is fetched from where using the concept of "layers".
Currently we have three of them: source, formula, facebook,
where the last layer represents the pre-built libraries.
The provided order will be used when looking for libraries.
A system to patch submodule source code has been added and it's currently used with googletest, libudev and util-linux.
Patches should be put under libraries/cmake/source/<library name>/patches/<submodule>,
where <submodule> is often one and is "src", but in other cases, like AWS,
there are multiple with a more specific name.
If for whatever reason the submodule cloning or the patching fails,
the submodule has to be unregistered and its folder should be cleared.
This should be achievable with "git submodule deinit -f <submodule path>"
Following some other changes on existing functionality:
- Changed the CMake variable BUILD_TESTING to OSQUERY_BUILD_TESTS
to avoid enabling tests on third party libraries.
Due to an issue with glog the BUILD_TESTING variable
will be always forced to OFF.
- Moved compiler and linker flags to their own file cmake/flags.cmake
- Moved all the third-party CMakeLists.txt used for pre-built libraries under libraries/cmake/facebook
- Added the --exclude-folders option to tools/format-check.py and tools/git-clang-format.py,
so that it's possible to ignore any third party library source code.
- The format and format_check target use the new --exclude-folders option
to exclude libraries/cmake/source from formatting.
- The test and osquery binaries are properly compiled with PIE (osquery/osquery#5611)
Co-authored-by: Stefano Bonicatti <stefano.bonicatti@gmail.com>
Co-authored-by: Teddy Reed <teddy@casualhacking.io>
Issue highlighted by asan activated in PR osquery/osquery#5628
Ensure WrappedMessage has no padding since its members are written
consecutively in the buffer.
Also use memcpy when retrieving a WrappedMessage from a buffer, since
it could be written at a misaligned address.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
EFI_DEVICE_PATH_PROTOCOL and HARDDRIVE_DEVICE_PATH were using
the wrong alignment/padding, since on disk they are written
with no padding.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Imprecisions between float -> double -> json -> double -> float
lead to out of range values been saved into a float variable.
Since json has only the notion of doubles as floating point numbers,
it's better to require to use them.
Also forced the json parser to parse floating point numbers
with full precision, otherwise the test testing for precision would fail.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
When shifting left or right a byte, that must be positive, so
ensure it is.
Light cleanup of a bugged and unused function.
A deeper look into the table implementation is needed.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Do not try to read the destination address of a netmask if such address
is a default route.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Reading a 8 byte field from a 4 byte aligned struct needs to be
done with memcpy.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Do not use memcpy to copy strings around.
Also, ifa_name size is not guaranteed to be IFNAMSIZ.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Do not use clear() on a vector inizialized with a fixed size to clear
it of its contents when using it as a char buffer.
PR: osquery/osquery#5665
When proactively searching disk for personal certificates, there is no
need to filter system accounts (SYSTEM, Local Service, etc) anymore
because `findUserPersonalCertsOnDisk` is now capable of handling those
accounts by dynamically finding a user's home dir (as rather than
constructing a hard coded path).
This now makes the table even more complete; any certificates found in
the system accounts directories will always be found. Previously they
could be found but only if there was a store location other than the
`Users` store location that had a system store string that looked like
`S-1-5-18\My` or `.DEFAULT\My`.
It does not look like the EC2 tables are available on Windows
(due to spec-file configuration).
Please see https://osquery.io/schema/3.3.2#ec2_instance_tags
This removes the EC2 library linking requirement until we revisit this
feature.
Add OSQUERY_VERSION cache variable to force the version to be set at
that value.
Add OSQUERY_VERSION_AUTODECTED cache variable to show the osquery
version that has been detected through git
Add OSQUERY_VERSION_SUFFIX cache variable to append its value to the
final version, if the version was autodected.
Add OSQUERY_VERSION_INTERNAL variable to globally store the final
osquery version, to be used in other CMake scripts.
Add OSQUERY_VERSION_COMPONENTS variable to globally store the components
of the final osquery version. Currently used by the packaging system.
Modify overwrite_cache_variable so that's possible to specify the type.
The version is always autodetected using "git describe --tags --always --dirty"
and OSQUERY_VERSION_SUFFIX is appended to it, unless OSQUERY_VERSION is manually set.
Incorporates suggestions by @directionless.
CMake part of osquery/osquery#5615
* Initial implementation
* Use case insensitive comparisons for all service names
Fixes a bug where certificates for services that correspond to Local Service or
Network Service may not have their sids appear correctly. This is because the
services table is inconsistent with its user_account column.
* Make service name cache query-local
Previously, the service name cache existed for the lifetime of the
osquery process, which made it susceptible to stale reads if a service
restarted under a different user during osquery's lifetime. Now the
cache is created for each query. Also refactor it to directly map to the
sid, rather than the account name, which removes the need to translate
from account name to sid every row.
* Fix reference to destroyed object
Previously, getCurrentUserInfo took a reference to data from a local
vector, whose data is free'd after the function. This refactors the code
to use a unique_ptr (similar to how getSidFromUsername) does it.
The code was failing on Windows because the '\Windows\%' pattern
is relative and presumes that the Windows folder is on the same
drive (C:\) as the test process.
This might not be true, so we find where precisely is the Windows
directory and use the full path to it as a pattern.
The failing test checks that were testing the equal and LIKE operator
have been moved to a new test, test_table_constraints, since they
are not related to joins.
Fix UsersTest.test_sanity on Windows.
uid and gid were returned as int (while they normally are unsigned int)
and converted to signed integers in the table row.
This is wrong because beyond uid and gid not being ints,
they are taken from the RID part of the SID which in some cases,
like for a Service SID, it can have a value higher than then maximum
value of an int, so in the end the number shown in table is negative.
Now they are returned as uint32_t and converted as BIGINTs for the table
that uses them.
Fix other functions return values and conversions depending on the meaning of
the value.
On Windows stick to its specific types where possible.
Convert CRLF to LF on some of the files modified.
