When requesting kSecClassIdentity items from the keychain sometimes
invalid items are returned. These cause errSecInvalidItemRef to
be returned from SecKeychainItemCopyAttributesAndData and result
in an empty row in the table. Catch the error and avoid returning
empty rows.
PR osquery/osquery#5464 makes it impossible to call plugins
which are only registered in the core, as its happening with osquery/osquery#5839.
What happens when this is working is that the filesystem plugin is not present
in the extension registry, so when Registry::call tries to search for a plugin
that provides the "config" feature, it won't find one and the resulting plugin name,
and later item_name, will be empty.
This results in the RegistryInterface::call reroute the call to the core,
through Thrift, which will end up in the plugin correctly answering.
Change the message that is logged when a required extension is not
loaded or found from "Extension not autoloaded: <extension id>" to
"Required extension not found or not loaded: <extension id>".
Change the message that is logged when the extension manager has an
error happening during startup from "Cannot start extension manager:
<error>" to "An error occurred during extension manager startup: <error>"
Add the possibility of startExtensionManager returning with an error
if the Dispatcher failed to properly add the ExtensionManagerWatcher
or ExtensionManagerRunner.
Fallback to logging the startExtensionManager error message with a
severity of INFO only if extensions are required to be disabled.
Add tests for starting the extension manager with a nonexistent
socket path and a nonexistent required extension.
Issue osquery/osquery#5679
This defines a new CMake option: OSQUERY_FUZZ, which appends compiler
and linker flags for libFuzzer to be effective.
There is one example harness that will fuzz config parsing.
Add a way to compile third-party libraries from source instead of downloading prebuilt ones.
Each library source code is downloaded with git into a submodule at configure time,
in response to the find_package(library_name) CMake call,
except for OpenSSL where the official source archive is used.
Each submodule is attached to a release tag on its own upstream repository.
All the libraries are built using CMake directly, except for OpenSSL which uses a formula system,
which permits to build libraries with a separate build system
when there's no easy way to integrate it directly with CMake.
This new dependency system determines which library is fetched from where using the concept of "layers".
Currently we have three of them: source, formula, facebook,
where the last layer represents the pre-built libraries.
The provided order will be used when looking for libraries.
A system to patch submodule source code has been added and it's currently used with googletest, libudev and util-linux.
Patches should be put under libraries/cmake/source/<library name>/patches/<submodule>,
where <submodule> is often one and is "src", but in other cases, like AWS,
there are multiple with a more specific name.
If for whatever reason the submodule cloning or the patching fails,
the submodule has to be unregistered and its folder should be cleared.
This should be achievable with "git submodule deinit -f <submodule path>"
Following some other changes on existing functionality:
- Changed the CMake variable BUILD_TESTING to OSQUERY_BUILD_TESTS
to avoid enabling tests on third party libraries.
Due to an issue with glog the BUILD_TESTING variable
will be always forced to OFF.
- Moved compiler and linker flags to their own file cmake/flags.cmake
- Moved all the third-party CMakeLists.txt used for pre-built libraries under libraries/cmake/facebook
- Added the --exclude-folders option to tools/format-check.py and tools/git-clang-format.py,
so that it's possible to ignore any third party library source code.
- The format and format_check target use the new --exclude-folders option
to exclude libraries/cmake/source from formatting.
- The test and osquery binaries are properly compiled with PIE (osquery/osquery#5611)
Co-authored-by: Stefano Bonicatti <stefano.bonicatti@gmail.com>
Co-authored-by: Teddy Reed <teddy@casualhacking.io>
Issue highlighted by asan activated in PR osquery/osquery#5628
Ensure WrappedMessage has no padding since its members are written
consecutively in the buffer.
Also use memcpy when retrieving a WrappedMessage from a buffer, since
it could be written at a misaligned address.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
EFI_DEVICE_PATH_PROTOCOL and HARDDRIVE_DEVICE_PATH were using
the wrong alignment/padding, since on disk they are written
with no padding.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Imprecisions between float -> double -> json -> double -> float
lead to out of range values been saved into a float variable.
Since json has only the notion of doubles as floating point numbers,
it's better to require to use them.
Also forced the json parser to parse floating point numbers
with full precision, otherwise the test testing for precision would fail.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
When shifting left or right a byte, that must be positive, so
ensure it is.
Light cleanup of a bugged and unused function.
A deeper look into the table implementation is needed.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Do not try to read the destination address of a netmask if such address
is a default route.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Reading a 8 byte field from a 4 byte aligned struct needs to be
done with memcpy.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Do not use memcpy to copy strings around.
Also, ifa_name size is not guaranteed to be IFNAMSIZ.
PR: osquery/osquery#5665
Issue highlighted by asan activated in PR osquery/osquery#5628
Do not use clear() on a vector inizialized with a fixed size to clear
it of its contents when using it as a char buffer.
PR: osquery/osquery#5665
When proactively searching disk for personal certificates, there is no
need to filter system accounts (SYSTEM, Local Service, etc) anymore
because `findUserPersonalCertsOnDisk` is now capable of handling those
accounts by dynamically finding a user's home dir (as rather than
constructing a hard coded path).
This now makes the table even more complete; any certificates found in
the system accounts directories will always be found. Previously they
could be found but only if there was a store location other than the
`Users` store location that had a system store string that looked like
`S-1-5-18\My` or `.DEFAULT\My`.
It does not look like the EC2 tables are available on Windows
(due to spec-file configuration).
Please see https://osquery.io/schema/3.3.2#ec2_instance_tags
This removes the EC2 library linking requirement until we revisit this
feature.
Add OSQUERY_VERSION cache variable to force the version to be set at
that value.
Add OSQUERY_VERSION_AUTODECTED cache variable to show the osquery
version that has been detected through git
Add OSQUERY_VERSION_SUFFIX cache variable to append its value to the
final version, if the version was autodected.
Add OSQUERY_VERSION_INTERNAL variable to globally store the final
osquery version, to be used in other CMake scripts.
Add OSQUERY_VERSION_COMPONENTS variable to globally store the components
of the final osquery version. Currently used by the packaging system.
Modify overwrite_cache_variable so that's possible to specify the type.
The version is always autodetected using "git describe --tags --always --dirty"
and OSQUERY_VERSION_SUFFIX is appended to it, unless OSQUERY_VERSION is manually set.
Incorporates suggestions by @directionless.
CMake part of osquery/osquery#5615
