* Initial implementation
* Use case insensitive comparisons for all service names
Fixes a bug where certificates for services that correspond to Local Service or
Network Service may not have their sids appear correctly. This is because the
services table is inconsistent with its user_account column.
* Make service name cache query-local
Previously, the service name cache existed for the lifetime of the
osquery process, which made it susceptible to stale reads if a service
restarted under a different user during osquery's lifetime. Now the
cache is created for each query. Also refactor it to directly map to the
sid, rather than the account name, which removes the need to translate
from account name to sid every row.
* Fix reference to destroyed object
Previously, getCurrentUserInfo took a reference to data from a local
vector, whose data is free'd after the function. This refactors the code
to use a unique_ptr (similar to how getSidFromUsername) does it.
The code was failing on Windows because the '\Windows\%' pattern
is relative and presumes that the Windows folder is on the same
drive (C:\) as the test process.
This might not be true, so we find where precisely is the Windows
directory and use the full path to it as a pattern.
The failing test checks that were testing the equal and LIKE operator
have been moved to a new test, test_table_constraints, since they
are not related to joins.
Fix UsersTest.test_sanity on Windows.
uid and gid were returned as int (while they normally are unsigned int)
and converted to signed integers in the table row.
This is wrong because beyond uid and gid not being ints,
they are taken from the RID part of the SID which in some cases,
like for a Service SID, it can have a value higher than then maximum
value of an int, so in the end the number shown in table is negative.
Now they are returned as uint32_t and converted as BIGINTs for the table
that uses them.
Fix other functions return values and conversions depending on the meaning of
the value.
On Windows stick to its specific types where possible.
Convert CRLF to LF on some of the files modified.
Calling GetLastError() will not return the error code for the Windows registry APIs,
since they return a LSTATUS value which already the error code.
This also fixes the RegistryTablesTest.test_registry_non_existing_key
test case which was incorrectly expecting success when querying
for a non-existent registry key.
Ported from https://github.com/osql/osql/pull/50
Only define BOOST_ASIO_DISABLE_STD_STRING_VIEW.
We shouldn't define BOOST_ASIO_HAS_STD_STRING_VIEW,
because even if we define BOOST_ASIO_DISABLE_STD_STRING_VIEW
the first define will actually enable parts of code that will use string_view.
This won't work on Windows and in general, string_view should not be
used unless compiling with C++17.
The hack has been also added to a test that was previously missed.
Taken from osql-experimental.
- Change CMake code license to the one present in osquery right now
- Package metadata doesn't mention Trail of Bits or osql anymore
- Set specific ACLs for the osqueryd on Windows when packaging
- Remove LLVM_INSTALL_PATH support on macOS, since we are using AppleClang
- Remove OSQUERY_SOURCE_DIR variable need and source in a submodule support
- Add targets format_check and format to check code formatting and
format it with clang-format
- Do not warn about not using Clang on macOS when using AppleClang
Summary:
I'm going to it as one of the decorators for streaming events. To be able to associate event with a certain machine on the backend side.
Why a new directory in osquery/? There are some libraries that depend on
core/database and any monster modules. To be able to keep utils simple,
lightweight and independent we should not put stuff like filesystem or network
under utils/ directory. But we need a single place to put all system assosiated
libs. So, let's make a `system` in the root of osquery.
Reviewed By: guliashvili
Differential Revision: D14706186
fbshipit-source-id: d40fde3872ca6b6677a1d8f89cfd8eda63c6b83d
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5553
Infer flagged this use of `unique_ptr` as potentially leaking the FD. I do not think this is the case but there is a missing check for a failed `open`. I am unsure of what the `ioctl` would do in this case.
I removed the custom decl to make the logic in this table easier (opinion).
Reviewed By: guliashvili
Differential Revision: D14700412
fbshipit-source-id: cfcfe9d2629aa27f18d39058830cb5c63881f163
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5552
This fixes a potential resource leak in the `syslog` event type. If the process is unable to `flock` then the FD is reset (overwritten to -1) and not closed.
Reviewed By: guliashvili
Differential Revision: D14700407
fbshipit-source-id: 092a22c3059d29aca748ee23ba30c5301cd065af
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5547
Since this diff an object of a class with defined schema (see type trait
schemer::has_schema) are allowed as memebers of anoter class with schema.
Example. C++ classes:
```
class Simple {
int alpha = 1;
public:
template <typename Archive, typename ValueType>
static void discloseSchema(Archive& a, ValueType& value) {
schemer::record(a, "alpha", value.alpha);
}
};
class Nested {
Frist beta;
int gama = 2;
public:
template <typename Archive, typename ValueType>
static void discloseSchema(Archive& a, ValueType& value) {
schemer::record(a, "beta", value.beta);
schemer::record(a, "gama", value.gama);
}
};
```
Json representation of `Nested`:
```
{
"beta": {
"alpha": 1
},
"gama": 2
}
```
Reviewed By: SAlexandru
Differential Revision: D14683589
fbshipit-source-id: 1f9e2f862d2bf64be166a717e49cf0f470f8ee36
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5548
This is a JSON deserializing formatter for schemer. It parse C++ object from
JSON object according to defined in C++ class schema. The implementation based
on rapidjson library.
Two methods with the same name: `osquery::schemer::fromJson`
Reviewed By: SAlexandru
Differential Revision: D14664162
fbshipit-source-id: ba73490dc19127a04a43b7ba1807d1f1a0bf4f43
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5545
This is a JSON serializing formatter for schemer. It represents C++ object as JSON object according to defined in C++ class schema. The implementation based on rapidjson library. It is very simple - just print all key:value pairs directly to rapidjson stream (it could be files stream or string stream). The second method of formatter converts C++ objects directly to JSON in string.
Two methods with the same name: `osquery::schemer::toJson`
Reviewed By: SAlexandru
Differential Revision: D14663996
fbshipit-source-id: 7a902b0777c9094faf39ee3111bbc09d8c3c7a55
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5544
This is a framework to declare a serialization and deserialization
schema for C++ classes. The schema can be used by different implementations
to represent C++ object as data-interchange format or to parse an object from
formatted representation.
It works without macro or any verbose transformation code. All you need to
do to be able to serialize and deserialize some C++ class is to define a
template static method `discloseSchema` in the class. Which describe all
members that have to be read by serializer and written by deserializer.
Everything else is a job of schemer formatters.
Support of specific types, including nested types support depends on the formatter implementation.
1. One place to describe class members instead of two different methods for serializer and deserializer.
2. One way do describe schema for many formatters (JSON, TOML, hasher, etc).
3. Schemer also sets the order of elements of class, therefore even binary formatters can use it, just ignoring the names.
4. It is simple - just one method, nothing more. Therefore there is zero dependencies. Everything is in formatters.
- JSON serializer/deserializer.
- Hasher - it helps to impelement transformation of osquery events in hash form for probabilistic filters.
Reviewed By: SAlexandru
Differential Revision: D14663949
fbshipit-source-id: effcc7e91a85514cee700a0d5484ab4d317ac199
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5550
This fixes a potential memory leak in keychain_items similar to D14567925. This leak was identified with the default options for OSS infer.
Reviewed By: guliashvili
Differential Revision: D14641455
fbshipit-source-id: 3fd776fe8a2af300435907ecd1d7746b3045066e
Summary: To clean it up, they are parts of the same experimental subsystem and shoul be close to each other in experimental directory.
Reviewed By: marekcirkos
Differential Revision: D14668951
fbshipit-source-id: 404be1f3c24699557edc0b9e5ada823ea04533f1
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5542
We replace deprecated calls to the Status class with newer ones. This will stop Lint from nagging me every time I open these files. There should be no change to functionality.
Specifically, we use a codemod on `.cpp` files to replace any instance of `return Status();` with `return Status::success();`.
Reviewed By: guliashvili
Differential Revision: D14652922
fbshipit-source-id: 243576195ed201d6b82fc942a785742c6b01cf83
Summary: Every time a differential query runs we save the results to the database. We were serializing those results with or without types based on the `FLAGS_log_numerics_as_numbers` flag. However we always collect results with proper types therefore, on subsequent executions of the same query, the new results (typed) were being compared with the results retrieved from the DB (untyped if FLAGS_log_numerics_as_numbers == false), causing all results to be "different" and therefore osquery to report all results as "added" (and eventually all old results as "removed").
Reviewed By: SAlexandru
Differential Revision: D14669476
fbshipit-source-id: 8abc68cbcac90c73bd92fc8d34572ba3ee2f2c75
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5543
This diff removes two vestigial functions from the osquery core. There are no references to these functions except for the unit test, which we also update.
Reviewed By: marekcirkos
Differential Revision: D14664005
fbshipit-source-id: 4e176014c803e5ec8cebdf0bd064874b483c8f76
Summary: setName was not used, other than the test, testing setName. Removing it to make class immutable
Reviewed By: jessek
Differential Revision: D14640959
fbshipit-source-id: d684939ed31e8ad275650847fadb2fb8c11b6fdc
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5538
This bug was found using the OSS version of infer with default options.
Reviewed By: guliashvili
Differential Revision: D14567925
fbshipit-source-id: 64272880464577207552e3436754b7f6d32ef4ec
Summary:
osquery returned status 1(error) always when osqueryd service install uninstall was requested by the flags.
Even if install/uninstall was successfull.
Add extra logging in case of success.
Exit with status 0 when successfully installed/uninstalled
Reviewed By: jessek
Differential Revision: D14622771
fbshipit-source-id: c0c7569a2b13434e2ad28ba92cba308024b044ec
Summary:
User could have requested install and uninstall of osqueryd service together. However it does not make sense and actually osquery would only install the service.
This additional check, will exit osquery in case when install and uninstall are requested together.
Reviewed By: jessek
Differential Revision: D14622634
fbshipit-source-id: b88dc519bd1ef38653eb1660712bbf2e75bad36f
Summary:
Replaced fprint(stderr with std::cerr
Other than that, could not find any use of
```
```
Reviewed By: akindyakov, fmanco
Differential Revision: D14621228
fbshipit-source-id: e996b2507757dc6af710e7ab0e25fc8136bbb92e
Summary: Somehow, still looking for how, D14577520 broke communication between osqueryi/osqueryd -S and extension. Revert.
Reviewed By: akindyakov
Differential Revision: D14620885
fbshipit-source-id: 98392f6e1e5da1b0ea68ee54dc00b3bbcd687315
Summary: It requires some effort to fix warnings properly, let's just mute them for now.
Reviewed By: marekcirkos
Differential Revision: D14596617
fbshipit-source-id: 184937d15cc186afe045131d6d648cdbfb4ad574
Summary:
Working to offering thrift service validating queries. It will be used by query canary service and helldriver.
For that, I need to execute osqueryd -S with the given query. However, osqueryd -S (with extension) crashes while it exits. I can filter out crash output, but it means, I will not be able to catch if query really crashes the osqueryd.
I also don't want to really fix osquery watcher, as we have the watcher 2.0 as an intern project.
Fixing the current problem by stopping osquery to ignore FLAGS_disable_watchdog flag when extension is avaliable.
Reviewed By: marekcirkos
Differential Revision: D14577520
fbshipit-source-id: 588c31319b875b8af9d73da6d691f22bcfdb8fc9
Summary:
Pull Request resolved: https://github.com/facebook/osquery/pull/5532
Ok, this is overcomplicated code for signal handling. Besides untested magic
with signals it makes quite a lot of assumptions about program state when it is
called. For instance, it uses instance of Watcher and Dispatcher singleton
which can be already destroyed or corrupted at the that time.
This complexity cause rare unpredictable and hardly reproducible problems, that
lead to data losses, cross process deadlocks, crashes of osqueryd because of
crash an extensions and os on ans so far.
Couple of examples.
When extensions killed by signal it 2 time from 3 lead to crash of osqueryd. It
very well reproducible but quite not very well diagnosable.
Another example is std::terminate called at shutdown of extensions. Something
could not be destroyed correctly and call std::terminate, which is already bad
but sometimes reasonable. After it everything goes to signal handler which use
already destroyed Watcher and cause another std::terminate. It leads to
recursive signal handler call and eventually to stack overflow.
Solution.
I believe it could be fixed less radical, absolutely. It takes some time,
though. And because it will cost us a lot there is a question: do we really
need it here? I could not find a place in osquery code where it is important to
finish something before shutdown. Moreover it is pretty rare case when code
handle shutdown condition correctly. That makes me think that we don't need
signal handling at all or at least not like this.
Reviewed By: mkareta
Differential Revision: D14458287
fbshipit-source-id: daf8388323e4dacabeee3338d185577335503da3
Summary: Before, we sent the user and kernel time in a separate time series. They are usefull separately, in order to measure which queries are syscall extensive. However, for measuring the total time we need them together. We have wall time, but user + kernel time is more precise as they measure resource usage of the thread.
Reviewed By: jessek
Differential Revision: D14502715
fbshipit-source-id: 7a03ec5b1d7b328e9657d94f752d33e7aa771e97
