valitydev/osquery-1
14
0
Fork 0
mirror of https://github.com/valitydev/osquery-1.git synced 2024-11-06 09:35:20 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Docs: fix all broken or redirected URLs and references (#6835)

Browse Source
This commit is contained in:
Mike Myers 2020-12-27 08:22:16 -08:00 committed by GitHub
parent 948a570c51
commit 1a91db92d6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
10 changed files with 31 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/wiki/deployment/aws-logging.md
View File

@ -1,6 +1,6 @@
# Logging osquery to AWS
As of osquery version 1.7.4, osquery can log results directly to Amazon AWS [Kinesis Streams](https://aws.amazon.com/kinesis/streams/) and [Kinesis Firehose](https://aws.amazon.com/kinesis/firehose/). For users of these services, `osqueryd` can eliminate the need for a separate log forwarding daemon running in your deployments.
As of osquery version 1.7.4, osquery can log results directly to Amazon AWS [Kinesis Streams](https://aws.amazon.com/kinesis/data-streams/) and [Kinesis Firehose](https://aws.amazon.com/kinesis/data-firehose/). For users of these services, `osqueryd` can eliminate the need for a separate log forwarding daemon running in your deployments.
## Configuration
@ -28,7 +28,7 @@ Some configuration is shared between the two plugins:
When working with AWS, osquery will look for credentials and region configuration in the following order:
1. Configuration flags
2. Profile from the [AWS config files](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html#cli-config-files) (only if `--aws_profile_name` is specified)
2. Profile from the [AWS config files](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) (only if `--aws_profile_name` is specified)
3. Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)
4. `default` profile in the AWS config files
5. Profile from the EC2 Instance Metadata Service

2
docs/wiki/deployment/configuration.md
View File

@ -247,7 +247,7 @@ When osquery's config parser is provided a string instead of inline dictionary t
**Where can I get more packs?**
We release (and bundle alongside RPMs/DEBs/PKGs/etc.) query packs that emit high signal events as well as event data that is worth storing in the case of future incidents and security events. The queries within each pack will be performance tested and well-formed (JOIN, select-limited, etc.). But it is always an exercise for the user to make sure queries are useful and are not impacting performance critical hosts. You can find the query packs that are released by the osquery team documented at [https://osquery.io/docs/packs](https://osquery.io/docs/packs) and the content in [**/packs**](https://github.com/osquery/osquery/blob/master/packs) within the osquery repository.
We release (and bundle alongside RPMs/DEBs/PKGs/etc.) query packs that emit high signal events as well as event data that is worth storing in the case of future incidents and security events. The queries within each pack will be performance tested and well-formed (JOIN, select-limited, etc.). But it is always an exercise for the user to make sure queries are useful and are not impacting performance critical hosts. You can find the query packs that are released by the osquery team in [**/packs**](https://github.com/osquery/osquery/blob/master/packs) within the osquery repository.
**How do I modify the default options in the provided packs?**

6
docs/wiki/deployment/log-aggregation.md
View File

@ -9,7 +9,7 @@ When it comes to aggregating the logs that `osqueryd` generates, you have severa
## Logstash
[LogStash](https://www.elastic.co/products/logstash) is an open source tool enabling you to collect, parse, index and forward logs. Logstash enables you to ingest osquery logs with its [file](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html) input plugin and then send the data to an aggregator via its extensive list of [output plugins](https://www.elastic.co/guide/en/logstash/current/output-plugins.html). A common datastore for logstash logs is [ElasticSearch](https://www.elastic.co/products/elasticsearch/).
[LogStash](https://www.elastic.co/logstash) is an open source tool enabling you to collect, parse, index and forward logs. Logstash enables you to ingest osquery logs with its [file](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html) input plugin and then send the data to an aggregator via its extensive list of [output plugins](https://www.elastic.co/guide/en/logstash/current/output-plugins.html). A common datastore for logstash logs is [ElasticSearch](https://www.elastic.co/elasticsearch/).
An example Logstash to ElasticSearch config may look like this:
@ -76,7 +76,7 @@ The way in which you analyze logs is very dependent on how you aggregate logs. A
### Kibana
If you are forwarding logs with [LogStash](https://www.elastic.co/products/logstash/) to [ElasticSearch](https://www.elastic.co/products/elasticsearch/), then you probably want to perform your analytics using [Kibana](https://www.elastic.co/products/kibana/).
If you are forwarding logs with [LogStash](https://www.elastic.co/logstash/) to [ElasticSearch](https://www.elastic.co/elasticsearch/), then you probably want to perform your analytics using [Kibana](https://www.elastic.co/kibana/).
Logstash will index logs into ElasticSearch using a default index format of logstash-YYYY-MM-DD. Kibana has a default Logstash dashboard and automatically field-extracts all log lines making them available for search.
@ -94,4 +94,4 @@ Splunk will automatically extract the relevant fields for analytics, as shown be
If you are using a log forwarder which has less requirements on how data is stored (for example, Splunk Forwarders require the use of Splunk, etc.), then you have many options on how you can interact with `osqueryd` data. It is recommended that you use whatever log analytics platform that you are comfortable with.
Many people are very comfortable with [Logstash](https://www.elastic.co/products/logstash/). If you already have an existing Logstash/Elasticsearch deployment, that is a great option to exercise. If your organization uses a different backend log management solution, osquery should tie into that with minimal effort.
Many people are very comfortable with [Logstash](https://www.elastic.co/logstash/). If you already have an existing Logstash/ElasticSearch deployment, that is a great option to exercise. If your organization uses a different backend log management solution, osquery should tie into that with minimal effort.

2
docs/wiki/deployment/logging.md
View File

@ -29,7 +29,7 @@ For information on configuring logger plugins, see [logging/results flags](../in
Status logs are generated by the [Glog logging framework](https://github.com/google/glog/). The default **filesystem** logger plugin writes these logs to disk the same way Glog would. Logger plugins may intercept these status logs and write them to system or otherwise.
As the above directory listing reveals, `osqueryd.INFO` is a symlink to the most recent execution's `INFO` log. The same is true for the `WARNING`, `ERROR` and `FATAL` logs. For more information on the format of Glog logs, please refer to the [Glog documentation](https://github.com/google/glog/blob/master/doc/glog.html).
As the above directory listing reveals, `osqueryd.INFO` is a symlink to the most recent execution's `INFO` log. The same is true for the `WARNING`, `ERROR` and `FATAL` logs. For more information on the format of Glog logs, please refer to the [Glog documentation](https://github.com/google/glog/blob/master/README.rst).
Note: The `osqueryi` shell only shows `WARNING` and `ERROR` status logs, the `INFO` logs are silenced for a better shell-like experience.

4
docs/wiki/deployment/syslog.md
View File

@ -4,13 +4,13 @@ osquery 1.7.3 introduced support for consuming and querying the macOS system log
## macOS Syslog
On macOS, the `asl` virtual table makes use of Apple's ASL store, querying this structured store using the routines provided in [`asl.h`](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/asl.3.html).
On macOS, the `asl` virtual table makes use of Apple's ASL store, querying this structured store using the routines provided in [`asl.h`](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/asl_set.3.html).
### macOS Configuration
No configuration is required to begin using the `asl` table. Note, however, that the table is only able to query logs that are available in the ASL store.
If your target logs are not already being sent to the ASL store by your current configuration, take a look at the [man page](https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man5/asl.conf.5.html) for `asl.conf`, and use the `store` action to ensure your logs of interest are available in the store. `asl.conf` is also responsible for the rotation and retention settings of the ASL store.
If your target logs are not already being sent to the ASL store by your current configuration, take a look at the [man page](https://www.unix.com/man-page/osx/5/asl.conf/) for `asl.conf`, and use the `store` action to ensure your logs of interest are available in the store. `asl.conf` is also responsible for the rotation and retention settings of the ASL store.
The configuration for `/var/log/install.log` and `/var/log/commerce.log` is hardcoded into the Apple provided syslog binaries, and we are not aware of a way to configure ASL to send these logs to the store.

2
docs/wiki/development/building.md
View File

@ -101,7 +101,7 @@ Note: It may be easier to install these prerequisites using [Chocolatey](https:/
- [Git for Windows](https://github.com/git-for-windows/git/releases/latest): Select "checkout as-is, commit as-is". Later check "Enable symbolic links" support.
- [Python 3](https://www.python.org/downloads/windows/), specifically the 64-bit version.
- [Wix Toolset](https://wixtoolset.org/releases/)
- [Strawberry Perl](http://strawberryperl.com/) for the OpenSSL formula. It is recommended to install it to the default destination path.
- [Strawberry Perl](https://strawberryperl.com/) for the OpenSSL formula. It is recommended to install it to the default destination path.
- [7-Zip](https://www.7-zip.org/) if building the Chocolatey package.
### Optional: Install Python tests prerequisites

2
docs/wiki/development/config-plugins.md
View File

@ -2,7 +2,7 @@
For details on how `osqueryd` schedules queries and loads information from a config, see the [configuration](../deployment/configuration.md) deployment guide.
You may need to distribute osquery configurations in a different way than the default method. To support all deployment types, the way that `osqueryd` retrieves its configuration is itself completely pluggable and customizable. By default, `osqueryd` will look for a JSON file on disk using the default config plugin: **filesystem**. If you distribute configurations via something like [Zookeeper](https://zookeeper.apache.org) or [etcd](https://github.com/coreos/etcd), you need to write a C++ function that can acquire a string of JSON. This developer tutorial will walk through the default filesystem config plugin as a demonstration for creating new config inputs.
You may need to distribute osquery configurations in a different way than the default method. To support all deployment types, the way that `osqueryd` retrieves its configuration is itself completely pluggable and customizable. By default, `osqueryd` will look for a JSON file on disk using the default config plugin: **filesystem**. If you distribute configurations via something like [Zookeeper](https://zookeeper.apache.org) or [etcd](https://github.com/etcd-io/etcd), you need to write a C++ function that can acquire a string of JSON. This developer tutorial will walk through the default filesystem config plugin as a demonstration for creating new config inputs.
## Example: Filesystem config

25
docs/wiki/development/osquery-sdk.md
View File

@ -1,6 +1,6 @@
# The osquery SDK
The osquery "public API" or SDK is the set of osquery headers and a subset of the source "cpp" files implementing what we call osquery **core**. The core code can be thought of as the framework or platform, it is everything except for the SQLite code and most table implementations. The public headers can be found in [osquery/include/osquery/](https://github.com/osquery/osquery/tree/master/include/osquery).
The osquery "public API" or SDK is the set of osquery headers and a subset of the source "cpp" files implementing what we call osquery **core**. The core code can be thought of as the framework or platform, it is everything except for the SQLite code and most table implementations. The public headers can be found in [/osquery/osquery/sdk/](https://github.com/osquery/osquery/tree/master/osquery/sdk).
osquery is organized into a **core**, **additional**, and **testing** during a default build from source. We call the set of public headers implementing **core** the 'osquery SDK'. This SDK can be used to build osquery outside of our CMake build system with a minimum set of dependencies. This organization better isolates OS API dependencies from development tools and libraries and provides a logical separation between code needed for extensions and module compiling.
@ -14,11 +14,13 @@ Extensions use osquery's [Thrift API](https://github.com/osquery/osquery/blob/ma
Only the osquery SDK provides the simple `startExtension` symbol that manages the life of your process, including the Thrift service threads and a watchdog.
osquery extensions should statically link the **core** code and use the `<osquery/sdk.h>` helper include file. C++ extensions should link: boost, thrift, glog, gflags, and optionally rocksdb for eventing. Let's walk through a basic example extension in C++ (source for [example_extension.cpp](https://github.com/osquery/osquery/blob/master/osquery/examples/example_extension.cpp)):
osquery extensions should statically link the **core** code and use the `<osquery/sdk.h>` helper include file. C++ extensions should link: boost, thrift, glog, gflags, and optionally rocksdb for eventing. Let's walk through a basic example extension in C++ (source for [read_only_table/main.cpp](https://github.com/osquery/osquery/tree/master/external/examples)):
```cpp
// Note 1: Include the sdk.h helper.
#include <osquery/sdk.h>
// Note 1: Include the SDK and helpers
#include <osquery/core/system.h>
#include <osquery/sdk/sdk.h>
#include <osquery/sql/dynamic_table_row.h>
using namespace osquery;
@ -32,13 +34,14 @@ class ExampleTablePlugin : public TablePlugin {
};
}
QueryData generate(QueryContext& request) override {
QueryData results;
Row r;
TableRows generate(QueryContext& request) {
TableRows results;
auto r = make_table_row();
r["example_text"] = "example";
r["example_integer"] = INTEGER(1);
results.push_back(r);
results.push_back(std::move(r));
return results;
}
};
@ -57,7 +60,7 @@ int main(int argc, char* argv[]) {
runner.requestShutdown(status.getCode());
}
// Finally, shutdown.
// Finally wait for a signal / interrupt to shutdown.
runner.waitForShutdown();
return 0;
}
@ -69,7 +72,7 @@ The `osqueryi` or `osqueryd` processes start an "extension manager" Thrift servi
Please see the deployment [guide on extensions](../deployment/extensions.md) for a more-complete overview of how and why extensions are used.
If you [build from source](../development/building.md), you will build an example extension. The code can be found in the [`osquery/examples`](https://github.com/osquery/osquery/blob/master/osquery/examples/example_extension.cpp) folder; it adds a config plugin called "example" and additional table called "example". There are two ways to run an extension: load the extension at an arbitrary time after shell or daemon execution, or request an "autoload" of extensions. The auto-loading method has several advantages, such as allowing dependencies on external config plugins and inheriting the same process monitoring as is applied to the osquery core worker processes.
If you [build from source](../development/building.md), you will build an example extension. The code can be found in the [`osquery/external/examples`](https://github.com/osquery/osquery/tree/master/external/examples) folder; it adds a config plugin called "example" and additional table called "example". There are two ways to run an extension: load the extension at an arbitrary time after shell or daemon execution, or request an "autoload" of extensions. The auto-loading method has several advantages, such as allowing dependencies on external config plugins and inheriting the same process monitoring as is applied to the osquery core worker processes.
The `osqueryi` shell also allows a quick and easy command-line autoload using `--extension`. Let's review both options.
@ -137,7 +140,7 @@ Your "external" extension, in the sense that the code is developed and contained
This will find and compile all `.*\.{cpp,c,mm}` files within your external directory. If you need something more complicated, add a `CMakeLists.txt` to your directory and add your targets to the `externals` target.
See [`CMake/CMakeLibs.cmake`](https://github.com/osquery/osquery/blob/master/CMake/CMakeLibs.cmake) for more information about the `ADD_OSQUERY_EXTENSION` CMake macro.
See [`/osquery/external/cmake/cmakelibs.cmake`](https://github.com/osquery/osquery/blob/master/external/cmake/cmakelibs.cmake) for more information about the `addOsqueryExtension` and `addOsqueryExtensionEx` CMake macros.
Example:

8
docs/wiki/development/reading-files.md
View File

@ -1,11 +1,13 @@
[include/osquery/filesystem.h](https://github.com/osquery/osquery/blob/master/include/osquery/filesystem.h) contains utilities for accessing the filesystem.
# Reading Files
[/osquery/filesystem/filesystem.h](https://github.com/osquery/osquery/blob/master/osquery/filesystem/filesystem.h) contains utilities for accessing the filesystem.
Consider the following example for reading a file from the filesystem:
```cpp
#include <iostream>
#include <string>
#include <osquery/filesystem.h>
#include <osquery/filesystem/filesystem.h>
const std::string kPath = "/foo/bar.txt"
@ -34,7 +36,7 @@ To internalize the main API, consider the same example without error checking:
```cpp
#include <iostream>
#include <string>
#include <osquery/filesystem.h>
#include <osquery/filesystem/filesystem.h>
int main(int argc, char* argv[]) {
std::string content;

2
docs/wiki/installation/custom-packages.md
View File

@ -55,7 +55,7 @@ $ ./tools/deployment/make_osx_package.sh -c ~/Desktop/osquery.conf
The distributable package can be found at `./build/darwin/osquery-VERSION.pkg`.
You can now use your existing package distribution system ([JAMF](https://www.jamf.com), [Chef](https://www.chef.io/products/chef-infra/), etc.) to push this package to your infrastructure.
You can now use your existing package distribution system ([JAMF](https://www.jamf.com), [Chef](https://www.chef.io/products/chef-infra), etc.) to push this package to your infrastructure.
### Custom LaunchDaemon