valitydev/osquery-1
14
0
Fork 0
mirror of https://github.com/valitydev/osquery-1.git synced 2024-11-07 18:08:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #1842 from theopolis/minor_extensions_docs_edit

Browse Source 
[#1810] Minor changes to extensions-use docs
This commit is contained in:
Teddy Reed 2016-02-11 08:36:49 -08:00
commit 2a76e90ac7
2 changed files with 67 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
docs/wiki/deployment/extensions.md
View File

@ -4,7 +4,7 @@ If you are interested in writing extensions please read the [SDK and Extensions]
## Autoloading Extensions
The following [CLI flags](../installation/cli-flags.md) control extension autoloading:
The following [CLI flags](../installation/cli-flags.md) control extension auto-loading:
```sh
--extensions_autoload=/etc/osquery/extensions.load
@ -12,7 +12,7 @@ The following [CLI flags](../installation/cli-flags.md) control extension autolo
--extensions_interval=3
```
`extensions_autoload` points to a line-delimited set of paths to executables. When osquery launches, each path is evaluated for "safe permissions" and executed as a monitored child process. Each executable receives 3 argument switches: `socket`, `timeout`, `interval`. An extension process may use these to find the osquery process's Thrift socket and have hint on retry/backoff configuration if any latency or errors occur.
`extensions_autoload` points to a line-delimited set of paths to executables. When osquery launches, each path is evaluated for "safe permissions" and executed as a monitored child process. Each executable receives 3 argument switches: `socket`, `timeout`, `interval`. An extension process may use these to find the osquery process's Thrift socket, as well as hints on retry/backoff configuration if any latency or errors occur.
The simplest `extensions.load` file contains a single extension path:
```sh
@ -22,29 +22,31 @@ $ file /usr/lib/osquery/extensions/fb_osquery.ext
/usr/lib/osquery/extensions/fb_osquery.ext: ELF 64-bit LSB executable
```
The autoload workflow is similar to:
The *autoload* workflow is similar to:
- Check if extensions are enabled.
- Read `--extensions_autoload` and check permissions/ownership of each path.
- Fork and exec each path with a few well-known switches.
- Treat each child process as a "worker" and enforce memory/cycle usage.
- Fork and execute each path with the switches described above.
- Treat each child process as a "worker" and enforce sane memory/cycle usage.
- Read set config plugin from `--config_plugin`.
- If the config plugin does not exist and at least 1 extension was autoload:
- Wait `--extensions_timeout` * `--extensions_interval` for the extension to register the config plugin.
- Wait `--extensions_timeout * --extensions_interval` for the extension to register the config plugin.
- Fail if the plugin is not registered or the plugin returns a failed status.
The same dependency check is applied to the logger plugin setting after a valid config is read. Every registered plugin is available throughout the run of osqueryd or osqueryi.
The same dependency check is applied to the logger plugin setting after a valid config is read. Every registered plugin is available throughout the run of the shell or daemon.
## More Options
Extensions are most useful when used to expose config or logger plugins. Along with autoloading extensions you can start osqueryd services with non-default plugins using `--flagfile=PATH`. The osqueryd init service on Linux searches for a `/etc/osquery/osquery.flags` path containing flags. This is a great place to add non-default extensions options or for replacing plugins:
Extensions are most useful when used to expose config or logger plugins. Along with auto-loading extensions, you can start daemon services with non-default plugins using `--flagfile=PATH`. The `osqueryd` initscript or SystemV service on Linux searches for a `/etc/osquery/osquery.flags` path containing flags. This is a great place to add non-default extensions options or for replacing plugins:
```sh
$ cat /etc/osquery/osquery.flags
--config_plugin=configerator
--config_plugin=custom_plugin
--logger_plugin=scribe
```
## Modules
The osquery extensions concept builds on the platform's concept of a "registry" of plugin types and plugins therein. The registry maintains a lookup of each plugin and its origin, internal or a transient UUID assigned to an extension. osquery supports extensions as dynamic loadable objects too.
The CLI flag(s):
@ -53,6 +55,6 @@ The CLI flag(s):
--modules_autoload=/etc/osquery/modules.load
```
work the same as extensions, each path is evaluated for safe permission and ownership, and `dlopen`ed when an osquery process starts. There is example code for writing a loaded module in the *./examples* folder. If you are building extensions using the osquery build process a module may be a better option.
work the same as extensions, each path is evaluated for safe permission and ownership, and `dlopen`ed when an osquery process starts. There is example code for writing a loaded module in the [`osquery/examples`](https://github.com/facebook/osquery/tree/master/osquery/examples) folder. If you are building extensions using the osquery build process a module may be a better option.

57
docs/wiki/development/osquery-sdk.md
View File

@ -57,13 +57,66 @@ Extensions use osquery's [thrift API](https://github.com/facebook/osquery/blob/m
The osqueryi or osqueryd processes start an "extension manager" thrift service thread that listens for extension register calls on a UNIX domain socket. Extensions may only communicate if the processes can read/write to this socket. An extension process running as a non-privileged user cannot register plugins to an osqueryd process running as root. Both osqueryi/osqueryd and C++ extensions using `startExtension` will deregister plugins if the communication becomes latent. Both are configurable using gflags or config options.
### Using the example extension
Please see the deployment [guide on extensions](../deployment/extensions.md) for a more-complete overview of how and why extensions are used.
If you [build from source](../development/building.md), you will build an example extension. The code can be found in the [`osquery/examples`](https://github.com/facebook/osquery/blob/master/osquery/examples/example_extension.cpp) folder; it adds a config plugin called "example" and additional table called "example". There are two ways to run an extension: load the extension at an arbitrary time after shell or daemon execution, or request an "autoload" of extensions. The auto-loading method has several advantages such as dependencies on external config plugins, and the same management and process monitoring applied to osquery worker processes.
To load the example extension in the shell try:
```
$ ./build/darwin/osquery/osqueryi
osquery> select path from osquery_extensions;
+-------------------------------------+
| path |
+-------------------------------------+
| /Users/USERNAME/.osquery/shell.em |
+-------------------------------------+
osquery> ^Z
[1] + 98777 suspended ./build/darwin/osquery/osqueryi
```
Here we have started a shell process, inspected the UNIX domain socket path used for extensions, and suspended the process temporarily.
```
$ ./build/darwin/osquery/example_extension.ext --help
osquery 1.7.0, your OS as a high-performance relational database
Usage: osqueryi [OPTION]...
osquery extension command line flags:
--interval VALUE Seconds delay between connectivity checks
--socket VALUE Path to the extensions UNIX domain socket
--timeout VALUE Seconds to wait for autoloaded extensions
osquery project page <https://osquery.io>.
$ ./build/darwin/osquery/example_extension.ext --socket /Users/USERNAME/.osquery/shell.em &
```
Before executing the extension we've inspected the potential CLI flags, which are a subset of the shell or daemon's [CLI flags](../installation/cli-flags.md). The example extension is executed in the background so we can resume the shell and use the provided 'example' table.
```
[2] 98795
$ fg
[1] - 98777 continued ./build/darwin/osquery/osqueryi
osquery> select * from example;
+--------------+-----------------+
| example_text | example_integer |
+--------------+-----------------+
| example | 1 |
+--------------+-----------------+
osquery>
```
If the responsible shell or daemon process ends the extension will soon after detect the loss of communication and also shutdown. Read more about the lifecycle of extensions in the deployment guide.
## Thrift API
[Thrift](https://thrift.apache.org/) is a code-generation/cross-language service development framework. osquery uses thrift to allow plugin extensions for config retrieval, log export, table implementations, event subscribers, and event publishers. We also use thrift to wrap our SQL implementation using SQLite.
**Extension API**
An extension process should implement the following API. During an extension's set up it will "broadcast" all the registered plugins to an osqueryi or osqueryd process. Then the extension will be asked to start a UNIX domain socket and thrift service thread implementing the `ping` and `call` methods.
An extension process should implement the following API. During an extension's set up it will "broadcast" all the registered plugins to a shell or daemon process. Then the extension will be asked to start a UNIX domain socket and thrift service thread implementing the `ping` and `call` methods.
```thrift
service Extension {
@ -80,7 +133,7 @@ service Extension {
}
```
When an extension becomes unavailable, the osqueryi or osqueryd process will automatically deregister those plugins.
When an extension becomes unavailable, the shell or daemon process will automatically deregister those plugins.
**Extension Manager API (osqueryi/osqueryd)**