- The most active NIC will be picked even if a formerly more active
interface still exists (previously, a NIC would stay primary as long
as it existed).
- Ignore link-local and loopback interfaces when choosing the primary.
- Fix bugs in which update status of the primary interface could be
reported incorrectly.
Fixes#2020
- Refactor configuration for logging to use separate plugins
- Move existing filesystem logging to filesystem plugin
- Create new AWS firehose plugin
- Update documentation around logging
Almost two years ago, we began referring to the project as Fleet, but there are
many occurences of the term "Kolide" throughout the UI and documentation. This
PR attempts to clear up those uses where it is easily achievable.
The term "Kolide" is used throughout the code as well, but modifying this would
be more likely to introduce bugs.
This PR adds support for the SMTP LOGIN authentication method. Office 365 Exchange removed support for PLAIN authentication some time ago, and only supports LOGIN and an OAuth2 authentication method. This patch has been tested with a licensed O365 account. This method should also be usable with any other email server that advertises LOGIN in its 250-AUTH response.
Note: If using this with O365, the account used must not have MFA enabled.
Closes#1663
- Delete duplicate queries in packs created by the UI (because the duplicates
were causing undefined behavior). Now it is not possible to schedule
duplicates in the UI (but is in fleetctl).
- Fix bug in which packs created in UI could not be loaded by fleetctl.
Replaces the UI endpoints for creating and modifying labels. These were removed
in #1686 because we thought we were killing the UI.
Now labels can be created and edited in the UI again.
Replaces (and appropriately refactors) a number of endpoints that were removed long ago when we decided to kill the UI with the fleetctl release. We turned out not to do this, and now need to restore these missing endpoints.
This is not a straight up replacement of the existing code because of refactoring to the DB schemas that was also done in the migration.
Most of the replaced code was removed in #1670 and #1686.
Fixes#1811, fixes#1810
This PR adds support for getting resources by name.
```
$ fleetctl get queries
no queries found
$ fleetctl apply -f ./query.yaml
[+] applied 1 queries
$ fleetctl get queries
+-----------------+--------------------------------+--------------------------------+
| NAME | DESCRIPTION | QUERY |
+-----------------+--------------------------------+--------------------------------+
| osquery_version | The version of the Launcher | select launcher.version, |
| | and Osquery process | osquery.version from |
| | | kolide_launcher_info launcher, |
| | | osquery_info osquery; |
+-----------------+--------------------------------+--------------------------------+
$ fleetctl get query osquery_version
apiVersion: v1
kind: query
spec:
description: The version of the Launcher and Osquery process
name: osquery_version
query: select launcher.version, osquery.version from kolide_launcher_info launcher,
osquery_info osquery;
```
With the UI, deleting by ID made sense. With fleetctl, we now want to delete
by name. Transition only the methods used for spec related entities, as others
will be removed soon.
Previously decorators were stored in a separate table. Now they are stored
directly with the config so that they can be modified on a per-platform basis.
Delete now unused decorators code.
- Add new Apply spec methods for queries and packs
- Remove now extraneous datastore/service methods
- Remove import service (unused, and had many dependencies that this breaks)
- Refactor tests as appropriate
After discussion with @groob and @marpaia, we have decided that the service
methods should not be aware of any YAML/JSON definitions, and should work
directly with objects. The new pattern we will use will involve converting YAML
to JSON at the client, and then sending the JSON which will be decoded using
the familiar go-kit mechanisms before being passed to the service methods.
Instead of trying to decode and re-encode status logs, we now write them directly as they come in.
This change prevents future changes to the osquery status log file format (addition and deletion of fields ) from
affecting Fleet. A similar change was implemented in #1636 for result logs.
Closes#1664
Initially fleet decoded the incoming JSON sent to the log endpoint.
Then the log event would be written to a log writer by calling json.Encoder{}.Encode.
Re-encoding logs is lossy; whenever a new field is sent by osqueryd we don't keep up with them.
Instead of caring about the content of the OsqueryResultLog, fleet will now write all log results
exactly as sent to the server by osqueryd.
Closes#1632Closes#1615
This PR adds support for file integrity monitoring. This is done by providing a simplified API that can be used to PATCH/GET FIM configurations. There is also code to build the FIM configuration to send back to osquery. Each PATCH request, if successful, replaces Fleet's existing FIM configuration. For example:
curl -X "PATCH" "https://localhost:8080/api/v1/kolide/fim" \
-H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzZXNzaW9uX2tleSI6IkVhaFhvZWswMGtWSEdaTTNCWndIMnhpYWxkNWZpcVFDR2hEcW1HK2UySmRNOGVFVE1DeTNTaUlFWmhZNUxhdW1ueFZDV2JiR1Bwdm5TKzdyK3NJUzNnPT0ifQ.SDCHAUA1vTuWGjXtcQds2GZLM27HAAiOUhR4WvgvTNY" \
-H "Content-Type: application/json; charset=utf-8" \
-d $'{
"interval": 500,
"file_paths": {
"etc": [
"/etc/%%"
],
"users": [
"/Users/%/Library/%%",
"/Users/%/Documents/%%"
],
"usr": [
"/usr/bin/%%"
]
}
}'
Closes issue #1475
The command line tool that uses this endpoint -> https://github.com/kolide/configimporter
* Added support for atomic imports and dry run imports
* Added code so that imports are idempotent
Closes issue #1456 This PR adds a single sign on option to the login form, exposes single sign on to the end user, and allows an admin user to set single sign on configuration options.
Closes#1502. This PR adds support for SSO to the new user creation process. An admin now has the option to select SSO when creating a new user. When the confirmation form is submitted, the user is automatically authenticated with the IDP, and if successful, is redirected to the Kolide home page. Password authentication, password change and password reset are not allowed for an SSO user.
This PR partially addresses #1456, providing SSO SAML support. The flow of the code is as follows.
A Kolide user attempts to access a protected resource and is directed to log in.
If SSO identity providers (IDP) have been configured by an admin, the user is presented with SSO log in.
The user selects SSO, which invokes a call the InitiateSSO passing the URL of the protected resource that the user was originally trying access. Kolide server loads the IDP metadata and caches it along with the URL. We then build an auth request URL for the IDP which is returned to the front end.
The IDP calls the server, invoking CallbackSSO with the auth response.
We extract the original request id from the response and use it to fetch the cached metadata and the URL. We check the signature of the response, and validate the timestamps. If everything passes we get the user id from the IDP response and use it to create a login session. We then build a page which executes some javascript that will write the token to web local storage, and redirect to the original URL.
I've created a test web page in tools/app/authtest.html that can be used to test and debug new IDP's which also illustrates how a front end would interact with the IDP and the server. This page can be loaded by starting Kolide with the environment variable KOLIDE_TEST_PAGE_PATH to the full path of the page and then accessed at https://localhost:8080/test
This PR contains a fix for a bug that turned up when I was testing configimporter. If the platform field is not specified, its supposed to default to all per the osquery configuration spec. The default was not properly implemented, and if the platform value was missing from the imported configuration it failed. The PR also added instructions to the api documentation describing how to import an osquery configuration.
Replaces the existing calculation that uses a global online interval. This method was lacking due to the fact that different hosts may have different checkin intervals set.
The new calculation uses `min(distributed_interval, config_tls_refresh) + 30` as the interval. This is calculated with the stored values for each host.
Closes#1321
Partially addresses #1456. This PR provides datastore support for SSO by creating a new entity IdentityProvider. This entity is an abstraction of the SAML IdentityProvider and contains the data needed to perform SAML authentication.
We now track the `config_tls_refresh`, `distributed_interval` and
`logger_tls_period` flag values for each host. Each value is updated by a
detail query agains the `osquery_flags` table, because they may be specified
outside of Kolide. The flags that can be specified within Kolide are also
updated when a config is returned to the host that changes their value.
This will enable us to do a more accurate per-host online status calculation as
discussed in #1419.
Closes issue #1388. The problem here is that previously, the reset button loaded a hard coded list of default options into the component state, instead of the proper behavior which is to reset the options to default values on the back end, and then load them back into the redux store. This PR adds a ResetOptions endpoint on the server, and wires up the UI so that it triggers the endpoint, then loads the default options from the backend server.
Push the calculation of target counts into the SQL query, rather than loading
all of the targets and then counting them. This provides a dramatic (>100x)
speedup in loading of the manage packs page when large numbers of hosts are
present.
Closes#1426
