- Add endpoints for osquery to register and continue a carve.
- Implement client functionality for retrieving carve details and contents in fleetctl.
- Add documentation on using file carving with Fleet.
Addresses kolide/fleet#1714
* Perform migration to delete any entries with `deleted` set, and
subsequently drop columns `deleted` and `deleted_at`.
* Remove `deleted` and `deleted_at` references.
Closes#2146
* Use YAML anchors to avoid repeating config blocks
* Use docker volumes to persist data for mysql
* Allow setting `FLEET_SERVER` (fixes#2127) when using the docker-compose file to spin up multiple osquery clients
Almost two years ago, we began referring to the project as Fleet, but there are
many occurences of the term "Kolide" throughout the UI and documentation. This
PR attempts to clear up those uses where it is easily achievable.
The term "Kolide" is used throughout the code as well, but modifying this would
be more likely to introduce bugs.
Add a gRPC server that will interact with osquery through Launcher. This endpoint will expose the osquery configuration suitable for use via the Launcher plugin, and collect log and query results.
- Add SSH configuration to allow checkout of Kolide private repos in CI
- Add kolide/agent-api repo to glide.yaml
- Update testify version to fix broken test build
Closes#1545
This PR partially addresses #1456, providing SSO SAML support. The flow of the code is as follows.
A Kolide user attempts to access a protected resource and is directed to log in.
If SSO identity providers (IDP) have been configured by an admin, the user is presented with SSO log in.
The user selects SSO, which invokes a call the InitiateSSO passing the URL of the protected resource that the user was originally trying access. Kolide server loads the IDP metadata and caches it along with the URL. We then build an auth request URL for the IDP which is returned to the front end.
The IDP calls the server, invoking CallbackSSO with the auth response.
We extract the original request id from the response and use it to fetch the cached metadata and the URL. We check the signature of the response, and validate the timestamps. If everything passes we get the user id from the IDP response and use it to create a login session. We then build a page which executes some javascript that will write the token to web local storage, and redirect to the original URL.
I've created a test web page in tools/app/authtest.html that can be used to test and debug new IDP's which also illustrates how a front end would interact with the IDP and the server. This page can be loaded by starting Kolide with the environment variable KOLIDE_TEST_PAGE_PATH to the full path of the page and then accessed at https://localhost:8080/test
If server is started without a JWT key, a message like the following is printed:
```
################################################################################
# ERROR:
# A value must be supplied for --auth_jwt_key. This value is used to create
# session tokens for users.
#
# Consider using the following randomly generated key:
# om3w95gMA2drT5xAdLd2Q5oE8fLw+Miz
################################################################################
```
Closes#1480.
On CentOS6 there is a bug in which osquery incorrectly reports an empty string
for platform. This PR fixes our detection of centos in this case.
Fixes#1339
the scripts in tools/pkgrepos can be used to update apt and yum repositories as well as create zip archives for a release.
The dl.kolide.co repository is archived as a static bucket on GCP.
Adds a `make lint-license` command that will crawl through the Golang and
Javascript dependencies, trying to automatically determine the license of each
dependency. If any dependencies have incompatible licensing (or if the
automatic detection fails), the tool will print an error and exit with a
non-zero status code. After a successful run, the
docs/third-party/dependencies.md file will be updated with attribution
information as determined by the crawl.
The configuration file has been bootstrapped with the manual analysis needed for
all of the existing dependencies.
This PR includes:
- `make demo-dump` which dumps all tables from your running development Kolide instance into `./tools/app/demo.sql`
```
$> make demo-dump
mysqldump --extended-insert=FALSE --skip-dump-date \
-u kolide -p \
-h 192.168.99.100 kolide \
> ./tools/app/demo.sql
Enter password: ******
$>
```
- `make demo-restore` which restores `./tools/app/demo.sql` into your MySQL database, to be demo'd and extended by your running Kolide instance
```
$> make demo-restore
mysql --binary-mode -u kolide -p \
-h 192.168.99.100 kolide \
< ./tools/app/demo.sql
Enter password: ******
```
This included demo dump contains the following content:
## Queries
- 34 queries shared as ~70 scheduled queries
## Packs
- 8 Packs
- 13 Scheduled Queries in the "Intrusion Detection" pack
- 7 Scheduled Queries in the "Osquery Monitoring" pack
- 4 Scheduled Queries in the "Asset Management" pack
- 12 Scheduled Queries in the "Hardware Monitoring" pack
- 8 Scheduled Queries in the "Incident Response" pack
- 8 Scheduled Queries in the "Compliance" pack
- 6 Scheduled Queries in the "Vulnerability Management" pack
- 10 Scheduled Queries in the "Systems Monitoring" pack
## Labels
- 2 Labels
## Hosts
There are intentionally no hosts. Live hosts should be fabricated using Docker and attached to a running demo instance, to better look and behave like a real infrastructure.
* No more hard deletes
* scaffolding for password reset endpoint
* Ensure password reset state is accounted for in VC checks
* password reset endpoints and data structures
* ability to change password with reset token
* smtp server connection pool management
* stubbing out the sending of the email
* adding mailhog via docker
* HTML emails with confgurable host name
* fixing typo in the comments
* Fixing merge which undid DatabaseError replacement
* documentation in the readme
* webpack shortcut for components
* removing a sneaky merge line that snuck in
* temporary email content api
* tests for password reset flow
* fixing go vet
* comments and making all db use `&value` rather than `reference`
* more correct usage of the errors library and moving email sending to it's own method
* using the wrong error
* fixing email mock object error
* less incorrect error usage
* rebasing and merging
* http constants for status code
* using ParseAndValidateJSON instead of BindJSON
* validate instead of binding in struct tags
* NewFromError instead of New
* Add sensible React base to the app for frontend
This PR attempts to "reactify" Kolide and provide a sane development environment
that a front-end engineer would probably expect.
This PR accomplishes by doing the following:
1. Reorganizes the app into a `server/` and `client/` folders to keep golang
logic separated from react logic.
2. Adds an "asset pipeline" via webpack which knows how to build a js
and css bundle.
3. Packages up all static assets in a go-bindata file so that the binary
remains portable without external file dependencies.
1. Add a Makefile with several targets that will be common in everyday
development. For example, we have `serve` target which spins up a nodejs
reverse proxy on port 8081 which then watches for changed files, automatically
rebuilds the app, and hot loads the new JS/CSS in.
**Note:** Please use `make` to build the app, not `go build` as there are
now several things that need to be orchestrated beyond the go code to build the app.
* Create build if it doesn't exist, and use `go get`
* Improve README to reflect new dev workflow
* Document css vars and funcs and use alias paths
* makefile and structure modifications
* Sessions in MySQL
* Reclaiming some names
* session renewal without new cookies on every request
* comments and docstrings
* light organization in vc generation
* go vet
* endpoints for session management
* Merging @zwass' commit with mine
* Updating salt generation to use crypt/rand
* use getRandomText for session keys
* VC no longer needs a DB or to return an error
* getRandomText docstring
* Only use session via the SessionBackend API
* Set session backend with the request, similar to db
This commit adds both a Dockerfile and updates the docker-compose.yml with local mounting so that you can standup a consistent dev environment. Please view the project README for more information.
