Commit Graph

20 Commits

Author SHA1 Message Date
Zachary Wasserman
3d6ca7d5a4 Use sockjs to gracefully degrade websockets (#1255)
Use the [SockJS Protocol](https://github.com/sockjs/sockjs-protocol) to handle
bidirectional communication instead of plain websockets. This allows
distributed queries to function in situations in which they previously failed
(Load balancers not supporting websockets, issues with Safari and self-signed
certs, etc.).

Also includes fixes to the JS message handling logic where slightly different
message delivery semantics (when using XHR) were exposing bugs.

Fixes #1241, #1327.
2017-03-01 13:14:26 -08:00
Zachary Wasserman
728c2c3475 Add license linting and dependencies.md (#1184)
Adds a `make lint-license` command that will crawl through the Golang and
Javascript dependencies, trying to automatically determine the license of each
dependency. If any dependencies have incompatible licensing (or if the
automatic detection fails), the tool will print an error and exit with a
non-zero status code. After a successful run, the
docs/third-party/dependencies.md file will be updated with attribution
information as determined by the crawl.

The configuration file has been bootstrapped with the manual analysis needed for
all of the existing dependencies.
2017-02-10 12:51:33 -08:00
Victor Vrantchan
a74063c1d1 update go-kit version and run glide up (#1182) 2017-02-06 20:25:47 -05:00
Mike Arpaia
5cc5220e5b Enforce uniqueness on query name (#915)
* Enforce uniqueness on query name

close #914

* catching the already exists error in MySQL
2017-01-16 15:20:15 -07:00
Zachary Wasserman
35806f1442 Separate tables from data migrations (#745)
This PR separates the table migrations from the data population migrations. Table migrations run before data migrations.

Now, we have the ability to create the database tables without populating them with data. This can be useful for running "unit" tests against a MySQL store that doesn't have any pre-populated data. When performing real migrations, or for more "integration" style testing, the data migrations can also be executed.

Note there are some special cases that must be observed with these migrations, and the README is updated to reflect those.
2017-01-05 09:27:56 -08:00
Zachary Wasserman
0b612eedab Push query results and metadata over websocket connection (#519)
- New route `/api/v1/kolide/results/{id}` with upgrade to websocket connection
- Query results pushed over websocket as they are received from pubsub
- Target totals updates pushed over websocket every second
- New datastore method to support retrieiving target totals
- Websocket package includes helpers and patterns for communicating over websockets
2016-11-22 16:35:43 -08:00
John Murphy
531535c168 Refactored database migrations to use goose (#515)
* Refactored database migrations to use goose
2016-11-19 01:02:51 +08:00
Mike Arpaia
eb3f3cd765 removing some dependencies from glide (#493) 2016-11-16 10:38:31 -08:00
John Murphy
6a825c11e3 Datastore refactor (#439)
Removed Gorm, replaced it with Sqlx

* Added SQL bundling command to Makfile

* Using go-kit logger

* Added soft delete capability

* Changed SearchLabel to accept a variadic param for optional omit list
instead of array

* Gorm removed

* Refactor table structures to use CURRENT_TIMESTAMP mysql function

* Moved Inmem datastore into it's own package

* Updated README

* Implemented code review suggestions from @zwass

* Removed reference to Gorm from glide.yaml
2016-11-16 21:47:49 +08:00
Zachary Wasserman
262a48f8eb Datastores for buffering distributed query results (#346)
A new datastore interface is needed for buffering incoming distributed query results to be sent to the client. This PR attempts to define and implement that interface.

It is intended that the ReadChannel() method be used by the goroutine that will push query results down a websocket to the client. Passing the results through this channel will allow that goroutine to perform a select on both the channel and the websocket, in order to properly handle IO.
2016-10-31 15:51:19 -07:00
Mike Arpaia
bdee0516e6 Removing sqlite from the codebase (#324) 2016-10-19 10:46:33 -04:00
Mike Arpaia
98ca32b783 Complete, working inmem datastore implementation (#319) 2016-10-17 15:30:47 -04:00
Zachary Wasserman
f9bb9de665 Add ordering options for List* methods (#318) 2016-10-17 10:01:14 -04:00
Victor Vrantchan
24b9baec1f add prometheus endpoint (#236)
generate metrics for Users, Appconfig and Session services
2016-09-28 07:35:15 -04:00
Zachary Wasserman
8f16bd8bcc Send configuration + label queries in distributed reads (#215)
This PR is the beginning of distributed query work. For now we are focusing on using the distributed query subsystem to retrieve the basic configuration information (currently just platform), and run the label queries.

A mockable clock interface is also added to the service struct, allowing us to inject a clock as a dependency, and write unit tests that can control the time.
2016-09-20 20:08:11 -07:00
Victor Vrantchan
0bb0ce8888 update glide 2016-08-29 09:10:09 -04:00
Victor Vrantchan
a07e9c7d77 go-kit server layout 2016-08-27 23:59:17 -04:00
Zachary Wasserman
41fe404ef1 HTTP Controller updates for label queries (#96)
Add controller methods for:

* Retrieving label queries
* Storing results of label queries
2016-08-24 16:50:28 -07:00
Zachary Wasserman
503ae54f46 Ingest status and result logs (#77)
* Implement log endpoint for status/result log ingestion
* Define interface for log handlers: OsqueryResultHandler and OsqueryStatusHandler
* Initial implementation of file logger handlers
* Unit + integration tests

Closes #7
2016-08-17 12:45:29 -07:00
Victor Vrantchan
0c51890b30 replace godep in favor of glide and using vendor directory
update make deps step in Makefile
update workflow for managing third party dependencies in README
closes #24, #47
2016-08-15 09:44:10 -04:00