Commit Graph

243 Commits

Author SHA1 Message Date
Zachary Wasserman
691eb55cf2 Return packs with queries (#575)
- New datastore method for loading packs associated with a query
- ListQueries and Query datastore methods now load packs

Addresses #388
2016-12-06 10:22:28 -08:00
Zachary Wasserman
bc6109fb56 Fixes for saved queries (#576)
- Only saved queries should be returned by ListQueries
- Bugfixes

Addresses #388
2016-12-06 10:16:04 -08:00
Zachary Wasserman
51e59712da Fixes in targets search (#565)
- Remove string formatting in favor of bindvars
- Fix errors when the query is empty
2016-12-05 11:16:23 -08:00
Zachary Wasserman
b901c4c0d3 Update models/service method for saving queries (#553)
- Add saved state to query (to differentiate queries explicitly saved from
  those just run as distributed queries)
- Remove unique constraint on query name

Closes #390
2016-12-01 13:21:27 -08:00
Zachary Wasserman
e1db2d4c27 Add cleanup method for distributed queries and metadata (#547)
- New datastore method for expiring queries and deleting metadata
- Start goroutine in serve to run this method on an hourly interval

Closes #392
2016-12-01 10:31:16 -08:00
John Murphy
f66904f240 Host details 405 (#535) 2016-12-02 01:00:00 +08:00
Zachary Wasserman
ed3c696c6e Refactor campaign methods into separate file (#540) 2016-11-29 10:20:06 -08:00
Victor Vrantchan
a9000c5af3 print connection sleep interval duration in seconds (#534)
The duration was printed with a "ns" interval, even
though the sleep time was in seconds.
2016-11-28 10:35:05 -05:00
John Murphy
44ef92550f Added built in labels (#526) 2016-11-26 02:08:22 +08:00
Zachary Wasserman
0b612eedab Push query results and metadata over websocket connection (#519)
- New route `/api/v1/kolide/results/{id}` with upgrade to websocket connection
- Query results pushed over websocket as they are received from pubsub
- Target totals updates pushed over websocket every second
- New datastore method to support retrieiving target totals
- Websocket package includes helpers and patterns for communicating over websockets
2016-11-22 16:35:43 -08:00
Mike Arpaia
a036c5da9f Add attributes to packs (#524)
* Adds created_by attribute to packs

This PR also updated the distributed query code to use the pattern
established here (service checks context)

* add enable/disable state to packs

* add query_count to packs API responses

* add host_count to packs API responses (very, very poorly)

* pack description should not be required

* counting hosts in packs via mysql

* removing extraneous newline in test

* Switch case instead of if/if else

* add description to update query for SavePack method

* change AND to WHERE in query as per @zwass

* add ordering and list options as per @murphybytes' suggestion
2016-11-22 13:56:05 -08:00
Zachary Wasserman
627b1ad6e6 Drop MySQL tables after test when migration fails (#517) 2016-11-18 11:14:10 -08:00
John Murphy
531535c168 Refactored database migrations to use goose (#515)
* Refactored database migrations to use goose
2016-11-19 01:02:51 +08:00
Zachary Wasserman
eff7f60dc2 Fixes for bugs in MySQL migration (#501)
* Fix users table name in MySQL ListUsers
* Fix invalid SQL
* Implement MarkHostSeen
* Partial fix for targets autocompletion
2016-11-16 15:12:59 -08:00
Zachary Wasserman
34625ce4d0 Add service method/endpoint for creating distributed query campaign (#485) 2016-11-16 13:07:50 -08:00
John Murphy
6a825c11e3 Datastore refactor (#439)
Removed Gorm, replaced it with Sqlx

* Added SQL bundling command to Makfile

* Using go-kit logger

* Added soft delete capability

* Changed SearchLabel to accept a variadic param for optional omit list
instead of array

* Gorm removed

* Refactor table structures to use CURRENT_TIMESTAMP mysql function

* Moved Inmem datastore into it's own package

* Updated README

* Implemented code review suggestions from @zwass

* Removed reference to Gorm from glide.yaml
2016-11-16 21:47:49 +08:00
Zachary Wasserman
0b6df0cd73 Fix changed ID column name in DistributedQueriesForHost query (#473) 2016-11-10 17:01:39 -08:00
Zachary Wasserman
af6a5ee1b6 Add datastore methods for distributed queries (#458)
New datastore methods are introduced for creating/updating
distributed query campaigns, as well as determining the active
distributed queries for a given host.
2016-11-09 15:33:16 -08:00
Zachary Wasserman
c8b9b15ef2 Fix failing MySQL tests for packs (#457) 2016-11-09 15:31:47 -08:00
Zachary Wasserman
991fd6c146 Ensure tables are dropped after panic in test (#456)
Prior to this, a panic in the test function would leave data in the database.
Now we ensure that tables are dropped even after a panic.
2016-11-08 13:46:36 -08:00
Victor Vrantchan
def24499b5 store WebAddress config in datastore (#421)
moves web address config to datastore so that it can be configured by a user
in the Web UI.
rename OrgInfo struct to AppConfig.

For #363
For #378
2016-11-04 16:44:38 -04:00
Mike Arpaia
97de2bc5aa Label API Updates (#413)
close #403
close #404
close #412
2016-11-02 21:17:23 -04:00
Mike Arpaia
7ebebbb7b1 Target search endpoint (#339) 2016-11-02 10:59:53 -04:00
Zachary Wasserman
262a48f8eb Datastores for buffering distributed query results (#346)
A new datastore interface is needed for buffering incoming distributed query results to be sent to the client. This PR attempts to define and implement that interface.

It is intended that the ReadChannel() method be used by the goroutine that will push query results down a websocket to the client. Passing the results through this channel will allow that goroutine to perform a select on both the channel and the websocket, in order to properly handle IO.
2016-10-31 15:51:19 -07:00
Victor Vrantchan
950586c100 update datastore logger (#344)
remove dependency on logrus
adapt go-kit/log interface to gorm logger
allow debug logging flag
2016-10-25 11:30:14 -04:00
Victor Vrantchan
efffe50a55 update mysql test setup (#343)
reuse a single conneciton pool
close connection gracefully at the end of tests
setup and teardown tables in between each test
2016-10-25 11:01:14 -04:00
Mike Arpaia
bdee0516e6 Removing sqlite from the codebase (#324) 2016-10-19 10:46:33 -04:00
Mike Arpaia
98ca32b783 Complete, working inmem datastore implementation (#319) 2016-10-17 15:30:47 -04:00
Zachary Wasserman
05cb15d63c Fix pointer handling in inmem nextID (#323) 2016-10-17 13:11:23 -04:00
Zachary Wasserman
f9bb9de665 Add ordering options for List* methods (#318) 2016-10-17 10:01:14 -04:00
Zachary Wasserman
1597ad23a4 Fix NewHost bug introduced in #312 (#317) 2016-10-16 07:56:20 -04:00
Zachary Wasserman
daeecfb244 Fix bug in inmem ID generation (#312)
Fixes #308
2016-10-14 09:15:04 -07:00
Zachary Wasserman
4b88ae6e2c Apply consistent naming conventions across server files (#310) 2016-10-14 08:59:27 -07:00
Zachary Wasserman
7f636aef4f Add pagination to List* endpoints (#309)
- Introduce kolide.ListOptions to store pagination params (in the future it can
  also store ordering/filtering params)
- Refactor service/datastore methods to take kolide.ListOptions
- Implement pagination
2016-10-13 11:21:47 -07:00
Zachary Wasserman
85a8e92b96 Implement endpoints for host service methods (#280)
- Remove service methods for modifying hosts
2016-10-05 17:10:44 -07:00
Zachary Wasserman
19f6eddfeb Update host details in distributed query ingestion (#274)
- Introduce a new pattern for defining/ingesting detail queries
- Add many relevant host details:
  - Platform
  - osquery Version
  - Memory
  - Hostname
  - UUID
  - OS Version
  - Uptime
  - Primary interface MAC
  - Primary interface IP
- Fix parsing for inconsistent JSON schema returned from osquery
- Tests
2016-10-04 17:17:55 -07:00
Mike Arpaia
12f8c0b671 Datastore cleaning (#262)
This PR reorganizes a bunch of the files in datastore such that all datastore implementations are consistently broken up into multiple files. Additionally, the datastore tests follow a similar pattern and can easily be applied to any complete datastore implementation.
2016-10-04 16:34:36 -04:00
Mike Arpaia
a03347489c Osquery Configuration Control (#244)
Label management APIs and an osquery config endpoint based on active pack and label state.
2016-10-02 20:14:35 -07:00
Zachary Wasserman
74aaa14dde Implement osquery datastore methods for inmem datastore (#255)
- Implement osquery datastore methods
- Update tests for compatibility with inmem

Closes #255
2016-09-29 18:19:51 -07:00
Zachary Wasserman
ba528a46f1 Build endpoints for osquery service methods (#245)
- Establish a pattern for host authentication
- Establish a pattern for error JSON
- Add transport and make endpoint functions
- Fix discovered bugs + update tests
2016-09-28 21:21:39 -07:00
Victor Vrantchan
6fb96d98f7 Adds endpoints to invite new users to the application. (#235)
User service checks that tokens are valid on new user signups.
Closes #230
2016-09-28 22:44:05 -04:00
Zachary Wasserman
a3878f0a3b Add LabelQueriesForHost to OsqueryStore (#242)
Also includes bug fixes and tests for related datastore methods.
2016-09-26 13:05:36 -07:00
Mike Arpaia
0482f12926 Organizing go code (#241) 2016-09-26 11:48:55 -07:00