Commit Graph

85 Commits

Author SHA1 Message Date
Mike Arpaia
704ddd424b Host summary endpoint (#742)
* Initial scaffolding of the host summary endpoint

* inmem datastore implementation of GenerateHostStatusStatistics

* HostSummary docstring

* changing the url of the host summary endpoint

* datastore tests for GenerateHostStatusStatistics

* MySQL datastore implementation of GenerateHostStatusStatistics

* <= and >= to catch exact time edge case

* removing clock interface method

* lowercase error wraps

* removin superfluous whitespace

* use updated_at

* adding a seen_at column to the hosts table

* moving the update of seen_time to the caller

* using db.Get instead of db.Select
2017-01-04 14:16:17 -07:00
Victor Vrantchan
046f75295e consolidate delete operations in mysql store (#746)
Adds a helper method which soft deletes entities from the database.
2017-01-04 13:18:21 -05:00
Mike Arpaia
0122f6cb0a Add host_ids and label_ids fields to the packs API (#737)
This PR adds the `host_ids` and `label_ids` field to the packs HTTP API so that one can operate on the hosts/labels which a pack is scheduled to be executed on. This replaces (and deletes) the `/api/v1/kolide/packs/123/labels/456` API in favor of `PATCH /api/v1/packs/123` and specifying the `label_ids` field. This also allows for bulk operations.

Consider the following API examples:

## Creating a pack with a known set of hosts and labels

The key addition is the `host_ids` and `label_ids` field in both the request and the response.

### Request

```
POST /api/v1/kolide/packs
```

```json
{
	"name": "My new pack",
	"description": "The newest of the packs",
	"host_ids": [1, 2, 3],
	"label_ids": [1, 3, 5]
}
```

### Response

```json
{
	"pack": {
		"id": 123,
		"name": "My new pack",
		"description": "The newest of the packs",
		"platform": "",
		"created_by": 1,
		"disabled": false,
		"query_count": 0,
		"total_hosts_count": 5,
		"host_ids": [1, 2, 3],
		"label_ids": [1, 3, 5]
	}
}
```

## Modifying the hosts and/or labels that a pack is scheduled to execute on

### Request

```
PATCH /api/v1/kolide/packs/123
```

```json
{
	"host_ids": [1, 2, 3, 4, 5],
	"label_ids": [1, 3, 5, 7]
}
```

### Response

```json
{
	"pack": {
		"id": 123,
		"name": "My new pack",
		"description": "The newest of the packs",
		"platform": "",
		"created_by": 1,
		"disabled": false,
		"query_count": 0,
		"total_hosts_count": 5,
		"host_ids": [1, 2, 3, 4, 5],
		"label_ids": [1, 3, 5, 7]
	}
}
```

close #633
2017-01-03 10:32:06 -07:00
John Murphy
d1ef37b92c Osquery Options - /config/osquery-options #365 (#729) 2016-12-31 11:56:54 -06:00
Mike Arpaia
9a146c5f92 Renaming targets_count to be more clear (#733) 2016-12-30 12:30:34 -07:00
Mike Arpaia
e9aa5bcc8c Return query_count and targets_count in all pack responses (#731) 2016-12-30 12:06:54 -07:00
Victor Vrantchan
154200db8a Add endpoint to retrieve an invite with the invite token. (#719)
Closes #579
2016-12-29 20:58:12 -05:00
Victor Vrantchan
6cb1026d86 always return errors to the client as a map slice (#724)
keep the format for error returns consistent by always returning a
[]map[string]string for json errors. This simplifies the error handling
on the frontend.

Use "name":"base" as the name field for errors which do not have
a specific or known form field.
2016-12-29 19:40:12 -05:00
Victor Vrantchan
a13042e11b Redirect frontend routes to setup if setup is not configured. (#721)
Closes #617
2016-12-29 18:36:36 -05:00
John Murphy
eec835a07a Osquery options 365 (#657) 2016-12-29 12:32:28 -06:00
Victor Vrantchan
52580ff872 Add invite email template and use the new invite mailer pattern (#711)
Closes #693
Closes #581
2016-12-28 11:55:03 -05:00
Zachary Wasserman
39ebd81dc5 Close orphaned distributed query campaign after failed publish attempt (#707)
A distributed query campaign can be "orphaned" (left in the QueryRunning state)
if the Kolide server restarts while it is running, or other weirdness occurs.
When this happens, no subscribers are waiting to read results written by
osqueryd agents, but the agents continue to receive the query. Previously, this
would cause us to error on ingestion.

The new behavior will instead set the campaign to completed when it detects
that it is orphaned. This should prevent sending queries for which there is no
subscriber.

- New NoSubscriber error interface in pubsub
- Detect NoSubscriber errors and close campaigns
- Tests on pubsub and service methods

Fixes #695
2016-12-27 10:35:19 -05:00
Victor Vrantchan
f092c614cf add error logging to distributed query campaign handler (#694) 2016-12-22 20:31:45 -05:00
Zachary Wasserman
2c7fc2a6f2 Minor refactoring in HostStatus function (#692)
- Refactor function
- Add test
2016-12-22 13:32:40 -08:00
Victor Vrantchan
3658e4833e add prometheus metrics to every http endpoint in the app (#680)
tracking the following metrics:

http_request_duration_microseconds
http_request_size_bytes
http_response_size_bytes
http_requests_total
2016-12-22 12:39:44 -05:00
Victor Vrantchan
e7e57ddbc0 prevent password reuse when changing passwords (#678)
For #375
Closes #448
2016-12-22 12:08:29 -05:00
John Murphy
d653cdf281 Minor App Config API tweaks requested by @mikestone14 (#681)
* Minor App Config API tweaks requested by @mikestone14

* Refactored mail test into separate method, implemented code review changes
2016-12-22 08:12:34 -06:00
Victor Vrantchan
8ef11f51bf create user from invite: set admin property from invite. (#675)
Closes #589
2016-12-21 12:34:10 -05:00
John Murphy
62748bfbb1 App Settings - /admin/settings #363 (#590) 2016-12-20 15:54:30 -06:00
Victor Vrantchan
41610c1f66 catch sql.ErrNoRows in mysql/invites (#667) 2016-12-20 16:31:09 -05:00
Victor Vrantchan
9d49dbc465 change the implementation of ErrNotFound and AlreadyExists to a struct type (#665)
with an exposed interface.

Not checking for a specific sentinel error reduces coupling between packages
and allows adding context like the resource ID and resource type.
2016-12-20 13:35:22 -05:00
Zachary Wasserman
41120ebc00 Improve user endpoint validations (#642)
- Add empty string checks for NewUser
- Create validations for ModifyUser
- Use GravatarURL when creating new user

Fixes #620
2016-12-15 09:28:53 -08:00
Mike Arpaia
1ada011f37 Remove existing bulk functionality (#641) 2016-12-14 17:22:17 -08:00
Zachary Wasserman
0e783e7887 Add change password endpoint (#628) 2016-12-14 10:11:43 -08:00
Mike Arpaia
bed279b919 Remove "selected" keyword from response (#635) 2016-12-13 15:59:59 -08:00
Mike Arpaia
f109b14f9d Moving query attributes from the query object to the pack-query relationship (#559)
* Moving query attributes from the query object to the pack-query relationship

* some additional tests

* http request parsing test

* QueryOptions in new test_util code

* initial scaffolding of new request structures

* service and datastore

* test outline

* l2 merge conflict scrub

* service tests for scheduled query service

* service and datastore tests

* most endpoints and transports

* order of values are not deterministic with inmem

* transport tests

* rename PackQuery to ScheduledQuery

* removing existing implementation of adding queries to packs

* accounting for the new argument to NewQuery

* fix alignment in sql query

* removing underscore

* add removed to the datastore

* removed differential from the schema
2016-12-13 14:22:05 -08:00
Zachary Wasserman
556cbf43bd Use canPerformActions authZ check appropriately in handlers (#625)
Fixes #282
2016-12-13 08:57:49 -08:00
Zachary Wasserman
0c62bda2f1 Fix session expiration logic with password reset (#619)
- Only expire sessions at reset request time when admin forces reset
- Expire sessions when reset completed

Prior to this, there was a possible DoS vector in which an attacker could
prevent a user from taking actions in the app by constantly requesting password
resets and expiring all the user's active sessions.

Fixes #612
2016-12-12 09:24:51 -08:00
Zachary Wasserman
b362682d90 Add service method for bulk deleting queries (#600)
- New datastore method for bulk deletion
- New service method calling this datastore method
- Endpoint, transport and handler connections for service method

Closes #389
2016-12-09 09:12:45 -08:00
Zachary Wasserman
d9190020fe Record author information with queries (#578) 2016-12-07 12:22:31 -08:00
Victor Vrantchan
6d9ad56e6d use validation error for expired invite tokens. (#587)
Closes #586
2016-12-07 11:25:48 -05:00
Victor Vrantchan
62a5e65c2e use random string instead of JWT for tokens. (#584)
uses a random URL encoded base64 string as the token for password reset
and invites.
2016-12-07 10:42:58 -05:00
John Murphy
7812b2f3bd Fixes host detail updates with MySQL backend (#568) 2016-12-07 03:51:11 +08:00
Zachary Wasserman
bc6109fb56 Fixes for saved queries (#576)
- Only saved queries should be returned by ListQueries
- Bugfixes

Addresses #388
2016-12-06 10:16:04 -08:00
John Murphy
e6b1ed9ade Added MIA status for hosts that haven't been updated for 30 days (#570) 2016-12-07 01:37:22 +08:00
Victor Vrantchan
b1bca7c98c enbable API route after setup is complete (#564)
Fixes #563
2016-12-02 13:46:31 -05:00
Zachary Wasserman
b901c4c0d3 Update models/service method for saving queries (#553)
- Add saved state to query (to differentiate queries explicitly saved from
  those just run as distributed queries)
- Remove unique constraint on query name

Closes #390
2016-12-01 13:21:27 -08:00
Zachary Wasserman
e1db2d4c27 Add cleanup method for distributed queries and metadata (#547)
- New datastore method for expiring queries and deleting metadata
- Start goroutine in serve to run this method on an hourly interval

Closes #392
2016-12-01 10:31:16 -08:00
John Murphy
f66904f240 Host details 405 (#535) 2016-12-02 01:00:00 +08:00
Zachary Wasserman
ed3c696c6e Refactor campaign methods into separate file (#540) 2016-11-29 10:20:06 -08:00
John Murphy
44ef92550f Added built in labels (#526) 2016-11-26 02:08:22 +08:00
Zachary Wasserman
0b612eedab Push query results and metadata over websocket connection (#519)
- New route `/api/v1/kolide/results/{id}` with upgrade to websocket connection
- Query results pushed over websocket as they are received from pubsub
- Target totals updates pushed over websocket every second
- New datastore method to support retrieiving target totals
- Websocket package includes helpers and patterns for communicating over websockets
2016-11-22 16:35:43 -08:00
Mike Arpaia
a036c5da9f Add attributes to packs (#524)
* Adds created_by attribute to packs

This PR also updated the distributed query code to use the pattern
established here (service checks context)

* add enable/disable state to packs

* add query_count to packs API responses

* add host_count to packs API responses (very, very poorly)

* pack description should not be required

* counting hosts in packs via mysql

* removing extraneous newline in test

* Switch case instead of if/if else

* add description to update query for SavePack method

* change AND to WHERE in query as per @zwass

* add ordering and list options as per @murphybytes' suggestion
2016-11-22 13:56:05 -08:00
Mike Arpaia
a8a7be7f20 Packs pages (#426)
* initial scaffolding

* pack info sidebar

* fixing the merge of the routes

* Remove radium from pack info sidepanel

* lint

* cards!

* redux entity config

* pack interface

* wiring up redux with fake dev data

* Add description attribute to packs

* move redux to top level page component to isolate data fetching

* initial scaffolding of all packs table

* adding redux entities back

* minimal

* alpha order in packs.js

* no newlines in HTML

* onclick handler to function on component class

* alpha order in router

* alpha order in paths.js

* no newline in side panel

* removing input field

* lint fixes
2016-11-21 11:49:36 -08:00
Zachary Wasserman
3cd841051b Add online count to target/label responses (#513)
Closes #508
2016-11-18 09:23:44 -08:00
Zachary Wasserman
34625ce4d0 Add service method/endpoint for creating distributed query campaign (#485) 2016-11-16 13:07:50 -08:00
John Murphy
6a825c11e3 Datastore refactor (#439)
Removed Gorm, replaced it with Sqlx

* Added SQL bundling command to Makfile

* Using go-kit logger

* Added soft delete capability

* Changed SearchLabel to accept a variadic param for optional omit list
instead of array

* Gorm removed

* Refactor table structures to use CURRENT_TIMESTAMP mysql function

* Moved Inmem datastore into it's own package

* Updated README

* Implemented code review suggestions from @zwass

* Removed reference to Gorm from glide.yaml
2016-11-16 21:47:49 +08:00
Mike Arpaia
752710ccf1 Return count of hosts in targets search for each label (#480) 2016-11-14 12:10:28 -08:00
Zachary Wasserman
9c38d6d19e Update osquery service methods for distributed queries (#476) 2016-11-14 10:22:54 -08:00
Zachary Wasserman
5d8a42a959 Fix page in RequireSetup check (#465)
The previous value would require setup unless there were at least 2 users. Now
one user is sufficient to proceed past setup.
2016-11-09 16:35:49 -08:00