Zach Wasserman
8723b83039
Fix data race in inmem query results test ( #1369 )
...
Missed locking caused a race condition detected with the --race flag:
```
==================
WARNING: DATA RACE
Read at 0x00c0004b2cf0 by goroutine 67:
runtime.mapaccess2_fast64()
/usr/local/Cellar/go/1.16.5/libexec/src/runtime/map_fast64.go:52 +0x0
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).WriteResult()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:37 +0x84
github.com/fleetdm/fleet/v4/server/service.(*Service).ingestDistributedQuery()
/Users/zwass/dev/fleet/server/service/service_osquery.go:1020 +0x258
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQueryRecordCompletionError()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1499 +0x61c
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
Previous write at 0x00c0004b2cf0 by goroutine 104:
runtime.mapassign_fast64()
/usr/local/Cellar/go/1.16.5/libexec/src/runtime/map_fast64.go:92 +0x0
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).getChannel()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:31 +0x1a4
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).ReadChannel()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:53 +0x64
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQueryRecordCompletionError.func1()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1493 +0x117
Goroutine 67 (running) created at:
testing.(*T).Run()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1238 +0x5d7
testing.runTests.func1()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1511 +0xa6
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
testing.runTests()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1509 +0x612
testing.(*M).Run()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1417 +0x3b3
main.main()
_testmain.go:303 +0x236
Goroutine 104 (running) created at:
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQueryRecordCompletionError()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1492 +0x558
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
==================
==================
WARNING: DATA RACE
Read at 0x00c0000ff2d8 by goroutine 67:
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).WriteResult()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:37 +0xa4
github.com/fleetdm/fleet/v4/server/service.(*Service).ingestDistributedQuery()
/Users/zwass/dev/fleet/server/service/service_osquery.go:1020 +0x258
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQueryRecordCompletionError()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1499 +0x61c
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
Previous write at 0x00c0000ff2d8 by goroutine 104:
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).getChannel()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:31 +0x1bc
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).ReadChannel()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:53 +0x64
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQueryRecordCompletionError.func1()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1493 +0x117
Goroutine 67 (running) created at:
testing.(*T).Run()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1238 +0x5d7
testing.runTests.func1()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1511 +0xa6
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
testing.runTests()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1509 +0x612
testing.(*M).Run()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1417 +0x3b3
main.main()
_testmain.go:303 +0x236
Goroutine 104 (running) created at:
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQueryRecordCompletionError()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1492 +0x558
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
==================
--- FAIL: TestIngestDistributedQueryRecordCompletionError (0.01s)
service_osquery_test.go:1502: PASS: QueryCompletedByHost(string,uint)
testing.go:1092: race detected during execution of test
==================
WARNING: DATA RACE
Read at 0x00c0000f8570 by goroutine 70:
runtime.mapaccess2_fast64()
/usr/local/Cellar/go/1.16.5/libexec/src/runtime/map_fast64.go:52 +0x0
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).WriteResult()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:37 +0x84
github.com/fleetdm/fleet/v4/server/service.(*Service).ingestDistributedQuery()
/Users/zwass/dev/fleet/server/service/service_osquery.go:1020 +0x258
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQuery()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1530 +0x5bc
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
Previous write at 0x00c0000f8570 by goroutine 71:
runtime.mapassign_fast64()
/usr/local/Cellar/go/1.16.5/libexec/src/runtime/map_fast64.go:92 +0x0
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).getChannel()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:31 +0x1a4
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).ReadChannel()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:53 +0x64
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQuery.func1()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1524 +0x117
Goroutine 70 (running) created at:
testing.(*T).Run()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1238 +0x5d7
testing.runTests.func1()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1511 +0xa6
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
testing.runTests()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1509 +0x612
testing.(*M).Run()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1417 +0x3b3
main.main()
_testmain.go:303 +0x236
Goroutine 71 (running) created at:
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQuery()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1523 +0x4f4
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
==================
==================
WARNING: DATA RACE
Read at 0x00c000c480d8 by goroutine 70:
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).WriteResult()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:37 +0xa4
github.com/fleetdm/fleet/v4/server/service.(*Service).ingestDistributedQuery()
/Users/zwass/dev/fleet/server/service/service_osquery.go:1020 +0x258
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQuery()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1530 +0x5bc
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
Previous write at 0x00c000c480d8 by goroutine 71:
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).getChannel()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:31 +0x1bc
github.com/fleetdm/fleet/v4/server/pubsub.(*inmemQueryResults).ReadChannel()
/Users/zwass/dev/fleet/server/pubsub/inmem_query_results.go:53 +0x64
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQuery.func1()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1524 +0x117
Goroutine 70 (running) created at:
testing.(*T).Run()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1238 +0x5d7
testing.runTests.func1()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1511 +0xa6
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
testing.runTests()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1509 +0x612
testing.(*M).Run()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1417 +0x3b3
main.main()
_testmain.go:303 +0x236
Goroutine 71 (running) created at:
github.com/fleetdm/fleet/v4/server/service.TestIngestDistributedQuery()
/Users/zwass/dev/fleet/server/service/service_osquery_test.go:1523 +0x4f4
testing.tRunner()
/usr/local/Cellar/go/1.16.5/libexec/src/testing/testing.go:1193 +0x202
==================
--- FAIL: TestIngestDistributedQuery (0.01s)
service_osquery_test.go:1532: PASS: QueryCompletedByHost(string,uint)
testing.go:1092: race detected during execution of test
FAIL
FAIL github.com/fleetdm/fleet/v4/server/service 42.743s
```
2021-07-13 12:27:41 -07:00
Zach Wasserman
c5280c0517
Add v4 suffix in go.mod ( #1224 )
2021-06-25 21:46:51 -07:00
Zach Wasserman
fb32f0cf40
Remove kolide types and packages from backend ( #974 )
...
Generally renamed `kolide` -> `fleet`
2021-06-06 15:07:29 -07:00
Mike Arpaia
af96e52a00
Update the Go import paths to new repo name ( #27 )
2020-11-11 09:59:12 -08:00
Zachary Wasserman
e8e4bc9879
Fix cleanup of orphaned queries ( #2316 )
...
The expiration logic was incorrect leading to queries not being cleaned
up properly. Tests added for the whole subroutine.
Fixes #2302
2020-10-06 09:30:24 -07:00
Zachary Wasserman
1eccf9a874
Add warning in query UI when Redis fails ( #2086 )
...
- Add warning message when Redis fails
- Disable query button when Redis fails
- Refactor SMTP warning banner into component for reuse
Closes #2073
2019-08-13 09:42:58 -07:00
Mike Arpaia
018b91ab2c
Rename project to Kolide Fleet ( #1529 )
2017-06-22 15:50:45 -04:00
Zachary Wasserman
715d908613
Update go-kit to 0.4.0 ( #1411 )
...
Notable refactoring:
- Use stdlib "context" in place of "golang.org/x/net/context"
- Go-kit no longer wraps errors, so we remove the unwrap in transport_error.go
- Use MakeHandler when setting up endpoint tests (fixes test bug caught during
this refactoring)
Closes #1411 .
2017-03-15 08:55:30 -07:00
Mike Arpaia
a000751bfe
renaming kolide-ose to kolide ( #1143 )
2017-02-01 10:20:50 -07:00
Zachary Wasserman
39ebd81dc5
Close orphaned distributed query campaign after failed publish attempt ( #707 )
...
A distributed query campaign can be "orphaned" (left in the QueryRunning state)
if the Kolide server restarts while it is running, or other weirdness occurs.
When this happens, no subscribers are waiting to read results written by
osqueryd agents, but the agents continue to receive the query. Previously, this
would cause us to error on ingestion.
The new behavior will instead set the campaign to completed when it detects
that it is orphaned. This should prevent sending queries for which there is no
subscriber.
- New NoSubscriber error interface in pubsub
- Detect NoSubscriber errors and close campaigns
- Tests on pubsub and service methods
Fixes #695
2016-12-27 10:35:19 -05:00
Zachary Wasserman
9c38d6d19e
Update osquery service methods for distributed queries ( #476 )
2016-11-14 10:22:54 -08:00
Zachary Wasserman
262a48f8eb
Datastores for buffering distributed query results ( #346 )
...
A new datastore interface is needed for buffering incoming distributed query results to be sent to the client. This PR attempts to define and implement that interface.
It is intended that the ReadChannel() method be used by the goroutine that will push query results down a websocket to the client. Passing the results through this channel will allow that goroutine to perform a select on both the channel and the websocket, in order to properly handle IO.
2016-10-31 15:51:19 -07:00
