Commit Graph

1241 Commits

Author SHA1 Message Date
Zachary Wasserman
a41431d3b6
Update osquery schema to 4.5.0 (#2317) 2020-10-06 10:31:51 -07:00
Zachary Wasserman
e8e4bc9879
Fix cleanup of orphaned queries (#2316)
The expiration logic was incorrect leading to queries not being cleaned
up properly. Tests added for the whole subroutine.

Fixes #2302
2020-10-06 09:30:24 -07:00
Zachary Wasserman
50dbdb3db4
Add explicit int64 for parsing physical_memory column (#2315)
Addresses an error when Fleet is compiled for a 32-bit architecture.

May not be comprehensive of all related errors.

Fixes #2314.
2020-10-05 17:03:32 -07:00
Zachary Wasserman
f2d3683d80
Log host identifier in EnrollAgent service logging (#2313)
This can help identify circumstances in which multiple hosts are
providing the same identifier and clobbering the node key for each
other.
2020-10-02 13:23:54 -07:00
Kilian
c61ba759dd
Add redis use_tls cfg (#2311)
Adding config parameter 'redis.use_tls' to enable tls communications with redis e.g. AWS ElastiCache

Closes #2247
2020-10-01 16:25:48 -07:00
Iain Steers
da99617882
Add DeleteHost func to service pkg (#2312) 2020-10-01 16:14:18 -07:00
James Alseth
a2f49dd620
Add hostname to delete host confirmation modal (#2310)
Fixes #2233
2020-10-01 09:38:43 -07:00
James Alseth
4a1d7c7785
Makefile: don't generate JS assets on xp-fleetctl target (#2308) 2020-09-27 09:45:22 -07:00
James Alseth
692171dfdb
Fix string(int) in Kinesis logger for Go 1.15 compatibility (#2309) 2020-09-27 09:44:55 -07:00
James Alseth
44c339699a
Remove -i flag from Go builds (#2307)
The -i (install) flag is not necessary and caused issues with building in some scenarios.

Fixes #2287
2020-09-25 10:30:29 -07:00
James Alseth
1d5f5d7eac
Add osquery_version to the table in fleetctl get hosts (#2304) 2020-09-25 10:15:42 -07:00
Stefhen Hovland
7ce0472c2c
Add -L option to curl (#2305)
The example in the README won't follow the github redirect without `-L`, so the example as is will download an html file instead of the actual `fleet.zip`. This fixes that.
2020-09-24 10:42:19 -07:00
Zachary Wasserman
d959a4e4af
Fix Slack invitation URL (#2303)
Closes #2299
2020-09-17 13:18:04 -07:00
Lars Lehtonen
d193ea1717
Remove Support for Deprecated TLSProfileOld (#2142)
Co-authored-by: Zachary Wasserman <zach@dactiv.llc>
2020-09-10 09:31:01 -07:00
James Alseth
3a63dac4a3
Add compression option for filesystem logs when they're rotated (#2292) 2020-09-09 13:33:32 -07:00
James Alseth
e88ea5dca5
Fix typo in logging_disable_banner docs (#2289) 2020-09-09 13:32:13 -07:00
billcobbler
20328b0f87
Add stdout and kinesis logger plugins and sts assume role to Firehose (#2282)
Co-authored-by: Brendan Shaklovitz <nyanshak@users.noreply.github.com>
2020-08-19 14:56:44 -07:00
James Alseth
b67cfc479b
Fix string(int) issues for Go 1.15 compatibility (#2286) 2020-08-19 13:36:36 -07:00
seph
c2d8dccb8d
Remove old package stuff (#2284)
Remove references to the long unused dl.kolide.co distribution site.
2020-08-11 18:14:59 -07:00
Brendan Shaklovitz
0cf47bf233
Fix fleetctl get options typo (#2281)
Fixes issue where `fleetctl get options` returns `option` (singular)
as `kind` instead of `options` (plural). This would cause `fleetctl
apply -f options.yml` to fail on options yaml generated by `fleetctl get
options` with this error: `unknown kind "option"`.
2020-08-07 15:28:24 -07:00
Zachary Wasserman
09632fb1ed
Add note about downtime during database migrations (#2279)
This should help clarify that migrations should be run with server
instances stopped.
2020-08-06 17:16:23 -07:00
Zachary Wasserman
c6ce648fef
Update changelog for 3.1.0 release (#2278) 2020-08-06 16:50:34 -07:00
Zachary Wasserman
81c72b286c
Fix URL Prefix for Edit Pack button (#2277)
Fixes #2275
2020-08-06 16:29:24 -07:00
Zachary Wasserman
34c4ce4ffb
Fix parsing of integer label_type in yaml (#2276)
Fixes #2273
2020-08-06 16:07:13 -07:00
Zachary Wasserman
0619581b03
Print single secret value if name specified (#2274)
Closes #2272
2020-08-03 11:51:29 -07:00
Stephan Miehe
2ad5205a4b
Add support for conn_max_lifetime (#2270)
This adds support to configure MySQL conn_max_lifetime.
2020-07-30 09:00:42 -07:00
Stephan Miehe
cf4d8ecfee
Add redis database number support (#2269)
Fixes #2268
2020-07-30 08:57:25 -07:00
Zachary Wasserman
0058d45902
Update go.sum (#2267) 2020-07-23 09:27:49 -07:00
Zachary Wasserman
af5fefa564
Changelog for 3.0.0 release (#2266) 2020-07-22 19:56:01 -07:00
Zachary Wasserman
1058f57767
Remove noisy User log (#2265)
This log appears to add little value as it is generated by almost any
web UI request.
2020-07-22 09:49:34 -07:00
Zachary Wasserman
2bfcecf0d3
Add CAdvisor container to dev docker-compose (#2263)
This container is useful for monitoring stats of other containers.
2020-07-21 15:36:11 -07:00
Zachary Wasserman
4023296d86
Log at debug level when configured (#2262)
We added debug/info log levels in #2225 but the configuration was not
actually used to set the level.
2020-07-21 15:35:57 -07:00
Zachary Wasserman
7494513400 Clean up and comments before merge. 2020-07-21 14:05:46 -07:00
Zachary Wasserman
96fc090723 Improve server performance for host operations
- Debounce frontend to reduce number of target searches in live query.
- More efficiently calculate label counts in live query and hosts
  dashboard. Instead of using the (slow) CountHostsInTargets function,
  retrieve the host counts while looking up the labels.
- Optimize targets search query. Removing the nested query retrieves the
  same logical result set, but substantially optimizes MySQL CPU usage.
  Testing indicates about a 50% reduction in MySQL CPU usage for the
  frontend targets search API call after applying this change.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
fcb8418b2f Add fleetctl get host capability to get single host with labels
Getting a single host with `fleetctl get host foobar` will look up the
host with the matching hostname, uuid, osquery identifier, or node key,
and provide the full host details along with the labels the host is a
member of.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
7f757d3144 Extract functionName into helper
Cleans up some repetition in tests.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
42bea2a144 Implement manual labels
"Manual" labels can be specified by hostname, allowing users to specify
the membership of a label without having to use a dynamic query. See the
included documentation.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
608772917c Refactor label membership storage
Label membership is now stored in the label_membership table. This is
done in preparation for adding "manual" labels, as previously label
membership was associated directly with label query executions.

Label queries are now all executed at the same time, rather than on
separate intervals. This simplifies the calculation of which distributed
queries need to be run when a host checks in.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
b0595748f1 Implement pagination of hosts in the web UI
This commit takes advantage of the existing pagination APIs in the Fleet
server, and provides additional APIs to support pagination in the web
UI. Doing this dramatically reduces the response sizes for requests from
the UI, and limits the performance impact of UI clients on the Fleet and
MySQL servers.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
0502412e15 Move live query operations from MySQL to Redis
This change optimizes live queries by pushing the computation of query
targets to the creation time of the query, and efficiently caching the
targets in Redis. This results in a huge performance improvement at both
steady-state, and when running live queries.

- Live queries are stored using a bitfield in Redis, and takes
advantage of bitfield operations to be extremely efficient.

- Only run Redis live query test when REDIS_TEST is set in environment

- Ensure that live queries are only sent to hosts when there is a client
listening for results. Addresses an existing issue in Fleet along with
appropriate cleanup for the refactored live query backend.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
ea93843203 Store only primary IP and MAC for hosts
Fleet used significant resources storing the full network interface
information for each host. This data was unused, except to get the
IP and MAC of the primary interface. With these changes, only those
pieces of data are stored.

- Calculate and store primary IP and MAC
- Remove transaction for storing full interfaces
- Update targets search to use new IP and MAC columns
- Update frontend to use new new columns
2020-07-21 14:05:46 -07:00
Zachary Wasserman
b76b23e6ee
Fix yaml separator handling with comments (#2257)
Fixes #2245
2020-07-08 16:01:03 -07:00
Zachary Wasserman
fd61dcab67
Clean up unused DB tables, migrations, and code (#2258)
This PR removes unused types, code, DB tables, and associated migrations that are unused since Fleet 2.0.

An existing migration was refactored, and should remain compatible with both existing and new Fleet installations.
2020-07-08 13:02:18 -07:00
Zachary Wasserman
def51ae930
Add Go Report Card badge to readme (#2259) 2020-07-07 15:45:49 -07:00
Zachary Wasserman
168e1f9007
Update table schema for osquery 4.4.0 (#2253)
Osquery schema JSON has changed, so parsing code has been updated

Closes #2232
2020-07-07 09:47:50 -07:00
Zachary Wasserman
2578466697
Upgrade SockJS JS and Go dependencies (#2256) 2020-07-06 20:04:04 -07:00
Zachary Wasserman
ccc8581229
Upgrade eslint and dependencies (#2255)
Updates configuration and fixes issues introduced
2020-07-06 19:31:48 -07:00
Zachary Wasserman
3fb76c4154
Remove typescript-require (#2254)
This JS import was unused and causes problems when running tests on Node
14.
2020-07-06 17:02:25 -07:00
William Theaker
b5b43e7aa7
Brief Auto Table Construction docs (#2249) 2020-06-30 09:18:50 -07:00
grantseltzer
755f63f8c0
Refactor fleetctl get subcommands so that they all have json and yaml output options (#2244) 2020-06-23 08:11:47 -07:00