Commit Graph

199 Commits

Author SHA1 Message Date
Tomas Touceda
f8b7a83cc6
Process stored CPEs and store found CVEs (#1533)
* WIP

* WIP

* Make path optional and fix tests

* Add first generate

* Move to nvd package

* remove replace

* Re-add replace

* It's path, not file name

* Change how db path is set and use etag

* Fix typos

* Make db generation faster

* Remove quotes

* Doesn't like comments

* Samitize etag and save to file

* Refactor some things and improve writing of etagenv

* Compress file and truncate amount of items for faster testing

* Remove quotes

* Try to improve performance

* Ignore truncate error if not exists

* Minor cleanup and make sqlite have cpe prefix

* Simplify code and test sync

* Add VCR for sync test

* Check for nvdRelease nil

* Add test for the actual translation

* Address review comments

* Rename generate command because we'll have a cve one too

* Move to its own dir

* Add first cve db generation

* WIP but with final strategy, preparring to merge main

* Fix merge conflicts

* WIP

* wip

* Insert CVEs to the db

* Remove unused code

* Use wg instead of counting

* Call cancelFunc to avoid ctx leak

* Fix logs for better readability

* Point code to fleetdm instead of my repo
2021-08-04 18:01:39 -03:00
Martavis Parker
53dbb2ad50
#1496 fixed sorting using API (#1536)
* #1496 fixed sorting using API

* #1496 added try catch

* #1496 removed test that no longer serves a purpose

* #1496 fixed linting

* #1496 cleanup

* #1496 added loading indicator back

* #1496 fix lint issues

* #1496 added changes log

* #1496 minor fixes
2021-08-03 12:42:48 -07:00
Tomas Touceda
46b0b7765b
Issue 1435 software to cpe (#1488)
* WIP

* WIP

* Make path optional and fix tests

* Add first generate

* Move to nvd package

* remove replace

* Re-add replace

* It's path, not file name

* Change how db path is set and use etag

* Fix typos

* Make db generation faster

* Remove quotes

* Doesn't like comments

* Samitize etag and save to file

* Refactor some things and improve writing of etagenv

* Compress file and truncate amount of items for faster testing

* Remove quotes

* Try to improve performance

* Ignore truncate error if not exists

* Minor cleanup and make sqlite have cpe prefix

* Simplify code and test sync

* Add VCR for sync test

* Check for nvdRelease nil

* Add test for the actual translation

* Address review comments

* Rename generate command because we'll have a cve one too

* Move to its own dir

* Address review comments
2021-07-29 13:10:34 -03:00
Tomas Touceda
d5e40f329e
Issue 1324 add activity feed (#1343)
* Add activities generation

* Add activities endpoint

* Fix merge error

* Fix indentation issue

* Add changes file

* Address PR review comments

* Add mock activity func

* Address codacy warings

* Set foreign key but on delete set null

* Make user_id set to null if deleted
2021-07-13 16:54:22 -03:00
Martavis Parker
bc4ae0fd1d
UI tweaks 4.0.0 (#1257)
* #1246 removed outline for focused tab

* #1247 adjusted styles for actions dropdown

* #1249 addressed styles for transfer modal dropdown
2021-06-29 11:10:47 -07:00
Zach Wasserman
07b89c4cbd
Implement release automation (#1215)
- Use goreleaser to automate release process.
- Add new dockerfiles for fleet (with fleetctl) and fleetctl (only).
- Add GitHub Action Workflow to run goreleaser on new tag.
- Update NPM to match new archive naming.
2021-06-25 18:40:26 -07:00
Martavis Parker
7af97579fe
Frontend of usage statistics (#1177)
* #454 added usage stats disclaimer to setup confirmation

* #454 added new section to settings for usage stats

* #454 fixed vulnerability for hrefs

* removed jsx file

* #454 added logic to checkbox

* #454 created modal to preview usage stats; cleanup

* fixed tests and linting
2021-06-23 15:19:01 -07:00
Zach Wasserman
0a77f79d22
Backend and fleetctl for usage analytics (#1167)
- Add enable_analytics column to database.
- Allow enable_analytics to be set via API.
- Add messaging in fleetctl setup.

Note that this defaults to off for existing installations, and defaults
on for newly set up installs.

No collection or sending of analytics yet exists, we are strictly
storing the preference at this time.

Part of #454
2021-06-22 18:02:20 -07:00
Zach Wasserman
2ad557e3b3 Merge branch 'main' into teams 2021-06-18 09:42:20 -07:00
dsbaha
47b423ee29
Add Redis cluster support (#1045)
This should support Redis in both cluster and non-cluster modes.

Updates were made separately to github.com/throttled/throttled to support the slight changes in types.

Co-authored-by: Joseph Macaulay <joseph.macaulay@uber.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-06-18 08:51:47 -07:00
Martavis Parker
121de5a87f
Fixed Manage Hosts Page Calling Teams API on Core (#1038)
* Closes #922

* #922 added Windows section to build docs

* go sum updated

* updated go sum

* fixed #963 - calling teams api if not on core
2021-06-09 15:17:50 -07:00
Zach Wasserman
db459d3363
Continue to update names in backend code and docs (#976) 2021-06-06 16:58:23 -07:00
Zach Wasserman
fa01df79d5 Merge branch 'master' into teams 2021-06-05 06:27:38 -07:00
Martavis Parker
cb2682c273
Remove detection for OS during contributor build (#922) (#933)
- No longer detecting for OS make commands
- Using the same internal commands for every OS
- go.sum auto-updated during build
- Document build process on Windows
2021-06-04 08:10:25 -07:00
Zach Wasserman
9876dbe6b6
Implement license loading and validation (#908)
- Load license from ES256 signed JWT key.
- Parse license claims into LicenseInfo struct.
- Update contribution documentation with sample license key.

Closes #816.
2021-05-31 11:35:15 -07:00
Noah Talerman
72882e8f9f Merge branch 'master' into teams 2021-05-19 13:16:54 -04:00
Zach Wasserman
69f73b56f8
Upgrade dependencies and remove replace directive (#802)
Now that https://github.com/throttled/throttled/pull/87 is merged, we
can remove the stopgap `replace` directive in `go.mod`.
2021-05-18 17:00:20 -07:00
Zach Wasserman
1638d5e2da
Update Redigo version and timeouts (#793)
- Use newer version of Redigo that properly handles TLS mismatch.
- Add timeouts for Redis TCP connection.

Fixes #792
2021-05-18 15:04:14 -07:00
WangXiang
468754f2b9
Format and clean code (#774)
1. use [staticcheck](https://staticcheck.io/) to check the code, and fix some issues.
2. use `go fmt` to format the code.
3. use `go mod tidy` clean the go mod.
2021-05-17 10:29:50 -07:00
Zach Wasserman
e8669818eb
Initial backend software inventory implementation (#678)
- Maintain software inventory with detail queries.
- Associated database migrations.
- Feature flagged off by default (see documentation for details to turn on).
- Documentation.
- New test helper for slice element comparisons skipping ID.
2021-04-26 08:44:22 -07:00
Zach Wasserman
9064ef9b59
Clean up Go dependencies (#619)
Removes unused libraries and switches a use of bmizerany/assert to
stretchr/testify/assert.
2021-04-08 16:45:04 -07:00
Zach Wasserman
505d53b02b
Refactor MySQL tests into package mysql (#586)
Moving the tests into the package allows the coverage tool to function properly.
2021-04-05 11:04:53 -07:00
Zach Wasserman
a79ad7d312
Improvements to built-in labels (#562)
- Fix issue with built-in labels showing multiple platforms when hosts
  are reinstalled with new platform.
- Add Red Hat Linux built-in label.
- Display more labels by default in target selector.

Fixes #546, #553
2021-03-30 15:37:33 -07:00
Zach Wasserman
2203bd81a2
Add support for IdP-initiated SSO login (#556)
This feature is off by default due to minor potential security concerns
with IdP-initiated SSO (see
https://auth0.com/docs/protocols/saml-protocol/saml-configuration-options/identity-provider-initiated-single-sign-on#risks-of-using-an-identity-provider-initiated-sso-flow).

Closes #478
2021-03-30 12:56:20 -07:00
Zach Wasserman
12d292164f
Add rate-limiting to login and password reset (#543)
Prevent abuse of these endpoints with rate limiting backed by Redis. The
limits assigned should be appropriate for almost any Fleet deployment.

Closes #530
2021-03-26 11:23:29 -07:00
Zach Wasserman
d621650b78
Initial implementation of agent update management (#516)
This PR contains the initial implementation of the fleetctl updates commands, along with documentation on using this to self-host an agent update server.

Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
2021-03-26 10:46:51 -07:00
Zach Wasserman
58d1d025a4
Upgrade go-bindata to maintained github.com/kevinburke/go-bindata (#512)
- This seems to be the maintained, trusted (by Homebrew, etc.) version
  of go-bindata.
- Add tools.go file to pin version with go modules.
- Use go run to run the binary, making easier configuration for new developers.
2021-03-22 11:03:03 -07:00
Zach Wasserman
db3c73d619
Test and correct semantics for MySQL withRetryTxx (#488)
- Differentiate retryable vs. non-retryable errors.
- Test functionality against mocked DB.
2021-03-17 11:23:52 -07:00
Zach Wasserman
4cfcb1b084
Upgrade fleetctl github.com/urfave/cli to v2 (#471)
This is intended to upgrade to the new API without changing fleetctl
functionality.
2021-03-12 16:42:38 -08:00
Zach Wasserman
e2e7e4707d
Update database migration tooling (#281)
- Use updated github.com/fleetdm/goose that generates templates
  requiring less editing.
- Add `make migration` to Makefile.
- Update documentation.
2021-02-05 09:48:06 -08:00
Zach Wasserman
bb124dc2b9
Improvements to fleetctl preview experience (#234)
- Make the preview directory in the default .fleet directory.
- Check for Docker daemon installed but not running.
- Add message for Chrome users on self-signed certs.
- Display login information on later invocations of command.
- Remove "Kolide" from error messages.

Closes #190

Part of #197
2021-01-28 07:57:32 -08:00
Zach Wasserman
44be83eefd
Improved error handling in host enrollment (#203)
This will make it easier to debug issues with the enrollment process,
such as #200.
2021-01-19 14:45:58 -08:00
Zach Wasserman
57812a532e
Merge pull request from GHSA-w3wf-cfx3-6gcx
* Update github.com/russellhaering/goxmldsig

* Update signature validation to include Mattermost XML validator
2020-12-14 18:14:14 -08:00
Zach Wasserman
beb7e8b965
Add host enrollment cooldown period (#112)
This addresses an issue some users experienced in which performance
problems were encountered when hosts were "competing" for enrollment
using the same osquery host identifier. The issue is addressed by adding
a cooldown period for host enrollment, preventing the same (as judged by
osquery host identifier) host from enrolling more than once per minute.

When users end up in the problematic scenario, they will see quite a bit
of error logs due to this issue. For now that's probably a good thing as
users need to be aware of the lack of visibility. We can explore rate
limiting the logging if that becomes an issue for someone.

Fixes #102
2020-12-10 11:04:58 -08:00
Zachary Wasserman
e452cc6a8a
Add file carving support (#15)
- Add endpoints for osquery to register and continue a carve.
- Implement client functionality for retrieving carve details and contents in fleetctl.
- Add documentation on using file carving with Fleet.

Addresses kolide/fleet#1714
2020-11-04 20:45:16 -08:00
Zachary Wasserman
7c923d9d19
Implement fleetctl user create (#9)
- Allow user creation via `fleetctl user create`
- Cleanup and rename existing methods for clarity

Fixes https://github.com/kolide/fleet/issues/2306
2020-11-04 17:06:55 -08:00
Zachary Wasserman
f06c3cff93
Add pretty-printing option for query results (#4)
Use --pretty to enable this feature.

This will likely break on non-ANSI terminals.
2020-11-04 09:56:57 -08:00
Zachary Wasserman
328e8a4c8d
Update github.com/go-sql-driver/mysql to 1.5.0 (#2321) 2020-10-08 09:17:08 -07:00
Zachary Wasserman
0058d45902
Update go.sum (#2267) 2020-07-23 09:27:49 -07:00
Zachary Wasserman
7494513400 Clean up and comments before merge. 2020-07-21 14:05:46 -07:00
Zachary Wasserman
0502412e15 Move live query operations from MySQL to Redis
This change optimizes live queries by pushing the computation of query
targets to the creation time of the query, and efficiently caching the
targets in Redis. This results in a huge performance improvement at both
steady-state, and when running live queries.

- Live queries are stored using a bitfield in Redis, and takes
advantage of bitfield operations to be extremely efficient.

- Only run Redis live query test when REDIS_TEST is set in environment

- Ensure that live queries are only sent to hosts when there is a client
listening for results. Addresses an existing issue in Fleet along with
appropriate cleanup for the refactored live query backend.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
2578466697
Upgrade SockJS JS and Go dependencies (#2256) 2020-07-06 20:04:04 -07:00
billcobbler
44c40e4255
Retry sqlx transactions (#2203)
- Add exponential backoff method to MySQL datastore
- Refactor queries using transaction to use backup.

Fixes #845
2020-03-10 18:14:02 -07:00
Zachary Wasserman
6dbc3bc77a
Implement fleetctl goquery (#2186)
- Update fleet APIs to support necessary operations in goquery
- Implement support for goquery in fleetctl
2020-01-23 21:27:20 -08:00
Zachary Wasserman
3b02640334
Update goxmldsig dependency (#2177)
Update the github.com/russellhaering/goxmldsig dependency and apply
the appropriate fixes for the API changes.

This is a preparation for integration with
github.com/AbGuthrie/goquery, which uses a newer version of the
dependency.
2020-01-13 16:15:14 -08:00
Christian Barcenas
b3a7fc83e1 Use canonical import path for gopkg.in/guregu/null (#2171)
The `gopkg.in` import path is used in the package's documentation, and
by downstream Linux distro maintainers (e.g. Debian Go Packaging Team).
2020-01-13 09:11:31 -08:00
Zachary Wasserman
bf9ae6ec0e
Update osquery-go dependency to fix thrift module download problem. (#2105)
Fixes #2102
2019-09-12 17:35:10 -07:00
Zachary Wasserman
a06ad6f4fa
Run go mod tidy (#2098) 2019-08-14 11:11:20 -07:00
Zachary Wasserman
27bbf2ae22
Migrate dependencies to go modules (#2094)
These changes should be transparent to users and developers.

Thanks to @davepgreene who made the first stab at this.
2019-08-13 10:32:23 -07:00