Commit Graph

159 Commits

Author SHA1 Message Date
Tomas Touceda
d6d53acda5
Issue 1600 fleetctl license expiration (#1800)
* Show banner in fleet and fleetctl if license expired

* Ignore if license is nil or tier is core

* Address review comments
2021-08-26 10:28:53 -03:00
Tomas Touceda
c6c63ab12a
Refactor app config (POC, for now) (#1685) 2021-08-20 12:27:41 -03:00
Tomas Touceda
402303bc5d
Add All Linux label (#1582)
* Add All Linux label

* Change name to Linux instead of All Linux to see if e2e likes it better

* Revert "Change name to Linux instead of All Linux to see if e2e likes it better"

This reverts commit 26b79f214e3b744e73270c544f89bb698575f6ea.

* Fix all linux label insert
2021-08-18 15:55:48 -03:00
Benjamin Edwards
2d5c7ed683
issue 1313 -- add file destination for convert command (#1672)
* add file destination flag for convert command, add tests
* add changes file
* remove leftover debugging code, add comment explaing why sorting was necessary
* update docs for fleetctl convert
* Update cmd/fleetctl/convert_test.go
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-08-13 20:42:58 -04:00
Tomas Touceda
96c0244c04
Fix redis locking receive (#1655)
* Receive redis data with timeout to not hold the connection

* Address review comments
2021-08-13 11:16:11 -03:00
Tomas Touceda
11dff7ca92
Issue 1588 allow disabling host users (#1611)
* Allow users to disable host_users

* Add missing files

* Address review comments

* Fix tests

* Check additional queries for nil, not the whole hostsettings

* Finally fix tests
2021-08-11 14:56:11 -03:00
Tomas Touceda
989e638cc0
Make creating dirs and files more secure by checking permissions (#1566)
* Add safe mkdirall and open

* Use secure as much as possible and merge gomodules for orbit to fleet

* Improve openfile and mkdirall to check for permissiveness instead of equality

* Don't shift

* Fix links

* Address review comments
2021-08-11 11:02:22 -03:00
Tomas Touceda
d51493274d
Issue 1570 stats perf (#1598)
* Dont delete pack stats before inserting new ones to prevent deadlocks

* Remove fk for scheduled_query_stats

* Remove fk removal

* Fix tests

* Remove unneeded comment
2021-08-10 18:17:06 -03:00
Benjamin Edwards
29611d8377
modify packs api to filter non-empty pack_type from response (#1558)
* modify packs api to filter non-empty pack_type from response
* change list packs store method to allow filtering for "system-level" packs, by default the api filters these packs from being returned
* add changes file
* don't allow modifications to global or team packs via apply spec
* refactor to use PackListOptions
2021-08-10 09:43:27 -04:00
Zach Wasserman
9db49a1128
Update JWT library to github.com/golang-jwt/jwt (#1568)
- The former repo has been migrated (see
  https://github.com/dgrijalva/jwt-go/issues/462).

- Dependabot alerted on a security issue in the old
  version (https://github.com/advisories/GHSA-w73w-5m7g-f7qc). This issue
  should not effect Fleet due to no use of `aud` claims.
2021-08-06 08:49:38 -07:00
Tomas Touceda
f8b7a83cc6
Process stored CPEs and store found CVEs (#1533)
* WIP

* WIP

* Make path optional and fix tests

* Add first generate

* Move to nvd package

* remove replace

* Re-add replace

* It's path, not file name

* Change how db path is set and use etag

* Fix typos

* Make db generation faster

* Remove quotes

* Doesn't like comments

* Samitize etag and save to file

* Refactor some things and improve writing of etagenv

* Compress file and truncate amount of items for faster testing

* Remove quotes

* Try to improve performance

* Ignore truncate error if not exists

* Minor cleanup and make sqlite have cpe prefix

* Simplify code and test sync

* Add VCR for sync test

* Check for nvdRelease nil

* Add test for the actual translation

* Address review comments

* Rename generate command because we'll have a cve one too

* Move to its own dir

* Add first cve db generation

* WIP but with final strategy, preparring to merge main

* Fix merge conflicts

* WIP

* wip

* Insert CVEs to the db

* Remove unused code

* Use wg instead of counting

* Call cancelFunc to avoid ctx leak

* Fix logs for better readability

* Point code to fleetdm instead of my repo
2021-08-04 18:01:39 -03:00
Martavis Parker
53dbb2ad50
#1496 fixed sorting using API (#1536)
* #1496 fixed sorting using API

* #1496 added try catch

* #1496 removed test that no longer serves a purpose

* #1496 fixed linting

* #1496 cleanup

* #1496 added loading indicator back

* #1496 fix lint issues

* #1496 added changes log

* #1496 minor fixes
2021-08-03 12:42:48 -07:00
Tomas Touceda
46b0b7765b
Issue 1435 software to cpe (#1488)
* WIP

* WIP

* Make path optional and fix tests

* Add first generate

* Move to nvd package

* remove replace

* Re-add replace

* It's path, not file name

* Change how db path is set and use etag

* Fix typos

* Make db generation faster

* Remove quotes

* Doesn't like comments

* Samitize etag and save to file

* Refactor some things and improve writing of etagenv

* Compress file and truncate amount of items for faster testing

* Remove quotes

* Try to improve performance

* Ignore truncate error if not exists

* Minor cleanup and make sqlite have cpe prefix

* Simplify code and test sync

* Add VCR for sync test

* Check for nvdRelease nil

* Add test for the actual translation

* Address review comments

* Rename generate command because we'll have a cve one too

* Move to its own dir

* Address review comments
2021-07-29 13:10:34 -03:00
Tomas Touceda
d5e40f329e
Issue 1324 add activity feed (#1343)
* Add activities generation

* Add activities endpoint

* Fix merge error

* Fix indentation issue

* Add changes file

* Address PR review comments

* Add mock activity func

* Address codacy warings

* Set foreign key but on delete set null

* Make user_id set to null if deleted
2021-07-13 16:54:22 -03:00
Martavis Parker
bc4ae0fd1d
UI tweaks 4.0.0 (#1257)
* #1246 removed outline for focused tab

* #1247 adjusted styles for actions dropdown

* #1249 addressed styles for transfer modal dropdown
2021-06-29 11:10:47 -07:00
Zach Wasserman
07b89c4cbd
Implement release automation (#1215)
- Use goreleaser to automate release process.
- Add new dockerfiles for fleet (with fleetctl) and fleetctl (only).
- Add GitHub Action Workflow to run goreleaser on new tag.
- Update NPM to match new archive naming.
2021-06-25 18:40:26 -07:00
Martavis Parker
7af97579fe
Frontend of usage statistics (#1177)
* #454 added usage stats disclaimer to setup confirmation

* #454 added new section to settings for usage stats

* #454 fixed vulnerability for hrefs

* removed jsx file

* #454 added logic to checkbox

* #454 created modal to preview usage stats; cleanup

* fixed tests and linting
2021-06-23 15:19:01 -07:00
Zach Wasserman
0a77f79d22
Backend and fleetctl for usage analytics (#1167)
- Add enable_analytics column to database.
- Allow enable_analytics to be set via API.
- Add messaging in fleetctl setup.

Note that this defaults to off for existing installations, and defaults
on for newly set up installs.

No collection or sending of analytics yet exists, we are strictly
storing the preference at this time.

Part of #454
2021-06-22 18:02:20 -07:00
Zach Wasserman
2ad557e3b3 Merge branch 'main' into teams 2021-06-18 09:42:20 -07:00
dsbaha
47b423ee29
Add Redis cluster support (#1045)
This should support Redis in both cluster and non-cluster modes.

Updates were made separately to github.com/throttled/throttled to support the slight changes in types.

Co-authored-by: Joseph Macaulay <joseph.macaulay@uber.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-06-18 08:51:47 -07:00
Martavis Parker
121de5a87f
Fixed Manage Hosts Page Calling Teams API on Core (#1038)
* Closes #922

* #922 added Windows section to build docs

* go sum updated

* updated go sum

* fixed #963 - calling teams api if not on core
2021-06-09 15:17:50 -07:00
Zach Wasserman
db459d3363
Continue to update names in backend code and docs (#976) 2021-06-06 16:58:23 -07:00
Zach Wasserman
fa01df79d5 Merge branch 'master' into teams 2021-06-05 06:27:38 -07:00
Martavis Parker
cb2682c273
Remove detection for OS during contributor build (#922) (#933)
- No longer detecting for OS make commands
- Using the same internal commands for every OS
- go.sum auto-updated during build
- Document build process on Windows
2021-06-04 08:10:25 -07:00
Zach Wasserman
9876dbe6b6
Implement license loading and validation (#908)
- Load license from ES256 signed JWT key.
- Parse license claims into LicenseInfo struct.
- Update contribution documentation with sample license key.

Closes #816.
2021-05-31 11:35:15 -07:00
Noah Talerman
72882e8f9f Merge branch 'master' into teams 2021-05-19 13:16:54 -04:00
Zach Wasserman
69f73b56f8
Upgrade dependencies and remove replace directive (#802)
Now that https://github.com/throttled/throttled/pull/87 is merged, we
can remove the stopgap `replace` directive in `go.mod`.
2021-05-18 17:00:20 -07:00
Zach Wasserman
1638d5e2da
Update Redigo version and timeouts (#793)
- Use newer version of Redigo that properly handles TLS mismatch.
- Add timeouts for Redis TCP connection.

Fixes #792
2021-05-18 15:04:14 -07:00
WangXiang
468754f2b9
Format and clean code (#774)
1. use [staticcheck](https://staticcheck.io/) to check the code, and fix some issues.
2. use `go fmt` to format the code.
3. use `go mod tidy` clean the go mod.
2021-05-17 10:29:50 -07:00
Zach Wasserman
e8669818eb
Initial backend software inventory implementation (#678)
- Maintain software inventory with detail queries.
- Associated database migrations.
- Feature flagged off by default (see documentation for details to turn on).
- Documentation.
- New test helper for slice element comparisons skipping ID.
2021-04-26 08:44:22 -07:00
Zach Wasserman
9064ef9b59
Clean up Go dependencies (#619)
Removes unused libraries and switches a use of bmizerany/assert to
stretchr/testify/assert.
2021-04-08 16:45:04 -07:00
Zach Wasserman
505d53b02b
Refactor MySQL tests into package mysql (#586)
Moving the tests into the package allows the coverage tool to function properly.
2021-04-05 11:04:53 -07:00
Zach Wasserman
a79ad7d312
Improvements to built-in labels (#562)
- Fix issue with built-in labels showing multiple platforms when hosts
  are reinstalled with new platform.
- Add Red Hat Linux built-in label.
- Display more labels by default in target selector.

Fixes #546, #553
2021-03-30 15:37:33 -07:00
Zach Wasserman
2203bd81a2
Add support for IdP-initiated SSO login (#556)
This feature is off by default due to minor potential security concerns
with IdP-initiated SSO (see
https://auth0.com/docs/protocols/saml-protocol/saml-configuration-options/identity-provider-initiated-single-sign-on#risks-of-using-an-identity-provider-initiated-sso-flow).

Closes #478
2021-03-30 12:56:20 -07:00
Zach Wasserman
12d292164f
Add rate-limiting to login and password reset (#543)
Prevent abuse of these endpoints with rate limiting backed by Redis. The
limits assigned should be appropriate for almost any Fleet deployment.

Closes #530
2021-03-26 11:23:29 -07:00
Zach Wasserman
d621650b78
Initial implementation of agent update management (#516)
This PR contains the initial implementation of the fleetctl updates commands, along with documentation on using this to self-host an agent update server.

Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
2021-03-26 10:46:51 -07:00
Zach Wasserman
58d1d025a4
Upgrade go-bindata to maintained github.com/kevinburke/go-bindata (#512)
- This seems to be the maintained, trusted (by Homebrew, etc.) version
  of go-bindata.
- Add tools.go file to pin version with go modules.
- Use go run to run the binary, making easier configuration for new developers.
2021-03-22 11:03:03 -07:00
Zach Wasserman
db3c73d619
Test and correct semantics for MySQL withRetryTxx (#488)
- Differentiate retryable vs. non-retryable errors.
- Test functionality against mocked DB.
2021-03-17 11:23:52 -07:00
Zach Wasserman
4cfcb1b084
Upgrade fleetctl github.com/urfave/cli to v2 (#471)
This is intended to upgrade to the new API without changing fleetctl
functionality.
2021-03-12 16:42:38 -08:00
Zach Wasserman
e2e7e4707d
Update database migration tooling (#281)
- Use updated github.com/fleetdm/goose that generates templates
  requiring less editing.
- Add `make migration` to Makefile.
- Update documentation.
2021-02-05 09:48:06 -08:00
Zach Wasserman
bb124dc2b9
Improvements to fleetctl preview experience (#234)
- Make the preview directory in the default .fleet directory.
- Check for Docker daemon installed but not running.
- Add message for Chrome users on self-signed certs.
- Display login information on later invocations of command.
- Remove "Kolide" from error messages.

Closes #190

Part of #197
2021-01-28 07:57:32 -08:00
Zach Wasserman
44be83eefd
Improved error handling in host enrollment (#203)
This will make it easier to debug issues with the enrollment process,
such as #200.
2021-01-19 14:45:58 -08:00
Zach Wasserman
57812a532e
Merge pull request from GHSA-w3wf-cfx3-6gcx
* Update github.com/russellhaering/goxmldsig

* Update signature validation to include Mattermost XML validator
2020-12-14 18:14:14 -08:00
Zach Wasserman
beb7e8b965
Add host enrollment cooldown period (#112)
This addresses an issue some users experienced in which performance
problems were encountered when hosts were "competing" for enrollment
using the same osquery host identifier. The issue is addressed by adding
a cooldown period for host enrollment, preventing the same (as judged by
osquery host identifier) host from enrolling more than once per minute.

When users end up in the problematic scenario, they will see quite a bit
of error logs due to this issue. For now that's probably a good thing as
users need to be aware of the lack of visibility. We can explore rate
limiting the logging if that becomes an issue for someone.

Fixes #102
2020-12-10 11:04:58 -08:00
Zachary Wasserman
e452cc6a8a
Add file carving support (#15)
- Add endpoints for osquery to register and continue a carve.
- Implement client functionality for retrieving carve details and contents in fleetctl.
- Add documentation on using file carving with Fleet.

Addresses kolide/fleet#1714
2020-11-04 20:45:16 -08:00
Zachary Wasserman
7c923d9d19
Implement fleetctl user create (#9)
- Allow user creation via `fleetctl user create`
- Cleanup and rename existing methods for clarity

Fixes https://github.com/kolide/fleet/issues/2306
2020-11-04 17:06:55 -08:00
Zachary Wasserman
f06c3cff93
Add pretty-printing option for query results (#4)
Use --pretty to enable this feature.

This will likely break on non-ANSI terminals.
2020-11-04 09:56:57 -08:00
Zachary Wasserman
328e8a4c8d
Update github.com/go-sql-driver/mysql to 1.5.0 (#2321) 2020-10-08 09:17:08 -07:00
Zachary Wasserman
0058d45902
Update go.sum (#2267) 2020-07-23 09:27:49 -07:00
Zachary Wasserman
7494513400 Clean up and comments before merge. 2020-07-21 14:05:46 -07:00
Zachary Wasserman
0502412e15 Move live query operations from MySQL to Redis
This change optimizes live queries by pushing the computation of query
targets to the creation time of the query, and efficiently caching the
targets in Redis. This results in a huge performance improvement at both
steady-state, and when running live queries.

- Live queries are stored using a bitfield in Redis, and takes
advantage of bitfield operations to be extremely efficient.

- Only run Redis live query test when REDIS_TEST is set in environment

- Ensure that live queries are only sent to hosts when there is a client
listening for results. Addresses an existing issue in Fleet along with
appropriate cleanup for the refactored live query backend.
2020-07-21 14:05:46 -07:00
Zachary Wasserman
2578466697
Upgrade SockJS JS and Go dependencies (#2256) 2020-07-06 20:04:04 -07:00
billcobbler
44c40e4255
Retry sqlx transactions (#2203)
- Add exponential backoff method to MySQL datastore
- Refactor queries using transaction to use backup.

Fixes #845
2020-03-10 18:14:02 -07:00
Zachary Wasserman
6dbc3bc77a
Implement fleetctl goquery (#2186)
- Update fleet APIs to support necessary operations in goquery
- Implement support for goquery in fleetctl
2020-01-23 21:27:20 -08:00
Zachary Wasserman
3b02640334
Update goxmldsig dependency (#2177)
Update the github.com/russellhaering/goxmldsig dependency and apply
the appropriate fixes for the API changes.

This is a preparation for integration with
github.com/AbGuthrie/goquery, which uses a newer version of the
dependency.
2020-01-13 16:15:14 -08:00
Christian Barcenas
b3a7fc83e1 Use canonical import path for gopkg.in/guregu/null (#2171)
The `gopkg.in` import path is used in the package's documentation, and
by downstream Linux distro maintainers (e.g. Debian Go Packaging Team).
2020-01-13 09:11:31 -08:00
Zachary Wasserman
bf9ae6ec0e
Update osquery-go dependency to fix thrift module download problem. (#2105)
Fixes #2102
2019-09-12 17:35:10 -07:00
Zachary Wasserman
a06ad6f4fa
Run go mod tidy (#2098) 2019-08-14 11:11:20 -07:00
Zachary Wasserman
27bbf2ae22
Migrate dependencies to go modules (#2094)
These changes should be transparent to users and developers.

Thanks to @davepgreene who made the first stab at this.
2019-08-13 10:32:23 -07:00