Commit Graph

89 Commits

Author SHA1 Message Date
Tomas Touceda
7f8099db57
Add sandbox fleet serve config (#6619)
* Add sandbox fleet serve config

* Update docs

* Maybe fix lint issues

* Add lint dep
2022-07-12 18:21:15 -03:00
dependabot[bot]
b26a5cdb4e
Bump github.com/open-policy-agent/opa from 0.40.0 to 0.42.0 (#6537)
* Bump github.com/open-policy-agent/opa from 0.40.0 to 0.42.0

Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.40.0 to 0.42.0.
- [Release notes](https://github.com/open-policy-agent/opa/releases)
- [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-policy-agent/opa/compare/v0.40.0...v0.42.0)

---
updated-dependencies:
- dependency-name: github.com/open-policy-agent/opa
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Include go.* files in tests

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-07-11 14:58:27 -03:00
Ahmed Elshaer
a9f48ff561
Create Bulk Users from CSV (#3372)
* Create Bulk Users

* WIP: Adding a test for bulk user import

* adding a user bulk create test

* Fixing description, removing password required, and adding more test cases

* Fixing description, removing password required, and adding more test cases

* Fixed all comments and added Random Password Generator

* returning an error in generateRandomPassword

* Using 2 loops to create user list and then create the actual users

* Adding a bulk user delete

* fixing a mistake in temp csv

* fixed lints and removed yamlFlag
2022-06-22 13:34:58 -03:00
Lucas Manuel Rodriguez
4cfeaa1580
Do not use golangci action for better reproducibility (use make lint-go) (#6175)
* Do not use golangci action for better reproducibility

* Add fix to trigger build

* Fix all reported issues

* fix more lint errors

* Add missing import

* Remove unused method

* Remove change not necessary
2022-06-10 18:52:24 -03:00
Juan Fernandez
ef73039559
Improve vulnerability detection for Ubuntu (#6102)
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts

To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
2022-06-07 21:09:47 -04:00
Roberto Dip
5f5095de2c
bump go version in go.mod to 1.17 (#6129) 2022-06-07 16:05:55 -03:00
Michal Nicpon
30a864c8d2
Sync CVE scores periodically (#5838) 2022-06-01 10:06:57 -06:00
dependabot[bot]
092940b862
Bump github.com/theupdateframework/go-tuf from 0.2.0 to 0.3.0 (#5894)
* Bump github.com/theupdateframework/go-tuf from 0.2.0 to 0.3.0

Bumps [github.com/theupdateframework/go-tuf](https://github.com/theupdateframework/go-tuf) from 0.2.0 to 0.3.0.
- [Release notes](https://github.com/theupdateframework/go-tuf/releases)
- [Commits](https://github.com/theupdateframework/go-tuf/compare/v0.2.0...v0.3.0)

---
updated-dependencies:
- dependency-name: github.com/theupdateframework/go-tuf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Remove err check for client.IsLatestSnapshot

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-05-26 16:13:19 -07:00
dependabot[bot]
0b92604c92
Bump github.com/open-policy-agent/opa from 0.24.0 to 0.40.0 (#5906)
Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.24.0 to 0.40.0.
- [Release notes](https://github.com/open-policy-agent/opa/releases)
- [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-policy-agent/opa/compare/v0.24.0...v0.40.0)

---
updated-dependencies:
- dependency-name: github.com/open-policy-agent/opa
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-26 12:15:40 -03:00
Roberto Dip
894fa22c71
implement a thin wrapper around stdlib errors (#5733)
This solves #5679 , and also implements #5515, #5509 and lays the ground for #5516

With the introduction of Wrap, Is and As in the standard library, we've now got built-in support for wrapping.

On top of that, a common pattern in the community is to define errors tailored to the context of each project while still conforming to the error and Unwrap interfaces (see Upspin, Chromium)

The output now includes stack traces and additional info
2022-05-18 11:47:55 -03:00
Lucas Manuel Rodriguez
a5349672eb
Amend fleetctl package to support /var/lib legacy orbit (legacy would mean <= 0.0.11) (#5532)
* Add logs to troubleshoot orbit

* Run journalctl on a different step

* Add legacy orbit support to opt version of fleetctl

* Fix macos logs permission error

* Checkout repository

* Compile fleetctl from branch
2022-05-03 16:46:02 -03:00
gillespi314
1dabf52834
Add Zendesk external service integration for vulnerability automations (#5372) 2022-05-02 15:58:34 -05:00
Michal Nicpon
15c69058bb
fix SELinux issue (#5335)
Install orbit to /opt instead of /var/lib. When installing to /var/lib,
the default selinux context of var_lib_t gets applied, which results in
an AVC error when running via systemd.

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2022-05-02 12:18:59 -06:00
Zach Wasserman
e5a80fa3f5
Add user agent to Orbit HTTP client (#5429)
Allows identification of which Orbit versions are in use from the update
server.

Refactored the build information into a separate `package build` to
support importing it from multiple places.
2022-05-02 11:03:49 -07:00
Martin Angers
d890970844
Implement the Jira API call (#4938) 2022-04-05 12:58:34 -04:00
Lucas Manuel Rodriguez
c82c580716
Orbit: Add Fleet Desktop support to Windows (#4873)
* Orbit: Add Fleet Desktop support to Windows

* Rename workflow, fix linux build

* Do not compile systray on linux

* nolint on unused

* Fix lint properly

* nolint both checkers

* Fix monitor logic in desktopRunner

* Fix interrupt and execute order
2022-04-01 17:28:51 -03:00
dependabot[bot]
7f4c88ad22
Bump github.com/valyala/fasthttp from 1.31.0 to 1.34.0 (#4716)
Bumps [github.com/valyala/fasthttp](https://github.com/valyala/fasthttp) from 1.31.0 to 1.34.0.
- [Release notes](https://github.com/valyala/fasthttp/releases)
- [Commits](https://github.com/valyala/fasthttp/compare/v1.31.0...v1.34.0)
2022-03-28 10:45:15 -07:00
Lucas Manuel Rodriguez
378b1dd34f
Upgrade osquery-go (#4728)
* Upgrade and replace kolide/osquery-go with osquery/osquery-go

* Upgrade macadmins/osquery-extension to v0.0.7

* Upgrade kolide/launcher to latest

* go mod tidy
2022-03-23 12:15:37 -03:00
Lucas Manuel Rodriguez
ecdfd627b6
Fleet Desktop MVP (#4530)
* WIP

* WIP2

* Fix orbit and fleetctl tests

* Amend macos-app default

* Add some fixes

* Use fleetctl updates roots command

* Add more fixes to Updater

* Fixes to app publishing and downloading

* Add more changes to support fleetctl cross generation

* Amend comment

* Add pkg generation to ease testing

* Make more fixes

* Add changes entry

* Add legacy targets (until our TUF system exposes the new app)

* Fix fleetctl preview

* Fix bool flag

* Fix orbit logic for disabled-updates and dev-mode

* Fix TestPreview

* Remove constant and fix zip-slip attack (codeql)

* Return unknown error

* Fix updater's checkExec

* Add support for executable signing in init_tuf.sh

* Try only signing orbit

* Fix init_tuf.sh targets, macos-app only for osqueryd

* Specify GOARCH to support M1s

* Add workflow to generate osqueryd.app.tar.gz

* Use 5.2.2 on init_tuf.sh

* Add unit test for tar.gz target

* Use artifacts instead of releases

* Remove copy paste residue

* Fleet Desktop Packaging WIP

* Ignore gosec warning

* Trigger on PR too

* Install Go in workflow

* Pass url parameter to desktop app

* Fix fleetctl package

* Final set of changes for v1 of Fleet Desktop

* Add changes

* PR fixes

* Fix CI build

* add larger menu bar icon

* Add transparency item

* Delete host_device_auth entry on host deletion

* Add SetTargetChannel

* Update white logo and add desktop to update runner

* Add fleet-desktop monitoring to orbit

* Define fleet-desktop app exec name

* Fix update runner creation

* Add API test before enabling the My device menu item

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-03-21 14:53:53 -03:00
Benjamin Edwards
74bb559645
Add public ip to hosts & derive geolocation when rendering host (#4652)
* geoip wip
* return nil if ip is empty string or if ParseIP returns nil
* add ui component to render geolocation if available, address PR feedback
* render public ip if available
* add changes file, document geoip in deployment guide
* update rest-api docs
2022-03-21 12:29:52 -04:00
Martin Angers
bb678b6b2e
Add support for downloading a list of hosts in CSV format (#4596) 2022-03-15 15:14:42 -04:00
Lucas Manuel Rodriguez
c641e39ef4
Allow disabling auto updates in fleetctl package and orbit (#4296)
* Add disable-updates flag to fleetctl and orbit

* Fix ruleguard execution error on make lint-go

* Introduce dev-mode for ease of development of orbit

* Add changes file
2022-02-18 15:42:39 -03:00
Zachary Winnerman
c5c72ed713
Add apm for testing apm (#4053)
* Add apm for testing apm

* Testing opentracing

* testing

* Testing

* go fmt

* Add config switch for tracing.

* fixup

* Update cmd/fleet/serve.go

Co-authored-by: Tomas Touceda <chiiph@gmail.com>

* Add support for both elasticapm and opentelemetry

* Fix driver stuff and config options

* Fixup

* fixup

* Add changes file

* Add config for sql driver

* fixup

* Add doc to exported field

* testing

* fixup

* fixup

* Testing again

* fixup

* testing

* Undo

Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-02-15 12:42:22 -05:00
Lucas Manuel Rodriguez
aa9e257a2a
Upgrade fleetdm/goose version to latest (#4195) 2022-02-14 17:33:37 -03:00
Lucas Manuel Rodriguez
be72dc356c
Add CentOS parsing+post-processing to reduce false positives in vulnerability processing (#4037)
* Add CentOS parsing and post-processing in fleet

* Add tests and amend SyncCPEDatabase

* Add test for centosPostProcessing

* Changes from PR comments

* Amend software test

* Fix sync test

* Add index to source and vendor

* Use os.MkdirTemp

* Rearrange migrations

* Regenerate test schema

* Add support for testing migrations (#4112)

* Add support for testing migrations

* Rename migration in tests

* Changes suggested in PR

* Go mod tidy
2022-02-14 15:13:44 -03:00
Lucas Manuel Rodriguez
9c25ea1641
Prepare LoadHostByNodeKey query once (#4128)
* Prepare LoadHostByNodeKey query once

* Use a protected map for storing statements

* Add proposed test
2022-02-14 12:13:38 -03:00
Zach Wasserman
1b1d3f8a61
Update go-tuf dependency (#3837)
Resolves the warning described in #3699 by updating to the latest
version of the dependency with the warning fixed.

The warning should go away on all clients after new metadata is 
generated with these changes.
2022-02-10 08:16:36 -08:00
Lucas Manuel Rodriguez
81672ee50e
Add dev sql interceptor for logging of queries (#3815)
* Add dev sql interceptor for logging of queries

* Remove extra spaces and tabs from queries

* Make regex global
2022-01-21 14:28:21 -03:00
Tomas Touceda
9d572309ae
Add sentry (#3669)
* Add sentry

* Fix gosum

* More gosum fixes

* Add missing def for config

* Enrich sentry scope a bit

* Add changes file

* Add goroutine safe scope to errors

* Encapsulate sentry logic

* Add documentation for new flag

* Add sentry capturing to crons and other background tasks

* Only send to sentry when enabled
2022-01-20 16:41:02 -03:00
Martin Angers
e988d16eb3
Update the prometheus go client library (#3140) 2021-12-20 09:20:58 -05:00
Zach Wasserman
4e72ebb68f
Build Orbit releases in GitHub Actions (#3413)
Complete build including macOS signing+notarization.
2021-12-19 17:06:17 -08:00
Tomas Touceda
a0404b6736
Add software count API (#3105)
* Add software count API

* Fix makefile

* Fine no mock generating at this point

* Actually, one last try

* Use go install instead

* Fix go sum/mod

* Improve documentation

* Try setting node to 14
2021-12-03 10:54:17 -03:00
Tomas Touceda
a36e029d95
App config caching per instance (#3071)
* Do caching of app config per instance instead of across all of them in redis

* Add changes file

* Simplify code based on review comment

* Use go-cache instead of creating our own

* Dont export consts

* Copy app config before returning it

* Fix lint

* Update go sum

* Update go sum
2021-11-29 12:51:57 -03:00
Martin Angers
c997f853e5
Make creation of http.Client uniform across the codebase (#3097) 2021-11-24 15:56:54 -05:00
Victor Vrantchan
a760cdc26d
Add MacAdmins tables in Orbit extension (#2140)
Uses the extension tables from https://github.com/macadmins/osquery-extension.

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-11-15 18:40:53 -08:00
Lucas Manuel Rodriguez
5c1edaf527
Kill existing instances of orbit and osqueryd in fleetctl preview (#2921)
* Kill existing instances of orbit and osqueryd in fleetctl preview

* Add changes file
2021-11-15 18:12:25 -03:00
Zach Wasserman
e279dc1682
Implement fleetctl updates rotate (#2831)
Add support for updating keys used in the TUF update system.
2021-11-15 10:01:48 -08:00
Martin Angers
348841c90b
Add explanation output to fleetctl package (#2864) 2021-11-15 08:40:58 -05:00
Lucas Manuel Rodriguez
837bfed167
Try goqu as sql builder on listSoftwareDB (#2773)
* Try goqu as sql builder on listSoftwareDB

* Fix case insensitive search and add test
2021-11-04 15:21:39 -03:00
Tomas Touceda
2902da76ca
Issue 2027 better error visibility (#2069) 2021-11-02 13:35:57 -04:00
Lucas Manuel Rodriguez
43d1a8c9a0
Fix races in go tests and run with -race on CI (#2712)
* Fix races in go tests and run with -race on CI

* Fix race in pubsub

* Increase timeout to 15m for go tests

* CI takes forever, try disabling race

* Remove timeout from go tests
2021-11-01 15:31:01 -03:00
Tomas Touceda
825939e3dc
Improve osquery perf (#2503)
* Improve osquery perf

* Update dependencies

* Embed template so this can be run from the root of the repo

* Fix lint

* Address review comments
2021-10-14 10:09:58 -03:00
Tomas Touceda
8600d71d35
Add osquery perf (#2190)
* Add osquery perf

* Update dockerfile and gh action
2021-09-22 17:18:55 -03:00
Martin Angers
0ac4fc4c1a
Update redisc and replace EachRedisNode implementation (#1949) 2021-09-07 13:25:36 -04:00
Tomas Touceda
d6d53acda5
Issue 1600 fleetctl license expiration (#1800)
* Show banner in fleet and fleetctl if license expired

* Ignore if license is nil or tier is core

* Address review comments
2021-08-26 10:28:53 -03:00
Tomas Touceda
c6c63ab12a
Refactor app config (POC, for now) (#1685) 2021-08-20 12:27:41 -03:00
Tomas Touceda
96c0244c04
Fix redis locking receive (#1655)
* Receive redis data with timeout to not hold the connection

* Address review comments
2021-08-13 11:16:11 -03:00
Tomas Touceda
989e638cc0
Make creating dirs and files more secure by checking permissions (#1566)
* Add safe mkdirall and open

* Use secure as much as possible and merge gomodules for orbit to fleet

* Improve openfile and mkdirall to check for permissiveness instead of equality

* Don't shift

* Fix links

* Address review comments
2021-08-11 11:02:22 -03:00
Zach Wasserman
9db49a1128
Update JWT library to github.com/golang-jwt/jwt (#1568)
- The former repo has been migrated (see
  https://github.com/dgrijalva/jwt-go/issues/462).

- Dependabot alerted on a security issue in the old
  version (https://github.com/advisories/GHSA-w73w-5m7g-f7qc). This issue
  should not effect Fleet due to no use of `aud` claims.
2021-08-06 08:49:38 -07:00
Tomas Touceda
f8b7a83cc6
Process stored CPEs and store found CVEs (#1533)
* WIP

* WIP

* Make path optional and fix tests

* Add first generate

* Move to nvd package

* remove replace

* Re-add replace

* It's path, not file name

* Change how db path is set and use etag

* Fix typos

* Make db generation faster

* Remove quotes

* Doesn't like comments

* Samitize etag and save to file

* Refactor some things and improve writing of etagenv

* Compress file and truncate amount of items for faster testing

* Remove quotes

* Try to improve performance

* Ignore truncate error if not exists

* Minor cleanup and make sqlite have cpe prefix

* Simplify code and test sync

* Add VCR for sync test

* Check for nvdRelease nil

* Add test for the actual translation

* Address review comments

* Rename generate command because we'll have a cve one too

* Move to its own dir

* Add first cve db generation

* WIP but with final strategy, preparring to merge main

* Fix merge conflicts

* WIP

* wip

* Insert CVEs to the db

* Remove unused code

* Use wg instead of counting

* Call cancelFunc to avoid ctx leak

* Fix logs for better readability

* Point code to fleetdm instead of my repo
2021-08-04 18:01:39 -03:00