Commit Graph

93 Commits

Author SHA1 Message Date
Zachary Winnerman
2103de275a
improve error handling in vulnerabilities cron (#7276)
* improve error handling in vulnerabilities cron

* fix tests

* Use errHandler and go mod tidy

* Add dep that got removed by mod tidy

* add dsl to tools

* Add changes file

Co-authored-by: Michal Nicpon <michal@fleetdm.com>
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-08-18 16:53:58 -03:00
dependabot[bot]
6f0e0873ca
Bump golang from 1.18.4-bullseye to 1.19.0-bullseye (#7039)
* Bump golang from 1.18.4-bullseye to 1.19.0-bullseye

Bumps golang from 1.18.4-bullseye to 1.19.0-bullseye.

---
updated-dependencies:
- dependency-name: golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update other golang deps as well

* Update missing go mods

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-08-10 17:00:01 -03:00
gillespi314
3a88afaf98
Add new database tables and queries for host operating systems (#6920) 2022-08-09 13:34:41 -05:00
Michal Nicpon
d4be5ad2a1
add upgrade tests (#6596)
* add upgrade tests

* fix lint issues

go.mod

* remove req.cnf

* revert unrelated changes

* make version configurable in test

* fix golangci-lint ruleguard issue

Related to https://github.com/go-critic/go-critic/issues/1152
Need to have github.com/quasilyte/go-ruleguard/dsl

* fix lint issues

* fix

* clean up docker-compose.yml

* fix http request

* add readme

* fix lint issues

* address feedback

* fix

* add platform

* address feedback

* run go fmt
2022-07-19 15:11:51 -06:00
Tomas Touceda
7f8099db57
Add sandbox fleet serve config (#6619)
* Add sandbox fleet serve config

* Update docs

* Maybe fix lint issues

* Add lint dep
2022-07-12 18:21:15 -03:00
dependabot[bot]
b26a5cdb4e
Bump github.com/open-policy-agent/opa from 0.40.0 to 0.42.0 (#6537)
* Bump github.com/open-policy-agent/opa from 0.40.0 to 0.42.0

Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.40.0 to 0.42.0.
- [Release notes](https://github.com/open-policy-agent/opa/releases)
- [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-policy-agent/opa/compare/v0.40.0...v0.42.0)

---
updated-dependencies:
- dependency-name: github.com/open-policy-agent/opa
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Include go.* files in tests

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-07-11 14:58:27 -03:00
Ahmed Elshaer
a9f48ff561
Create Bulk Users from CSV (#3372)
* Create Bulk Users

* WIP: Adding a test for bulk user import

* adding a user bulk create test

* Fixing description, removing password required, and adding more test cases

* Fixing description, removing password required, and adding more test cases

* Fixed all comments and added Random Password Generator

* returning an error in generateRandomPassword

* Using 2 loops to create user list and then create the actual users

* Adding a bulk user delete

* fixing a mistake in temp csv

* fixed lints and removed yamlFlag
2022-06-22 13:34:58 -03:00
Lucas Manuel Rodriguez
4cfeaa1580
Do not use golangci action for better reproducibility (use make lint-go) (#6175)
* Do not use golangci action for better reproducibility

* Add fix to trigger build

* Fix all reported issues

* fix more lint errors

* Add missing import

* Remove unused method

* Remove change not necessary
2022-06-10 18:52:24 -03:00
Juan Fernandez
ef73039559
Improve vulnerability detection for Ubuntu (#6102)
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts

To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
2022-06-07 21:09:47 -04:00
Roberto Dip
5f5095de2c
bump go version in go.mod to 1.17 (#6129) 2022-06-07 16:05:55 -03:00
Michal Nicpon
30a864c8d2
Sync CVE scores periodically (#5838) 2022-06-01 10:06:57 -06:00
dependabot[bot]
092940b862
Bump github.com/theupdateframework/go-tuf from 0.2.0 to 0.3.0 (#5894)
* Bump github.com/theupdateframework/go-tuf from 0.2.0 to 0.3.0

Bumps [github.com/theupdateframework/go-tuf](https://github.com/theupdateframework/go-tuf) from 0.2.0 to 0.3.0.
- [Release notes](https://github.com/theupdateframework/go-tuf/releases)
- [Commits](https://github.com/theupdateframework/go-tuf/compare/v0.2.0...v0.3.0)

---
updated-dependencies:
- dependency-name: github.com/theupdateframework/go-tuf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Remove err check for client.IsLatestSnapshot

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-05-26 16:13:19 -07:00
dependabot[bot]
0b92604c92
Bump github.com/open-policy-agent/opa from 0.24.0 to 0.40.0 (#5906)
Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.24.0 to 0.40.0.
- [Release notes](https://github.com/open-policy-agent/opa/releases)
- [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-policy-agent/opa/compare/v0.24.0...v0.40.0)

---
updated-dependencies:
- dependency-name: github.com/open-policy-agent/opa
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-26 12:15:40 -03:00
Roberto Dip
894fa22c71
implement a thin wrapper around stdlib errors (#5733)
This solves #5679 , and also implements #5515, #5509 and lays the ground for #5516

With the introduction of Wrap, Is and As in the standard library, we've now got built-in support for wrapping.

On top of that, a common pattern in the community is to define errors tailored to the context of each project while still conforming to the error and Unwrap interfaces (see Upspin, Chromium)

The output now includes stack traces and additional info
2022-05-18 11:47:55 -03:00
Lucas Manuel Rodriguez
a5349672eb
Amend fleetctl package to support /var/lib legacy orbit (legacy would mean <= 0.0.11) (#5532)
* Add logs to troubleshoot orbit

* Run journalctl on a different step

* Add legacy orbit support to opt version of fleetctl

* Fix macos logs permission error

* Checkout repository

* Compile fleetctl from branch
2022-05-03 16:46:02 -03:00
gillespi314
1dabf52834
Add Zendesk external service integration for vulnerability automations (#5372) 2022-05-02 15:58:34 -05:00
Michal Nicpon
15c69058bb
fix SELinux issue (#5335)
Install orbit to /opt instead of /var/lib. When installing to /var/lib,
the default selinux context of var_lib_t gets applied, which results in
an AVC error when running via systemd.

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2022-05-02 12:18:59 -06:00
Zach Wasserman
e5a80fa3f5
Add user agent to Orbit HTTP client (#5429)
Allows identification of which Orbit versions are in use from the update
server.

Refactored the build information into a separate `package build` to
support importing it from multiple places.
2022-05-02 11:03:49 -07:00
Martin Angers
d890970844
Implement the Jira API call (#4938) 2022-04-05 12:58:34 -04:00
Lucas Manuel Rodriguez
c82c580716
Orbit: Add Fleet Desktop support to Windows (#4873)
* Orbit: Add Fleet Desktop support to Windows

* Rename workflow, fix linux build

* Do not compile systray on linux

* nolint on unused

* Fix lint properly

* nolint both checkers

* Fix monitor logic in desktopRunner

* Fix interrupt and execute order
2022-04-01 17:28:51 -03:00
dependabot[bot]
7f4c88ad22
Bump github.com/valyala/fasthttp from 1.31.0 to 1.34.0 (#4716)
Bumps [github.com/valyala/fasthttp](https://github.com/valyala/fasthttp) from 1.31.0 to 1.34.0.
- [Release notes](https://github.com/valyala/fasthttp/releases)
- [Commits](https://github.com/valyala/fasthttp/compare/v1.31.0...v1.34.0)
2022-03-28 10:45:15 -07:00
Lucas Manuel Rodriguez
378b1dd34f
Upgrade osquery-go (#4728)
* Upgrade and replace kolide/osquery-go with osquery/osquery-go

* Upgrade macadmins/osquery-extension to v0.0.7

* Upgrade kolide/launcher to latest

* go mod tidy
2022-03-23 12:15:37 -03:00
Lucas Manuel Rodriguez
ecdfd627b6
Fleet Desktop MVP (#4530)
* WIP

* WIP2

* Fix orbit and fleetctl tests

* Amend macos-app default

* Add some fixes

* Use fleetctl updates roots command

* Add more fixes to Updater

* Fixes to app publishing and downloading

* Add more changes to support fleetctl cross generation

* Amend comment

* Add pkg generation to ease testing

* Make more fixes

* Add changes entry

* Add legacy targets (until our TUF system exposes the new app)

* Fix fleetctl preview

* Fix bool flag

* Fix orbit logic for disabled-updates and dev-mode

* Fix TestPreview

* Remove constant and fix zip-slip attack (codeql)

* Return unknown error

* Fix updater's checkExec

* Add support for executable signing in init_tuf.sh

* Try only signing orbit

* Fix init_tuf.sh targets, macos-app only for osqueryd

* Specify GOARCH to support M1s

* Add workflow to generate osqueryd.app.tar.gz

* Use 5.2.2 on init_tuf.sh

* Add unit test for tar.gz target

* Use artifacts instead of releases

* Remove copy paste residue

* Fleet Desktop Packaging WIP

* Ignore gosec warning

* Trigger on PR too

* Install Go in workflow

* Pass url parameter to desktop app

* Fix fleetctl package

* Final set of changes for v1 of Fleet Desktop

* Add changes

* PR fixes

* Fix CI build

* add larger menu bar icon

* Add transparency item

* Delete host_device_auth entry on host deletion

* Add SetTargetChannel

* Update white logo and add desktop to update runner

* Add fleet-desktop monitoring to orbit

* Define fleet-desktop app exec name

* Fix update runner creation

* Add API test before enabling the My device menu item

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-03-21 14:53:53 -03:00
Benjamin Edwards
74bb559645
Add public ip to hosts & derive geolocation when rendering host (#4652)
* geoip wip
* return nil if ip is empty string or if ParseIP returns nil
* add ui component to render geolocation if available, address PR feedback
* render public ip if available
* add changes file, document geoip in deployment guide
* update rest-api docs
2022-03-21 12:29:52 -04:00
Martin Angers
bb678b6b2e
Add support for downloading a list of hosts in CSV format (#4596) 2022-03-15 15:14:42 -04:00
Lucas Manuel Rodriguez
c641e39ef4
Allow disabling auto updates in fleetctl package and orbit (#4296)
* Add disable-updates flag to fleetctl and orbit

* Fix ruleguard execution error on make lint-go

* Introduce dev-mode for ease of development of orbit

* Add changes file
2022-02-18 15:42:39 -03:00
Zachary Winnerman
c5c72ed713
Add apm for testing apm (#4053)
* Add apm for testing apm

* Testing opentracing

* testing

* Testing

* go fmt

* Add config switch for tracing.

* fixup

* Update cmd/fleet/serve.go

Co-authored-by: Tomas Touceda <chiiph@gmail.com>

* Add support for both elasticapm and opentelemetry

* Fix driver stuff and config options

* Fixup

* fixup

* Add changes file

* Add config for sql driver

* fixup

* Add doc to exported field

* testing

* fixup

* fixup

* Testing again

* fixup

* testing

* Undo

Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-02-15 12:42:22 -05:00
Lucas Manuel Rodriguez
aa9e257a2a
Upgrade fleetdm/goose version to latest (#4195) 2022-02-14 17:33:37 -03:00
Lucas Manuel Rodriguez
be72dc356c
Add CentOS parsing+post-processing to reduce false positives in vulnerability processing (#4037)
* Add CentOS parsing and post-processing in fleet

* Add tests and amend SyncCPEDatabase

* Add test for centosPostProcessing

* Changes from PR comments

* Amend software test

* Fix sync test

* Add index to source and vendor

* Use os.MkdirTemp

* Rearrange migrations

* Regenerate test schema

* Add support for testing migrations (#4112)

* Add support for testing migrations

* Rename migration in tests

* Changes suggested in PR

* Go mod tidy
2022-02-14 15:13:44 -03:00
Lucas Manuel Rodriguez
9c25ea1641
Prepare LoadHostByNodeKey query once (#4128)
* Prepare LoadHostByNodeKey query once

* Use a protected map for storing statements

* Add proposed test
2022-02-14 12:13:38 -03:00
Zach Wasserman
1b1d3f8a61
Update go-tuf dependency (#3837)
Resolves the warning described in #3699 by updating to the latest
version of the dependency with the warning fixed.

The warning should go away on all clients after new metadata is 
generated with these changes.
2022-02-10 08:16:36 -08:00
Lucas Manuel Rodriguez
81672ee50e
Add dev sql interceptor for logging of queries (#3815)
* Add dev sql interceptor for logging of queries

* Remove extra spaces and tabs from queries

* Make regex global
2022-01-21 14:28:21 -03:00
Tomas Touceda
9d572309ae
Add sentry (#3669)
* Add sentry

* Fix gosum

* More gosum fixes

* Add missing def for config

* Enrich sentry scope a bit

* Add changes file

* Add goroutine safe scope to errors

* Encapsulate sentry logic

* Add documentation for new flag

* Add sentry capturing to crons and other background tasks

* Only send to sentry when enabled
2022-01-20 16:41:02 -03:00
Martin Angers
e988d16eb3
Update the prometheus go client library (#3140) 2021-12-20 09:20:58 -05:00
Zach Wasserman
4e72ebb68f
Build Orbit releases in GitHub Actions (#3413)
Complete build including macOS signing+notarization.
2021-12-19 17:06:17 -08:00
Tomas Touceda
a0404b6736
Add software count API (#3105)
* Add software count API

* Fix makefile

* Fine no mock generating at this point

* Actually, one last try

* Use go install instead

* Fix go sum/mod

* Improve documentation

* Try setting node to 14
2021-12-03 10:54:17 -03:00
Tomas Touceda
a36e029d95
App config caching per instance (#3071)
* Do caching of app config per instance instead of across all of them in redis

* Add changes file

* Simplify code based on review comment

* Use go-cache instead of creating our own

* Dont export consts

* Copy app config before returning it

* Fix lint

* Update go sum

* Update go sum
2021-11-29 12:51:57 -03:00
Martin Angers
c997f853e5
Make creation of http.Client uniform across the codebase (#3097) 2021-11-24 15:56:54 -05:00
Victor Vrantchan
a760cdc26d
Add MacAdmins tables in Orbit extension (#2140)
Uses the extension tables from https://github.com/macadmins/osquery-extension.

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-11-15 18:40:53 -08:00
Lucas Manuel Rodriguez
5c1edaf527
Kill existing instances of orbit and osqueryd in fleetctl preview (#2921)
* Kill existing instances of orbit and osqueryd in fleetctl preview

* Add changes file
2021-11-15 18:12:25 -03:00
Zach Wasserman
e279dc1682
Implement fleetctl updates rotate (#2831)
Add support for updating keys used in the TUF update system.
2021-11-15 10:01:48 -08:00
Martin Angers
348841c90b
Add explanation output to fleetctl package (#2864) 2021-11-15 08:40:58 -05:00
Lucas Manuel Rodriguez
837bfed167
Try goqu as sql builder on listSoftwareDB (#2773)
* Try goqu as sql builder on listSoftwareDB

* Fix case insensitive search and add test
2021-11-04 15:21:39 -03:00
Tomas Touceda
2902da76ca
Issue 2027 better error visibility (#2069) 2021-11-02 13:35:57 -04:00
Lucas Manuel Rodriguez
43d1a8c9a0
Fix races in go tests and run with -race on CI (#2712)
* Fix races in go tests and run with -race on CI

* Fix race in pubsub

* Increase timeout to 15m for go tests

* CI takes forever, try disabling race

* Remove timeout from go tests
2021-11-01 15:31:01 -03:00
Tomas Touceda
825939e3dc
Improve osquery perf (#2503)
* Improve osquery perf

* Update dependencies

* Embed template so this can be run from the root of the repo

* Fix lint

* Address review comments
2021-10-14 10:09:58 -03:00
Tomas Touceda
8600d71d35
Add osquery perf (#2190)
* Add osquery perf

* Update dockerfile and gh action
2021-09-22 17:18:55 -03:00
Martin Angers
0ac4fc4c1a
Update redisc and replace EachRedisNode implementation (#1949) 2021-09-07 13:25:36 -04:00
Tomas Touceda
d6d53acda5
Issue 1600 fleetctl license expiration (#1800)
* Show banner in fleet and fleetctl if license expired

* Ignore if license is nil or tier is core

* Address review comments
2021-08-26 10:28:53 -03:00
Tomas Touceda
c6c63ab12a
Refactor app config (POC, for now) (#1685) 2021-08-20 12:27:41 -03:00