* improve error handling in vulnerabilities cron
* fix tests
* Use errHandler and go mod tidy
* Add dep that got removed by mod tidy
* add dsl to tools
* Add changes file
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* Create Bulk Users
* WIP: Adding a test for bulk user import
* adding a user bulk create test
* Fixing description, removing password required, and adding more test cases
* Fixing description, removing password required, and adding more test cases
* Fixed all comments and added Random Password Generator
* returning an error in generateRandomPassword
* Using 2 loops to create user list and then create the actual users
* Adding a bulk user delete
* fixing a mistake in temp csv
* fixed lints and removed yamlFlag
* Do not use golangci action for better reproducibility
* Add fix to trigger build
* Fix all reported issues
* fix more lint errors
* Add missing import
* Remove unused method
* Remove change not necessary
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts
To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
This solves #5679 , and also implements #5515, #5509 and lays the ground for #5516
With the introduction of Wrap, Is and As in the standard library, we've now got built-in support for wrapping.
On top of that, a common pattern in the community is to define errors tailored to the context of each project while still conforming to the error and Unwrap interfaces (see Upspin, Chromium)
The output now includes stack traces and additional info
* Add logs to troubleshoot orbit
* Run journalctl on a different step
* Add legacy orbit support to opt version of fleetctl
* Fix macos logs permission error
* Checkout repository
* Compile fleetctl from branch
Install orbit to /opt instead of /var/lib. When installing to /var/lib,
the default selinux context of var_lib_t gets applied, which results in
an AVC error when running via systemd.
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
Allows identification of which Orbit versions are in use from the update
server.
Refactored the build information into a separate `package build` to
support importing it from multiple places.
* Orbit: Add Fleet Desktop support to Windows
* Rename workflow, fix linux build
* Do not compile systray on linux
* nolint on unused
* Fix lint properly
* nolint both checkers
* Fix monitor logic in desktopRunner
* Fix interrupt and execute order
* Upgrade and replace kolide/osquery-go with osquery/osquery-go
* Upgrade macadmins/osquery-extension to v0.0.7
* Upgrade kolide/launcher to latest
* go mod tidy
* WIP
* WIP2
* Fix orbit and fleetctl tests
* Amend macos-app default
* Add some fixes
* Use fleetctl updates roots command
* Add more fixes to Updater
* Fixes to app publishing and downloading
* Add more changes to support fleetctl cross generation
* Amend comment
* Add pkg generation to ease testing
* Make more fixes
* Add changes entry
* Add legacy targets (until our TUF system exposes the new app)
* Fix fleetctl preview
* Fix bool flag
* Fix orbit logic for disabled-updates and dev-mode
* Fix TestPreview
* Remove constant and fix zip-slip attack (codeql)
* Return unknown error
* Fix updater's checkExec
* Add support for executable signing in init_tuf.sh
* Try only signing orbit
* Fix init_tuf.sh targets, macos-app only for osqueryd
* Specify GOARCH to support M1s
* Add workflow to generate osqueryd.app.tar.gz
* Use 5.2.2 on init_tuf.sh
* Add unit test for tar.gz target
* Use artifacts instead of releases
* Remove copy paste residue
* Fleet Desktop Packaging WIP
* Ignore gosec warning
* Trigger on PR too
* Install Go in workflow
* Pass url parameter to desktop app
* Fix fleetctl package
* Final set of changes for v1 of Fleet Desktop
* Add changes
* PR fixes
* Fix CI build
* add larger menu bar icon
* Add transparency item
* Delete host_device_auth entry on host deletion
* Add SetTargetChannel
* Update white logo and add desktop to update runner
* Add fleet-desktop monitoring to orbit
* Define fleet-desktop app exec name
* Fix update runner creation
* Add API test before enabling the My device menu item
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
* geoip wip
* return nil if ip is empty string or if ParseIP returns nil
* add ui component to render geolocation if available, address PR feedback
* render public ip if available
* add changes file, document geoip in deployment guide
* update rest-api docs
* Add disable-updates flag to fleetctl and orbit
* Fix ruleguard execution error on make lint-go
* Introduce dev-mode for ease of development of orbit
* Add changes file
* Add CentOS parsing and post-processing in fleet
* Add tests and amend SyncCPEDatabase
* Add test for centosPostProcessing
* Changes from PR comments
* Amend software test
* Fix sync test
* Add index to source and vendor
* Use os.MkdirTemp
* Rearrange migrations
* Regenerate test schema
* Add support for testing migrations (#4112)
* Add support for testing migrations
* Rename migration in tests
* Changes suggested in PR
* Go mod tidy
Resolves the warning described in #3699 by updating to the latest
version of the dependency with the warning fixed.
The warning should go away on all clients after new metadata is
generated with these changes.
* Add sentry
* Fix gosum
* More gosum fixes
* Add missing def for config
* Enrich sentry scope a bit
* Add changes file
* Add goroutine safe scope to errors
* Encapsulate sentry logic
* Add documentation for new flag
* Add sentry capturing to crons and other background tasks
* Only send to sentry when enabled
* Add software count API
* Fix makefile
* Fine no mock generating at this point
* Actually, one last try
* Use go install instead
* Fix go sum/mod
* Improve documentation
* Try setting node to 14
* Do caching of app config per instance instead of across all of them in redis
* Add changes file
* Simplify code based on review comment
* Use go-cache instead of creating our own
* Dont export consts
* Copy app config before returning it
* Fix lint
* Update go sum
* Update go sum
* Fix races in go tests and run with -race on CI
* Fix race in pubsub
* Increase timeout to 15m for go tests
* CI takes forever, try disabling race
* Remove timeout from go tests
* Add safe mkdirall and open
* Use secure as much as possible and merge gomodules for orbit to fleet
* Improve openfile and mkdirall to check for permissiveness instead of equality
* Don't shift
* Fix links
* Address review comments
* WIP
* WIP
* Make path optional and fix tests
* Add first generate
* Move to nvd package
* remove replace
* Re-add replace
* It's path, not file name
* Change how db path is set and use etag
* Fix typos
* Make db generation faster
* Remove quotes
* Doesn't like comments
* Samitize etag and save to file
* Refactor some things and improve writing of etagenv
* Compress file and truncate amount of items for faster testing
* Remove quotes
* Try to improve performance
* Ignore truncate error if not exists
* Minor cleanup and make sqlite have cpe prefix
* Simplify code and test sync
* Add VCR for sync test
* Check for nvdRelease nil
* Add test for the actual translation
* Address review comments
* Rename generate command because we'll have a cve one too
* Move to its own dir
* Add first cve db generation
* WIP but with final strategy, preparring to merge main
* Fix merge conflicts
* WIP
* wip
* Insert CVEs to the db
* Remove unused code
* Use wg instead of counting
* Call cancelFunc to avoid ctx leak
* Fix logs for better readability
* Point code to fleetdm instead of my repo
* WIP
* WIP
* Make path optional and fix tests
* Add first generate
* Move to nvd package
* remove replace
* Re-add replace
* It's path, not file name
* Change how db path is set and use etag
* Fix typos
* Make db generation faster
* Remove quotes
* Doesn't like comments
* Samitize etag and save to file
* Refactor some things and improve writing of etagenv
* Compress file and truncate amount of items for faster testing
* Remove quotes
* Try to improve performance
* Ignore truncate error if not exists
* Minor cleanup and make sqlite have cpe prefix
* Simplify code and test sync
* Add VCR for sync test
* Check for nvdRelease nil
* Add test for the actual translation
* Address review comments
* Rename generate command because we'll have a cve one too
* Move to its own dir
* Address review comments
- Use goreleaser to automate release process.
- Add new dockerfiles for fleet (with fleetctl) and fleetctl (only).
- Add GitHub Action Workflow to run goreleaser on new tag.
- Update NPM to match new archive naming.
This should support Redis in both cluster and non-cluster modes.
Updates were made separately to github.com/throttled/throttled to support the slight changes in types.
Co-authored-by: Joseph Macaulay <joseph.macaulay@uber.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
1. use [staticcheck](https://staticcheck.io/) to check the code, and fix some issues.
2. use `go fmt` to format the code.
3. use `go mod tidy` clean the go mod.
- Maintain software inventory with detail queries.
- Associated database migrations.
- Feature flagged off by default (see documentation for details to turn on).
- Documentation.
- New test helper for slice element comparisons skipping ID.
- Fix issue with built-in labels showing multiple platforms when hosts
are reinstalled with new platform.
- Add Red Hat Linux built-in label.
- Display more labels by default in target selector.
Fixes#546, #553
Prevent abuse of these endpoints with rate limiting backed by Redis. The
limits assigned should be appropriate for almost any Fleet deployment.
Closes#530
This PR contains the initial implementation of the fleetctl updates commands, along with documentation on using this to self-host an agent update server.
Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
- This seems to be the maintained, trusted (by Homebrew, etc.) version
of go-bindata.
- Add tools.go file to pin version with go modules.
- Use go run to run the binary, making easier configuration for new developers.
- Make the preview directory in the default .fleet directory.
- Check for Docker daemon installed but not running.
- Add message for Chrome users on self-signed certs.
- Display login information on later invocations of command.
- Remove "Kolide" from error messages.
Closes#190
Part of #197
This addresses an issue some users experienced in which performance
problems were encountered when hosts were "competing" for enrollment
using the same osquery host identifier. The issue is addressed by adding
a cooldown period for host enrollment, preventing the same (as judged by
osquery host identifier) host from enrolling more than once per minute.
When users end up in the problematic scenario, they will see quite a bit
of error logs due to this issue. For now that's probably a good thing as
users need to be aware of the lack of visibility. We can explore rate
limiting the logging if that becomes an issue for someone.
Fixes#102
- Add endpoints for osquery to register and continue a carve.
- Implement client functionality for retrieving carve details and contents in fleetctl.
- Add documentation on using file carving with Fleet.
Addresses kolide/fleet#1714
This change optimizes live queries by pushing the computation of query
targets to the creation time of the query, and efficiently caching the
targets in Redis. This results in a huge performance improvement at both
steady-state, and when running live queries.
- Live queries are stored using a bitfield in Redis, and takes
advantage of bitfield operations to be extremely efficient.
- Only run Redis live query test when REDIS_TEST is set in environment
- Ensure that live queries are only sent to hosts when there is a client
listening for results. Addresses an existing issue in Fleet along with
appropriate cleanup for the refactored live query backend.
Update the github.com/russellhaering/goxmldsig dependency and apply
the appropriate fixes for the API changes.
This is a preparation for integration with
github.com/AbGuthrie/goquery, which uses a newer version of the
dependency.
