Related to #5898, this reports an anonymized summary of errors stored in Redis into the analytics payload.
For each error stored, this includes:
- A `count` attribute with the number of occurrences of the error
- A `loc` attribute with the 3 topmost lines in the stack trace. Note that stack traces only contain package name + line number (example: github.com/fleetdm/fleet/server.go:12
This also includes a minor refactor around error types.
* Create Bulk Users
* WIP: Adding a test for bulk user import
* adding a user bulk create test
* Fixing description, removing password required, and adding more test cases
* Fixing description, removing password required, and adding more test cases
* Fixed all comments and added Random Password Generator
* returning an error in generateRandomPassword
* Using 2 loops to create user list and then create the actual users
* Adding a bulk user delete
* fixing a mistake in temp csv
* fixed lints and removed yamlFlag
As reported in #6127, the `fleetctl debug` `archive` and `errors` commands were failing on Windows because filenames are not allowed to contain colons `:`.
This changeset removes colina from the filename of the archives generated by both commands.
* Do not use golangci action for better reproducibility
* Add fix to trigger build
* Fix all reported issues
* fix more lint errors
* Add missing import
* Remove unused method
* Remove change not necessary
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts
To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
* Add support for orbit_info to osquery-perf
* Add orbit_info support to osquery-perf
* Use 0.5 as default for orbit_info
* Fix CodeQL security warning on osquery-perf
This adds documentation about our debugging endpoints and a brief excerpt about the fleetctl debug command with instructions to generate the archive so we have a place with instructions to which we can direct people.
It also adds the .prof file extension to profiling files, which hopefully clarifies that they are meant to be used by go tool pprof.
This adds compatibility in `fleetctl preview` to work with docker compose (version 2). Since this version was released this April, we are still keeping backwards compatibility and using docker-compose as a fallback.
As v2 is now the recommended version and v1 is deprecated, this also rewords all prompts and help messages to say "docker compose".
Rel: #5746
this change prevents errors from being automatically cleared once they are read. A new flag `-flush` is introduced to flush errors on read if necessary.
Related to https://github.com/fleetdm/fleet/issues/5504, this change attempts to improve the output of the `fleetctl debug errors` command by:
- Adding a warning message to redact sensitive data
- Adding a `json` extension to the output file
- Allowing to stream the output to stdout via the `-stdout` flag or the `STDOUT` env var
The output after this changes is:
```
~/projects/fleet $ ./build/fleetctl debug errors
################################################################################
# WARNING:
# The generated file may contain sensitive data.
# Please review the file before sharing.
#
# Output written to: fleet-errors-2022-05-05T12:46:42-03:00.json
################################################################################
```
It also modifies the output of `fleetctl debug archive`
```
################################################################################
# WARNING:
# The files in the generated archive may contain sensitive data.
# Please review them before sharing.
#
# Archive written to: fleet-profiles-archive-2022-05-05T12:46:59-03:00.tar.gz
################################################################################
```
* Add fleetctl preview automation for latest changes
* Fix pwd invocation and remove slack notification
* Just run on ubuntu-latest and macos-latest
* Fix path
* Bug 5066: Format config durations
Change duration values returned by 'fleetctl get config --include-server-config' from nanoseconds to a human readable format.
* Remove required password reset flag when creating new API-only user
* Add test for 'fleetctl user create' command
* Set NewUserFuncInvoked to false in between tests
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
* Make Orbit update interval configurable
- Also increase default interval from 10s to 15m
* Add update-interval configuration to fleetctl package (#5050)
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
not set on the INSERT.
- OUT: Only sets the ID on the passed session and returns it. (`CreatedAt`, `AccessedAt`, are not set.)
New version:
```go
func (ds *Datastore) NewSession(ctx context.Context, userID uint, sessionKey string) (*fleet.Session, error) {
sqlStatement := `
INSERT INTO sessions (
user_id,
` + "`key`" + `
)
VALUES(?,?)
`
result, err := ds.writer.ExecContext(ctx, sqlStatement, userID, sessionKey)
if err != nil {
return nil, ctxerr.Wrap(ctx, err, "inserting session")
}
id, _ := result.LastInsertId() // cannot fail with the mysql driver
return ds.sessionByID(ctx, ds.writer, uint(id))
}
```
- IN: Define arguments that are truly used when creating a session.
- OUT: Load and return the fleet.Session struct with all values set (using the `ds.writer` to support read replicas correctly).
PS: The new `NewSession` version mimics what we already do with other entities, like policies (`Datastore.NewGlobalPolicy`).
* WIP
* WIP2
* Fix orbit and fleetctl tests
* Amend macos-app default
* Add some fixes
* Use fleetctl updates roots command
* Add more fixes to Updater
* Fixes to app publishing and downloading
* Add more changes to support fleetctl cross generation
* Amend comment
* Add pkg generation to ease testing
* Make more fixes
* Add changes entry
* Add legacy targets (until our TUF system exposes the new app)
* Fix fleetctl preview
* Fix bool flag
* Fix orbit logic for disabled-updates and dev-mode
* Fix TestPreview
* Remove constant and fix zip-slip attack (codeql)
* Return unknown error
* Fix updater's checkExec
* Add support for executable signing in init_tuf.sh
* Try only signing orbit
* Fix init_tuf.sh targets, macos-app only for osqueryd
* Specify GOARCH to support M1s
* Add workflow to generate osqueryd.app.tar.gz
* Use 5.2.2 on init_tuf.sh
* Add unit test for tar.gz target
* Use artifacts instead of releases
* Remove copy paste residue
* Fleet Desktop Packaging WIP
* Ignore gosec warning
* Trigger on PR too
* Install Go in workflow
* Pass url parameter to desktop app
* Fix fleetctl package
* Final set of changes for v1 of Fleet Desktop
* Add changes
* PR fixes
* Fix CI build
* add larger menu bar icon
* Add transparency item
* Delete host_device_auth entry on host deletion
* Add SetTargetChannel
* Update white logo and add desktop to update runner
* Add fleet-desktop monitoring to orbit
* Define fleet-desktop app exec name
* Fix update runner creation
* Add API test before enabling the My device menu item
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
* geoip wip
* return nil if ip is empty string or if ParseIP returns nil
* add ui component to render geolocation if available, address PR feedback
* render public ip if available
* add changes file, document geoip in deployment guide
* update rest-api docs
* WIP
* WIP2
* Fix orbit and fleetctl tests
* Amend macos-app default
* Add some fixes
* Use fleetctl updates roots command
* Add more fixes to Updater
* Fixes to app publishing and downloading
* Add more changes to support fleetctl cross generation
* Amend comment
* Add pkg generation to ease testing
* Make more fixes
* Add changes entry
* Add legacy targets (until our TUF system exposes the new app)
* Fix fleetctl preview
* Fix bool flag
* Fix orbit logic for disabled-updates and dev-mode
* Fix TestPreview
* Remove constant and fix zip-slip attack (codeql)
* Return unknown error
* Fix updater's checkExec
* Add support for executable signing in init_tuf.sh
* Try only signing orbit
* Fix init_tuf.sh targets, macos-app only for osqueryd
* Specify GOARCH to support M1s
* Add workflow to generate osqueryd.app.tar.gz
* Use 5.2.2 on init_tuf.sh
* Add unit test for tar.gz target
* Use artifacts instead of releases
* Remove copy paste residue
* Trigger workflow on PR
* Fixes to ease handling of artifact
* Fix, do not use target name as dir
* Remove workaround
* Add disable-updates flag to fleetctl and orbit
* Fix ruleguard execution error on make lint-go
* Introduce dev-mode for ease of development of orbit
* Add changes file
* Add CentOS parsing and post-processing in fleet
* Add tests and amend SyncCPEDatabase
* Add test for centosPostProcessing
* Changes from PR comments
* Amend software test
* Fix sync test
* Add index to source and vendor
* Use os.MkdirTemp
* Rearrange migrations
* Regenerate test schema
* Add support for testing migrations (#4112)
* Add support for testing migrations
* Rename migration in tests
* Changes suggested in PR
* Go mod tidy
* Change webhooks db lock duration to one hour
* Reload interval and check time left
* Change interval to one hour to match the db lock
* Add missing continue in case of failure
* Fix lint-go
