* Add fleetctl command to get missing migrations
* Fix copy paste and lint
* Detect migrations applied out of order
* Add extra bullet to changes
* Trigger creation of migration status tables
* Fix unit tests
* PR fixes
* PR comment fixes
- Orbit automatically loads the flagfile when it exists in the orbit
root.
- Add packaging support to include flagfile with package.
- Fix a panic when osquery fails to start up.
* Add host count to software API
* Update docs
* Update fleetctl tests to account for host counts
* Update docs to mention host_count special case
* Update func comment
* Serialize hosts writes per instance
* Write hosts asynchronously
* Dont make the save in a goroutine
* Revert "Dont make the save in a goroutine"
This reverts commit 4a890c5271142755dec69a741582e7eca5c4c62c.
* Make all savehosts async
* Address review comments and make this approach configurable
* Address review comments
* Disable bulk seen time marking for a test
* Move host seen times to a new table
* Remove unused
* Add seen_time to list hosts
* Add some jitter to seen time flushing
* Remove unused
* Add timeout to deferred save host
* Add tests for serialSaveHost
* Update hosts in labels and policy executions in a serial way
* Address review comments and remove fk constraints in host software
* Make errCh buffered
* Add changes file
* Readd key
* Fix races in go tests and run with -race on CI
* Fix race in pubsub
* Increase timeout to 15m for go tests
* CI takes forever, try disabling race
* Remove timeout from go tests
* Make preview work when run from scratch (no orbit running)
* Make vulnerability processing less RAM hungry
* Add changes file
* Only get the cpe list once
* Remove cache
* Try killing osquery as well as orbit and clear their db dir
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
- Fix Windows MSI generation by changing permissions (#2655).
- Refactor temp directory initialization.
- Use root user for Wine in WiX Docker container.
- Support .pkg packaging on Linux without dependencies (besides Docker)
* Download and launch orbit for the current device when starting preview
* Add windows kill
* Fix imports
* Fix import
* Disable secure directory checks on windows for now
* Add missing import
* Start a fleetctl preview test
* Add tests for fleetctl preview
* Fix setting of fleetctl auth token in test
* Add fleet instance vulnerabilities config to response of GetAppConfig
* Add checks that fleetctl preview enables vulnerability detection
* Adjust doc for get config API response
* Add the include-server-config flag to fleetctl get config
* Update test now that some of the PRs have been merged
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* Implement fleetctl get software and the underlying API
* Add documentation
* Simplify list software implementation
* Lint fixes
* Make team name unique
* Address review comments
* Fix lint
* Fix tests
* Support close `LiveQueryResultsHandler`
* Start adding test
* Make LiveQuery exit when the context is Done
* Fix lint and remove debug print
* Update server/service/client_live_query.go
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
* Revert "Update server/service/client_live_query.go"
This reverts commit be67ca1512fe502503e821393c2b9e84f5e6e82e.
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
* added package command from orbit as fleetctl command
* update deployment docs
* add changes file
* added tests for package command, run go mod tidy & go mod verify
* validate that package files exist
* comment out msi packaging test until we can investigate github runner permission issues
* Rename core->free and basic->premium
* Fix lint js
* Comment out portion of test that seems to timeout
* Rename tier to premium if basic is still loaded
* fix get hosts command to properly output json/yaml based on command line flag
* add changes file
* added tests for get hosts when specifiying host
* added additional hosts to be returned in test cases
* go fmt
* wip
* Add tests and finish implementation
* Add proper default for periodicity, changes file, and documentation
* Fix tests and add defaults also to new installs
* EnableHostUsers should be true if undefined as well
* In some cases, periodicity can be zero because of the migrations
* Apply defaults when migrating appconfig
* Fix lint
* lint
* Address review comments
Add a relatively minimal set of linters that raise safe and
mostly un-opinionated issues with the code. It runs
automatically on CI via a github action.
Load queries before starting simulated hosts -- This is a much quicker
step and allows the user to see the queries in the interface while they
wait for hosts to start.
* add team_id filter to fleetctl via get hosts --team flag & api via api/v1/fleet/hosts and api/v1/fleet/labels/id/hosts
* update tests & add changes file
* Add safe mkdirall and open
* Use secure as much as possible and merge gomodules for orbit to fleet
* Improve openfile and mkdirall to check for permissiveness instead of equality
* Don't shift
* Fix links
* Address review comments
* fleetctl preview downloads standard query library and populates console
* add change log file, use specGroupFromBytes to parse standard query library file
* update printHost to serialize HostResponse rather than just Host
* added change log file
* Update changes/issue-1373-add-status-to-fleetctl-get-command
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* WIP
* Add get user_roles and apply for a user_roles spec to fleetctl
* Uncomment other tests
* Update test to check output
* Update test with the new struct
* Mock token so that it doesn't pick up the one in the local machine
* Address review comments
* Fix printJSON and printYaml
* Fix merge conflict error
* WIP
* wip
* wip
* Finish implementation
* Address review comments
* Fix flaky test
* WIP
* Add get user_roles and apply for a user_roles spec to fleetctl
* Uncomment other tests
* Update test to check output
* Update test with the new struct
* Mock token so that it doesn't pick up the one in the local machine
* Address review comments
* Fix printJSON and printYaml
* Fix merge conflict error
* If both roles are specified, fail
* Fix test
* Switch arguments around
* Update test with the new rule
* Fix other tests that fell through the cracks
* Remove username from UI code
* Remove username from tests
* Remove username from database
* Modify server endpoints for removing username
* Implement backend aspects of removing username
* Update API docs
* Add name to fleetctl
- Add enable_analytics column to database.
- Allow enable_analytics to be set via API.
- Add messaging in fleetctl setup.
Note that this defaults to off for existing installations, and defaults
on for newly set up installs.
No collection or sending of analytics yet exists, we are strictly
storing the preference at this time.
Part of #454
The previous strategy unintentionally cleared existing environment
values, preventing us from setting a specific version of the Fleet
container to run.
- Add policy.rego file defining authorization policies.
- Add Go integrations to evaluate Rego policies (via OPA).
- Add middleware to ensure requests without authorization check are rejected (guard against programmer error).
- Add authorization checks to most service endpoints.
- Add `team_id` field to secrets.
- Remove secret `name` and `active` fields (migration deletes inactive secrets).
- Assign hosts to Team based on secret provided.
- Add API for retrieving secrets by Team.
Some datastore and service methods would return slices of structs,
rather than slices to pointers of structs (which most methods used).
Make this more consistent.
- Allow agent options to be set on per-team basis.
- Move global agent options into app configs.
- Update logic for calculating agent options for hosts.
- Updates to relevant testing.
This PR contains the initial implementation of the fleetctl updates commands, along with documentation on using this to self-host an agent update server.
Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
- Return same error in all cases for login endpoint.
- Log error details in server logs.
- Make most login errors take ~1s to prevent timing attacks.
- Don't return forgot password errors.
- Log password errors in server logs.
- Make most forgot password requests take ~1s to prevent timing attacks.
Fixes#531
Make fleetctl preview available over HTTP
Run a second copy of the Fleet server listening over HTTP on
localhost:1337 so that the UI can be used without the errors displayed
with a self-signed TLS certificate. Osquery clients and fleetctl
continue to communicate with the existing Fleet server on
https://localhost:8412.
Closes#504
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
- Download configuration on every run. This ensures that users always
get the latest version.
- Pull latest Docker containers on every run.
- Add stop and reset commands (for cleanup).
Fixes#378
- Better documentation and error for missing docker-compose.
- Handle case of no existing config file.
- Make logs directory world-writable (to allow writes from inside container).
Fixes#286
- Make the preview directory in the default .fleet directory.
- Check for Docker daemon installed but not running.
- Add message for Chrome users on self-signed certs.
- Display login information on later invocations of command.
- Remove "Kolide" from error messages.
Closes#190
Part of #197
- Run the `fleetctl setup` command to set up the Fleet server with
default username ('admin') and password ('admin123#').
- Configures fleetctl if it has not yet been configured.
Closes#152
This adds a `fleetctl preview` command to fleetctl that will download
and start https://github.com/fleetdm/osquery-in-a-box. This is
potentially the easiest way for a user to try out Fleet.
Adds endpoints and fleetctl commands to retrieve various debug profiles
from the Fleet server.
The best summary is from the help text:
```
fleetctl debug
NAME:
fleetctl debug - Tools for debugging Fleet
USAGE:
fleetctl debug command [command options] [arguments...]
COMMANDS:
profile Record a CPU profile from the Fleet server.
cmdline Get the command line used to invoke the Fleet server.
heap Report the allocated memory in the Fleet server.
goroutine Get stack traces of all goroutines (threads) in the Fleet server.
trace Record an execution trace on the Fleet server.
archive Create an archive with the entire suite of debug profiles.
OPTIONS:
--config value Path to the Fleet config file (default: "/Users/zwass/.fleet/config") [$CONFIG]
--context value Name of Fleet config context to use (default: "default") [$CONTEXT]
--help, -h show help
```
- Add endpoints for osquery to register and continue a carve.
- Implement client functionality for retrieving carve details and contents in fleetctl.
- Add documentation on using file carving with Fleet.
Addresses kolide/fleet#1714
Fixes issue where `fleetctl get options` returns `option` (singular)
as `kind` instead of `options` (plural). This would cause `fleetctl
apply -f options.yml` to fail on options yaml generated by `fleetctl get
options` with this error: `unknown kind "option"`.
Getting a single host with `fleetctl get host foobar` will look up the
host with the matching hostname, uuid, osquery identifier, or node key,
and provide the full host details along with the labels the host is a
member of.
This commit takes advantage of the existing pagination APIs in the Fleet
server, and provides additional APIs to support pagination in the web
UI. Doing this dramatically reduces the response sizes for requests from
the UI, and limits the performance impact of UI clients on the Fleet and
MySQL servers.
Added a feature to select queries by name. If --query-name flag is defined, fleetctl makes a call to get the query already saved in Fleet, and then, if it exists, fill flQuery variable with the selected query statement.
Closes#2175
- Add the server_url_prefix flag for configuring this functionality
- Add prefix handling to the server routes
- Refactor JS to use appropriate paths from modules
- Use JS template to get URL prefix into JS environment
- Update webpack config to support prefixing
Thanks to securityonion.net for sponsoring the development of this feature.
Closes#1661
This commit:
- adds a new sub-command for fetching hosts to `fleetctl get` command.
Why?
- this allows for listing of all hosts via the fleetctl interface.
There may be additional attributes of the host that we'd like to
display, but this should be a good start.
Closes#1962
Previously we stripped all non-word characters which worked for some cases, but
removed necessary characters in others. What we really want (and now do) is to
remove only whitespace characters.
Fixes#1854
This PR adds support for getting resources by name.
```
$ fleetctl get queries
no queries found
$ fleetctl apply -f ./query.yaml
[+] applied 1 queries
$ fleetctl get queries
+-----------------+--------------------------------+--------------------------------+
| NAME | DESCRIPTION | QUERY |
+-----------------+--------------------------------+--------------------------------+
| osquery_version | The version of the Launcher | select launcher.version, |
| | and Osquery process | osquery.version from |
| | | kolide_launcher_info launcher, |
| | | osquery_info osquery; |
+-----------------+--------------------------------+--------------------------------+
$ fleetctl get query osquery_version
apiVersion: v1
kind: query
spec:
description: The version of the Launcher and Osquery process
name: osquery_version
query: select launcher.version, osquery.version from kolide_launcher_info launcher,
osquery_info osquery;
```
This PR implements a program called `fleetctl` which scaffolds a high level CLI interface which can be used to manage a Fleet server. Configuration is articulated using an intent-based API that resembles the API that is used to configure Kubernetes clusters. The idea here is to use the Kubernetes file format as a pattern to reduce the need for operators to become too intimately familiar with dramatically different file formats. (#1578)
