Commit Graph

215 Commits

Author SHA1 Message Date
Zachary Winnerman
2fbe53b6c9
Reorg infrastructure and add changes for frontend's loadtesting environment (#4947)
* Reorganized infrastructure, updated for frontend's loadtesting

* Add changes suggested by @chiiph

* Moved files per suggestion by Ben

* Update docs with new links

* Add config for multi account assume role
2022-04-12 12:49:00 -04:00
Zach Wasserman
d15957b431
Make Orbit update interval configurable (#5032)
* Make Orbit update interval configurable

- Also increase default interval from 10s to 15m

* Add update-interval configuration to fleetctl package (#5050)

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
2022-04-11 17:42:36 -03:00
Martin Angers
7187f1adac
Queue jobs for Jira integration when enabled and new vulnerabilities are found. (#4975) 2022-04-11 16:42:16 -04:00
Lucas Manuel Rodriguez
60b7425bef
Add http basic auth to /metrics (#4974)
* Add http basic auth to /metrics

* Fixes after testing applying of a --config sample.yml

* Add unit test
2022-04-07 09:40:53 -03:00
Martin Angers
90b15071a4
Introduce API version 2022-04, deprecate use of /global in paths (#4731) 2022-04-05 11:35:53 -04:00
Lucas Manuel Rodriguez
57816592ba
Add read replica testing helpers and fix non-sso login bug (#4908)
not set on the INSERT.
- OUT: Only sets the ID on the passed session and returns it. (`CreatedAt`, `AccessedAt`, are not set.)

New version:

```go
func (ds *Datastore) NewSession(ctx context.Context, userID uint, sessionKey string) (*fleet.Session, error) {
	sqlStatement := `
		INSERT INTO sessions (
			user_id,
			` + "`key`" + `
		)
		VALUES(?,?)
	`
	result, err := ds.writer.ExecContext(ctx, sqlStatement, userID, sessionKey)
	if err != nil {
		return nil, ctxerr.Wrap(ctx, err, "inserting session")
	}

	id, _ := result.LastInsertId() // cannot fail with the mysql driver
	return ds.sessionByID(ctx, ds.writer, uint(id))
}
```

- IN: Define arguments that are truly used when creating a session.
- OUT: Load and return the fleet.Session struct with all values set (using the `ds.writer` to support read replicas correctly).

PS: The new `NewSession` version mimics what we already do with other entities, like policies (`Datastore.NewGlobalPolicy`).
2022-04-04 16:52:05 -07:00
Lucas Manuel Rodriguez
c82c580716
Orbit: Add Fleet Desktop support to Windows (#4873)
* Orbit: Add Fleet Desktop support to Windows

* Rename workflow, fix linux build

* Do not compile systray on linux

* nolint on unused

* Fix lint properly

* nolint both checkers

* Fix monitor logic in desktopRunner

* Fix interrupt and execute order
2022-04-01 17:28:51 -03:00
Zachary Winnerman
2ea256b9b0
Fix codeowners and revert pull 4798 as it stepped on 4834 (#4887) 2022-03-31 12:36:18 -04:00
Guillaume Ross
22dda3adf5
First batch of tf security improvements (#4798) 2022-03-30 13:24:30 -04:00
Zachary Winnerman
0e9a3ff044
Add some TF fixes (#4834) 2022-03-28 12:08:57 -04:00
Noah Talerman
8e9a814d4b
Prepare for 4.12.0 (#4797)
- Update CHANGELOG
- Bump versioning
- Tweak documentation
  - Default `session_duration` to `5d`
  - Add extra `#` to "Team policies" section so it doesn't show up in top level nav for docs
2022-03-24 17:24:08 -07:00
Guillaume Ross
07689baa39
Update ecs.tf (#4775)
Adding feature to drop invalid headers at LB level. This closes #4774
2022-03-24 15:00:31 -04:00
Zach Wasserman
cc687d9d1e
Add Notarization for Fleet Desktop (#4720) 2022-03-21 15:01:50 -07:00
Lucas Manuel Rodriguez
ecdfd627b6
Fleet Desktop MVP (#4530)
* WIP

* WIP2

* Fix orbit and fleetctl tests

* Amend macos-app default

* Add some fixes

* Use fleetctl updates roots command

* Add more fixes to Updater

* Fixes to app publishing and downloading

* Add more changes to support fleetctl cross generation

* Amend comment

* Add pkg generation to ease testing

* Make more fixes

* Add changes entry

* Add legacy targets (until our TUF system exposes the new app)

* Fix fleetctl preview

* Fix bool flag

* Fix orbit logic for disabled-updates and dev-mode

* Fix TestPreview

* Remove constant and fix zip-slip attack (codeql)

* Return unknown error

* Fix updater's checkExec

* Add support for executable signing in init_tuf.sh

* Try only signing orbit

* Fix init_tuf.sh targets, macos-app only for osqueryd

* Specify GOARCH to support M1s

* Add workflow to generate osqueryd.app.tar.gz

* Use 5.2.2 on init_tuf.sh

* Add unit test for tar.gz target

* Use artifacts instead of releases

* Remove copy paste residue

* Fleet Desktop Packaging WIP

* Ignore gosec warning

* Trigger on PR too

* Install Go in workflow

* Pass url parameter to desktop app

* Fix fleetctl package

* Final set of changes for v1 of Fleet Desktop

* Add changes

* PR fixes

* Fix CI build

* add larger menu bar icon

* Add transparency item

* Delete host_device_auth entry on host deletion

* Add SetTargetChannel

* Update white logo and add desktop to update runner

* Add fleet-desktop monitoring to orbit

* Define fleet-desktop app exec name

* Fix update runner creation

* Add API test before enabling the My device menu item

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-03-21 14:53:53 -03:00
Zachary Winnerman
92cd700b4d
Add new loadtesting infra and scaledown option to save costs (#4662) 2022-03-21 13:18:21 -04:00
Lucas Manuel Rodriguez
f4d3159cc9
Fleetctl to package .app bundles for osquery (and changes for orbit to support them) (#4393)
* WIP

* WIP2

* Fix orbit and fleetctl tests

* Amend macos-app default

* Add some fixes

* Use fleetctl updates roots command

* Add more fixes to Updater

* Fixes to app publishing and downloading

* Add more changes to support fleetctl cross generation

* Amend comment

* Add pkg generation to ease testing

* Make more fixes

* Add changes entry

* Add legacy targets (until our TUF system exposes the new app)

* Fix fleetctl preview

* Fix bool flag

* Fix orbit logic for disabled-updates and dev-mode

* Fix TestPreview

* Remove constant and fix zip-slip attack (codeql)

* Return unknown error

* Fix updater's checkExec

* Add support for executable signing in init_tuf.sh

* Try only signing orbit

* Fix init_tuf.sh targets, macos-app only for osqueryd

* Specify GOARCH to support M1s

* Add workflow to generate osqueryd.app.tar.gz

* Use 5.2.2 on init_tuf.sh

* Add unit test for tar.gz target

* Use artifacts instead of releases

* Remove copy paste residue

* Trigger workflow on PR

* Fixes to ease handling of artifact

* Fix, do not use target name as dir

* Remove workaround
2022-03-15 16:04:12 -03:00
Noah Talerman
937cd7f2eb
Update changelog and bump versioning (#4497) 2022-03-07 11:08:51 -06:00
eashaw
0af5e161d0
Remove numbers from documentation filenames in Fleet repo (#4313)
* Renaming files and a lot of find and replace

* pageRank meta tags, sorting by page rank

* reranking

* removing numbers

* revert changing links that are locked to a commit

* update metatag name, uncomment github contributers

* Update basic-documentation.page.js

* revert link change

* more explicit errors, change pageOrderInSection numbers, updated sort

* Update build-static-content.js

* update comment

* update handbook link

* handbook entry

* update sort

* update changelog doc links to use fleetdm.com

* move standard query library back to old location, update links/references to location

* revert unintentional link changes

* Update handbook/community.md

Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>

Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
2022-02-23 12:17:55 -06:00
Lucas Manuel Rodriguez
b5be858071
Fix update checks for orbit at startup (#3835)
* Fix update checks for orbit at startup

* Add tests

* Add scripts for testing local TUF server

* Remove -x used for debugging
2022-02-23 14:58:07 -03:00
Lucas Manuel Rodriguez
e39ba93a90
Orbit to wait for osqueryd extension manager socket to be ready (#3836)
* Wait for osquery extension manager socket to be ready

* PR review feedback

* Add WIP Github Action to test orbit

* Set fleet address on fleetctl

* Add logging for troubleshooting

* Add prepare db statement

* Use tls-skip-verify on fleetctl

* Move steps around

* Fix addresses

* Fix fleetctl path

* Add certs.pem to orbit root dir

* Add orbit logs

* Increase timeout

* Add proper orbit log paths

* Fix tmp path orbit

* Add get hosts command to troubleshoot

* Fix orbit job termination

* Add comments to workflows

* Wait for server to go down

* Add orbit Windows job

* Use bash on windws-latest

* Fix missing quote

* Run orbit on Ubuntu

* Bump host count

* Increase timeout for extensions and not terminate on err

* Add comment to clarify high timeout value

* Revert change, we do want to exit in case of err
2022-02-22 15:05:32 -03:00
RachelElysia
7cdc7bbec2
E2e Test: Team admin flow (#4301) 2022-02-21 21:09:36 -05:00
Benjamin Edwards
12eac152c4
GCP Terraform (#4303)
* gcp wip
* more edits on services, secrets manager, readme
* updated readme with required variables
2022-02-18 20:01:42 -05:00
Zachary Winnerman
899a643c15
Elastic is now managed through ansible for quicker config changes and more stability (#4243)
* Add elasticsearch being managed through ansible for better managability

* Testing

* testing

* testing

* testing

* testing

* testing

* testing

* fixup

* fixup

* Added docker stuff

* fixup

* fixup

* fixup

* Remove old code

* fixup

* fix health checks

* Add elastic apm configuration

* fixup

* fixup

* fixup

* testing

* restart always

* fixup

* fixup
2022-02-18 11:07:32 -05:00
Zachary Winnerman
b72e9bd1c1
Split elasticsearch's security groups (#4231) 2022-02-16 11:11:39 -05:00
Zachary Winnerman
69c0434e88
Add terraform for the loadtesting environment (#4172)
* Add terraform for the loadtesting environment

* Add some checkov stuff and minor updates

* Remove defunct commented stuff

* Add separate cloudwatch namespace for migrations

* Remove defunct code

* checkin
2022-02-15 13:00:24 -05:00
dependabot[bot]
8589cdd553
Bump follow-redirects from 1.14.4 to 1.14.8 in /tools/fleetctl-npm (#4183)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.4 to 1.14.8.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.14.4...v1.14.8)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-15 10:08:07 -06:00
Noah Talerman
67827474c2
Prepare for Fleet 4.10.0 (#4161)
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-02-13 18:13:06 -08:00
eashaw
9af92b23aa
Update links to documentation (#4163)
* update links to documentation

* revert .sailsrc change

* fix broken link
2022-02-10 19:26:18 -06:00
Zach Wasserman
600549d8a4
Prepare for 4.9.1 release (#3996) (#4005)
Merging back to `main` from patch branch.
2022-02-10 10:46:21 -05:00
Luke Heath
5866d68f13
Set max length for name inputs (#4063) 2022-02-08 19:40:38 -06:00
Benjamin Edwards
d4312bec59
remove broken terraform references (#3856) 2022-01-24 15:58:36 -05:00
Tomas Touceda
35f806f7d1
Merge Release candidate 4.9.0 into main (#3844)
* Better jitter (#3716) (#3744)

* Better jitter (#3716)

* Better jitter

* Fix lint

* Use milliseconds

* Make duration milliseconds

* Update based on Lucas' suggestion

* Add changes file

* Panic on error

* Fix compilation error

* Use time.after instead of time.tick to not leak

* Include browser extensions in software inventory (#3733) (#3757)

Use appropriate JOINs against users table to include all results.

For #3557

* Build Docker images on push to any branch (#3756) (#3758)

Allows for testing release branches and any others.

* Optimize users detail query (#3754) (#3770)

@Smjert reported instances of Windows Domain Controllers having massive
resource utilization and being killed by the watchdog when running this
query. In his test environment, this new query performs much better.

* Refactor async host processing to avoid redis SCAN keys (for policies) (#3657) (#3755)

* Issue 3707 clean targets on delete (#3739) (#3776)

* wip

* Delete targets when deleting teams, hosts, and labels

* Add changes file

* Fix error message

* Remove unused teamsTable

* Cleanup new pack

* Clean new packs at end of test

* Update operating system policy (#3779)

- Update policy's query to check for operating system versions greater than or equal to

* Improve loading of manage policies page (#3695) (#3781)

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

* Fix logo (#3765)

* fixed duplicate query modal not closing (#3787)

* fixed style for connection line on activity feed (#3789)

* Improve loading state for query platform compatibility (#3752) (#3783)

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

* Disable pagination for teams page (#3794)

* Shorten "simple" query API period to 25s (#3775) (#3804)

This helps the period stay under the default request timeouts for most
load balancers.

Some default timeouts:
* AWS ALB - 60s
* Nginx - 60s
* GCP LB - 30s

* Clear user checkbox selections after successfully removing selected row from table UI #3798 (#3801)

* Alias fleetctl sandbox to fleetctl preview (#3803) (#3806)

Preparing for some work the product team is planning to rebrand preview
to sandbox.

* Remove flash of spinner to no spinner (#3799)

* Revert "Better jitter (#3716) (#3744)" (#3807)

This reverts commit c857b52d2b373ed9e302d643e6d2212af7353408.

* Fix teams dropdown sizing (#3759) (#3810)

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

* fixed link for expiration message (#3788)

* Fixed empty string for compatible platforms (#3812)

* fixed empty string for compatible platforms

* default checked for platforms on policy query

* Block autocomplete for username and password in SMTP settings (#3732) (#3816)

* No hosts for software/policy renders better message (#3701) (#3821)

Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>

* Get failing policies webhook data on page load (#3824)

* Fix reset password flow (#3826)

* Prepare for 4.9.0 (#3820)

- Add CHANGELOG
- Bump versioning
- Remove changes files
- Spelling fix for operating system policy's description

* Remove merge artifact

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>
2022-01-24 15:32:25 -03:00
Benjamin Edwards
d650423be0
Reference Architectures (#3712)
Added reference architectures using https://docs.gitlab.com/ee/administration/reference_architectures/ as inspiration.

- updated terraform based on some feedback of usage
- pinned fleet docker version in terraform as to no get unexpected upgrades when applying
- updated some documentation around apply migration tasks
2022-01-21 16:27:55 -08:00
Tomas Touceda
43b78e6d0d
Add redis stress (#3363)
* Add redis stress tool

* Parametrize more things

* Rename folder back

* Log error

* Fix lint
2022-01-20 16:18:17 -03:00
Benjamin Edwards
6b551acf95
add percona terraform (#3468) 2022-01-06 18:07:40 -05:00
Noah Talerman
09654d77ee
Prepare for Fleet 4.8.0 (#3536) 2021-12-31 09:58:08 -08:00
Noah Talerman
8957f00d86
Prepare for Fleet 4.7.0 (#3355) 2021-12-14 11:25:13 -08:00
Zach Wasserman
a6d60cc193
Configure nofiles ulimit in Terraform ECS task (#3249)
The low default ulimit `nofiles` value (`4096`) in Fargate was observed
to cause errors when running with a large number of hosts and a small
number of servers. Each server should be able to server more than 4096
simultaneous clients.
2021-12-08 15:08:48 -08:00
Tomas Touceda
142006cbdd
Issue 2099 enable vulnerability processing new installs (#3104)
* Enable vulnerability processing by default

* Add changes file

* Remove wrong env var

* Fix import

* Fix tests

* Update log line

* Remove unneeded import
2021-12-03 14:44:57 -03:00
Tomas Touceda
052e41851b
Prepare for Fleet 4.6.2 (#3146) (#3179)
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
2021-12-02 16:56:56 -03:00
Martin Angers
c997f853e5
Make creation of http.Client uniform across the codebase (#3097) 2021-11-24 15:56:54 -05:00
Zach Wasserman
2b2e9f9f44
Prepare for 4.6.1 release (#3067) 2021-11-21 21:23:25 -08:00
Noah Talerman
cca1e2e043
Update "Adding hosts" documentation to emphasize osquery installers (#2956)
- Emphasize generating an "osquery installer" (also referred as "Orbit") using the `fleetctl package` command
- Add instructions for adding multiple hosts and automatically adding hosts to a team
- Remove instructions for generating an enrollment package with the tooling in `tools/mac/`
- Remove the `tools/mac/` directory from the repository because it is no longer used or referenced in any documentation
- Update "Automatically adding hosts to a team" section of "Teams" documentation to point to "Adding hosts" documentation
- Add instructions for migrating from plain osquery to Fleet's osquery installers
2021-11-19 09:38:01 -05:00
Noah Talerman
a6e9d551fd
Prepare for Fleet 4.6.0 (#3018) 2021-11-18 17:46:57 -08:00
Zach Wasserman
1bd50d8965
Prepare for 4.5.1 release (#2913) 2021-11-12 17:03:19 -08:00
Zach Wasserman
e1f273fdc0
Add multiple users for SSO testing (#2885) 2021-11-10 15:15:41 -08:00
Benjamin Edwards
dac043f1f5
refactor terraform to allow bootstrapping (#2662)
* refactor to allow bootstrapping* move monitoring into its own package, update readme
* add variable for license_key
* replication lag alarm less sensitive
2021-11-09 20:14:05 -05:00
Lucas Manuel Rodriguez
660796bf72
Add documentation for running live queries with GET request (#2755) 2021-11-03 14:35:17 -03:00
Noah Talerman
249927f750
Prepare for Fleet 4.5.0 release (#2741) 2021-11-01 11:39:55 -04:00
Zach Wasserman
8dc34be3a3
Updates and fixes for packaging (#2682)
- Fix Windows MSI generation by changing permissions (#2655).
- Refactor temp directory initialization.
- Use root user for Wine in WiX Docker container.
- Support .pkg packaging on Linux without dependencies (besides Docker)
2021-10-27 16:17:41 -07:00