Commit Graph

200 Commits

Author SHA1 Message Date
Lucas Manuel Rodriguez
f4d3159cc9
Fleetctl to package .app bundles for osquery (and changes for orbit to support them) (#4393)
* WIP

* WIP2

* Fix orbit and fleetctl tests

* Amend macos-app default

* Add some fixes

* Use fleetctl updates roots command

* Add more fixes to Updater

* Fixes to app publishing and downloading

* Add more changes to support fleetctl cross generation

* Amend comment

* Add pkg generation to ease testing

* Make more fixes

* Add changes entry

* Add legacy targets (until our TUF system exposes the new app)

* Fix fleetctl preview

* Fix bool flag

* Fix orbit logic for disabled-updates and dev-mode

* Fix TestPreview

* Remove constant and fix zip-slip attack (codeql)

* Return unknown error

* Fix updater's checkExec

* Add support for executable signing in init_tuf.sh

* Try only signing orbit

* Fix init_tuf.sh targets, macos-app only for osqueryd

* Specify GOARCH to support M1s

* Add workflow to generate osqueryd.app.tar.gz

* Use 5.2.2 on init_tuf.sh

* Add unit test for tar.gz target

* Use artifacts instead of releases

* Remove copy paste residue

* Trigger workflow on PR

* Fixes to ease handling of artifact

* Fix, do not use target name as dir

* Remove workaround
2022-03-15 16:04:12 -03:00
Noah Talerman
937cd7f2eb
Update changelog and bump versioning (#4497) 2022-03-07 11:08:51 -06:00
eashaw
0af5e161d0
Remove numbers from documentation filenames in Fleet repo (#4313)
* Renaming files and a lot of find and replace

* pageRank meta tags, sorting by page rank

* reranking

* removing numbers

* revert changing links that are locked to a commit

* update metatag name, uncomment github contributers

* Update basic-documentation.page.js

* revert link change

* more explicit errors, change pageOrderInSection numbers, updated sort

* Update build-static-content.js

* update comment

* update handbook link

* handbook entry

* update sort

* update changelog doc links to use fleetdm.com

* move standard query library back to old location, update links/references to location

* revert unintentional link changes

* Update handbook/community.md

Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>

Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
2022-02-23 12:17:55 -06:00
Lucas Manuel Rodriguez
b5be858071
Fix update checks for orbit at startup (#3835)
* Fix update checks for orbit at startup

* Add tests

* Add scripts for testing local TUF server

* Remove -x used for debugging
2022-02-23 14:58:07 -03:00
Lucas Manuel Rodriguez
e39ba93a90
Orbit to wait for osqueryd extension manager socket to be ready (#3836)
* Wait for osquery extension manager socket to be ready

* PR review feedback

* Add WIP Github Action to test orbit

* Set fleet address on fleetctl

* Add logging for troubleshooting

* Add prepare db statement

* Use tls-skip-verify on fleetctl

* Move steps around

* Fix addresses

* Fix fleetctl path

* Add certs.pem to orbit root dir

* Add orbit logs

* Increase timeout

* Add proper orbit log paths

* Fix tmp path orbit

* Add get hosts command to troubleshoot

* Fix orbit job termination

* Add comments to workflows

* Wait for server to go down

* Add orbit Windows job

* Use bash on windws-latest

* Fix missing quote

* Run orbit on Ubuntu

* Bump host count

* Increase timeout for extensions and not terminate on err

* Add comment to clarify high timeout value

* Revert change, we do want to exit in case of err
2022-02-22 15:05:32 -03:00
RachelElysia
7cdc7bbec2
E2e Test: Team admin flow (#4301) 2022-02-21 21:09:36 -05:00
Benjamin Edwards
12eac152c4
GCP Terraform (#4303)
* gcp wip
* more edits on services, secrets manager, readme
* updated readme with required variables
2022-02-18 20:01:42 -05:00
Zachary Winnerman
899a643c15
Elastic is now managed through ansible for quicker config changes and more stability (#4243)
* Add elasticsearch being managed through ansible for better managability

* Testing

* testing

* testing

* testing

* testing

* testing

* testing

* fixup

* fixup

* Added docker stuff

* fixup

* fixup

* fixup

* Remove old code

* fixup

* fix health checks

* Add elastic apm configuration

* fixup

* fixup

* fixup

* testing

* restart always

* fixup

* fixup
2022-02-18 11:07:32 -05:00
Zachary Winnerman
b72e9bd1c1
Split elasticsearch's security groups (#4231) 2022-02-16 11:11:39 -05:00
Zachary Winnerman
69c0434e88
Add terraform for the loadtesting environment (#4172)
* Add terraform for the loadtesting environment

* Add some checkov stuff and minor updates

* Remove defunct commented stuff

* Add separate cloudwatch namespace for migrations

* Remove defunct code

* checkin
2022-02-15 13:00:24 -05:00
dependabot[bot]
8589cdd553
Bump follow-redirects from 1.14.4 to 1.14.8 in /tools/fleetctl-npm (#4183)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.4 to 1.14.8.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.14.4...v1.14.8)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-15 10:08:07 -06:00
Noah Talerman
67827474c2
Prepare for Fleet 4.10.0 (#4161)
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-02-13 18:13:06 -08:00
eashaw
9af92b23aa
Update links to documentation (#4163)
* update links to documentation

* revert .sailsrc change

* fix broken link
2022-02-10 19:26:18 -06:00
Zach Wasserman
600549d8a4
Prepare for 4.9.1 release (#3996) (#4005)
Merging back to `main` from patch branch.
2022-02-10 10:46:21 -05:00
Luke Heath
5866d68f13
Set max length for name inputs (#4063) 2022-02-08 19:40:38 -06:00
Benjamin Edwards
d4312bec59
remove broken terraform references (#3856) 2022-01-24 15:58:36 -05:00
Tomas Touceda
35f806f7d1
Merge Release candidate 4.9.0 into main (#3844)
* Better jitter (#3716) (#3744)

* Better jitter (#3716)

* Better jitter

* Fix lint

* Use milliseconds

* Make duration milliseconds

* Update based on Lucas' suggestion

* Add changes file

* Panic on error

* Fix compilation error

* Use time.after instead of time.tick to not leak

* Include browser extensions in software inventory (#3733) (#3757)

Use appropriate JOINs against users table to include all results.

For #3557

* Build Docker images on push to any branch (#3756) (#3758)

Allows for testing release branches and any others.

* Optimize users detail query (#3754) (#3770)

@Smjert reported instances of Windows Domain Controllers having massive
resource utilization and being killed by the watchdog when running this
query. In his test environment, this new query performs much better.

* Refactor async host processing to avoid redis SCAN keys (for policies) (#3657) (#3755)

* Issue 3707 clean targets on delete (#3739) (#3776)

* wip

* Delete targets when deleting teams, hosts, and labels

* Add changes file

* Fix error message

* Remove unused teamsTable

* Cleanup new pack

* Clean new packs at end of test

* Update operating system policy (#3779)

- Update policy's query to check for operating system versions greater than or equal to

* Improve loading of manage policies page (#3695) (#3781)

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

* Fix logo (#3765)

* fixed duplicate query modal not closing (#3787)

* fixed style for connection line on activity feed (#3789)

* Improve loading state for query platform compatibility (#3752) (#3783)

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

* Disable pagination for teams page (#3794)

* Shorten "simple" query API period to 25s (#3775) (#3804)

This helps the period stay under the default request timeouts for most
load balancers.

Some default timeouts:
* AWS ALB - 60s
* Nginx - 60s
* GCP LB - 30s

* Clear user checkbox selections after successfully removing selected row from table UI #3798 (#3801)

* Alias fleetctl sandbox to fleetctl preview (#3803) (#3806)

Preparing for some work the product team is planning to rebrand preview
to sandbox.

* Remove flash of spinner to no spinner (#3799)

* Revert "Better jitter (#3716) (#3744)" (#3807)

This reverts commit c857b52d2b373ed9e302d643e6d2212af7353408.

* Fix teams dropdown sizing (#3759) (#3810)

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

* fixed link for expiration message (#3788)

* Fixed empty string for compatible platforms (#3812)

* fixed empty string for compatible platforms

* default checked for platforms on policy query

* Block autocomplete for username and password in SMTP settings (#3732) (#3816)

* No hosts for software/policy renders better message (#3701) (#3821)

Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>

* Get failing policies webhook data on page load (#3824)

* Fix reset password flow (#3826)

* Prepare for 4.9.0 (#3820)

- Add CHANGELOG
- Bump versioning
- Remove changes files
- Spelling fix for operating system policy's description

* Remove merge artifact

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>
2022-01-24 15:32:25 -03:00
Benjamin Edwards
d650423be0
Reference Architectures (#3712)
Added reference architectures using https://docs.gitlab.com/ee/administration/reference_architectures/ as inspiration.

- updated terraform based on some feedback of usage
- pinned fleet docker version in terraform as to no get unexpected upgrades when applying
- updated some documentation around apply migration tasks
2022-01-21 16:27:55 -08:00
Tomas Touceda
43b78e6d0d
Add redis stress (#3363)
* Add redis stress tool

* Parametrize more things

* Rename folder back

* Log error

* Fix lint
2022-01-20 16:18:17 -03:00
Benjamin Edwards
6b551acf95
add percona terraform (#3468) 2022-01-06 18:07:40 -05:00
Noah Talerman
09654d77ee
Prepare for Fleet 4.8.0 (#3536) 2021-12-31 09:58:08 -08:00
Noah Talerman
8957f00d86
Prepare for Fleet 4.7.0 (#3355) 2021-12-14 11:25:13 -08:00
Zach Wasserman
a6d60cc193
Configure nofiles ulimit in Terraform ECS task (#3249)
The low default ulimit `nofiles` value (`4096`) in Fargate was observed
to cause errors when running with a large number of hosts and a small
number of servers. Each server should be able to server more than 4096
simultaneous clients.
2021-12-08 15:08:48 -08:00
Tomas Touceda
142006cbdd
Issue 2099 enable vulnerability processing new installs (#3104)
* Enable vulnerability processing by default

* Add changes file

* Remove wrong env var

* Fix import

* Fix tests

* Update log line

* Remove unneeded import
2021-12-03 14:44:57 -03:00
Tomas Touceda
052e41851b
Prepare for Fleet 4.6.2 (#3146) (#3179)
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
2021-12-02 16:56:56 -03:00
Martin Angers
c997f853e5
Make creation of http.Client uniform across the codebase (#3097) 2021-11-24 15:56:54 -05:00
Zach Wasserman
2b2e9f9f44
Prepare for 4.6.1 release (#3067) 2021-11-21 21:23:25 -08:00
Noah Talerman
cca1e2e043
Update "Adding hosts" documentation to emphasize osquery installers (#2956)
- Emphasize generating an "osquery installer" (also referred as "Orbit") using the `fleetctl package` command
- Add instructions for adding multiple hosts and automatically adding hosts to a team
- Remove instructions for generating an enrollment package with the tooling in `tools/mac/`
- Remove the `tools/mac/` directory from the repository because it is no longer used or referenced in any documentation
- Update "Automatically adding hosts to a team" section of "Teams" documentation to point to "Adding hosts" documentation
- Add instructions for migrating from plain osquery to Fleet's osquery installers
2021-11-19 09:38:01 -05:00
Noah Talerman
a6e9d551fd
Prepare for Fleet 4.6.0 (#3018) 2021-11-18 17:46:57 -08:00
Zach Wasserman
1bd50d8965
Prepare for 4.5.1 release (#2913) 2021-11-12 17:03:19 -08:00
Zach Wasserman
e1f273fdc0
Add multiple users for SSO testing (#2885) 2021-11-10 15:15:41 -08:00
Benjamin Edwards
dac043f1f5
refactor terraform to allow bootstrapping (#2662)
* refactor to allow bootstrapping* move monitoring into its own package, update readme
* add variable for license_key
* replication lag alarm less sensitive
2021-11-09 20:14:05 -05:00
Lucas Manuel Rodriguez
660796bf72
Add documentation for running live queries with GET request (#2755) 2021-11-03 14:35:17 -03:00
Noah Talerman
249927f750
Prepare for Fleet 4.5.0 release (#2741) 2021-11-01 11:39:55 -04:00
Zach Wasserman
8dc34be3a3
Updates and fixes for packaging (#2682)
- Fix Windows MSI generation by changing permissions (#2655).
- Refactor temp directory initialization.
- Use root user for Wine in WiX Docker container.
- Support .pkg packaging on Linux without dependencies (besides Docker)
2021-10-27 16:17:41 -07:00
Zach Wasserman
32bee9f975
Update fleetctl NPM dependencies (#2658)
These dependencies had vulnerabilities that caused alerting on `npm
install` and in GitHub.

The vulnerabilities did not effect the use in our context.
2021-10-24 19:24:44 -07:00
Benjamin Edwards
0fb6416d45
Add Cloudwatch monitoring to AWS Terraform configs (#2485)
* add support for minio backend file carving
* add changes file
* rds alarm and sns topic
* added cloudwatch alarm documenation
* Update docs/01-Using-Fleet/06-Monitoring-Fleet.md
* update aws provider version to fix bug in ecs container insights, add more redis alerts
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-10-22 15:38:00 -04:00
Zach Wasserman
03b81245ee
Update files for 4.4.3 release (#2636) 2021-10-22 09:41:50 -07:00
Lucas Manuel Rodriguez
91ae5ba870
By default, force password reset on CreateUser (#2570)
* By default, force password reset on CreateUser

* Use ptr.Bool and add test

* Add changes file

* Simple fix for e2e tests

* Add API changes
2021-10-21 15:00:38 -03:00
Zach Wasserman
42c7933b22
Add WiX Dockerfile and update image name (#2548)
Use a different base image and newer version of Wine to try to mitigate
crashes experienced by users in #2527.
2021-10-20 15:30:46 -07:00
Martin Angers
07cf136d03
Add TLS support to Redis connections (#2568) 2021-10-20 10:09:18 -04:00
Martin Angers
eda6d60be8
Add replicas to the redis docker-compose setup. (#2511) 2021-10-18 08:33:26 -04:00
Zach Wasserman
a6e8f22d83
Update changelog and configs for patch releases (#2531) 2021-10-14 18:17:47 -07:00
Tomas Touceda
d904d501ac
Add infra for loadtest (#2218)
* Add infra for loadtest

* Move loadtest stuff to a new file and parametrize fleet min/max capacity

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* Update to be ready for review

* Update link and other variables needed

* Address review comments and update links
2021-10-14 12:04:27 -03:00
Benjamin Edwards
16c5823692
add support for minio backend file carving (#2448)
* add support for minio backend file carving
* add changes file
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-10-12 15:32:06 -04:00
Luke Heath
94a10cb6b9
Update README.md 2021-10-12 10:02:31 -05:00
RachelElysia
981028705e
Sentence Case: Throughout UI, tests, validators, docs (#2455) 2021-10-11 10:58:27 -04:00
Zach Wasserman
e61dc765fd
Prepare for 4.4.0 release (#2423) 2021-10-06 14:14:12 -07:00
Zach Wasserman
980a853e3c
Update changelog for 4.3.2 release (#2276) 2021-09-30 15:23:22 -07:00
Benjamin Edwards
1cc68eea3f
Feature/infra updates (#2183)
* complete terraform state migration

* split firehose results & status streams

* extract more variables, with sane defaults

* fix fargate configs
2021-09-30 16:22:34 -04:00