#9609
This PR also fixes#10777.
The issue is: We were using `svc.AppConfig` instead of
`svc.ds.AppConfig` to retrieve the SMTP credentials.
`svc.AppConfig` obfuscates credentials, whereas `svc.ds.AppConfig` does
not.
To help prevent this from happening again I've renamed `svc.AppConfig`
to `svc.AppConfigObfuscated`.
I've also added a new test SMTP server
(https://github.com/axllent/mailpit) that supports Basic Authentication
and tests that make use of it to catch these kind of bugs (the tests are
executed when running `go test` with `MAIL_TEST=1`).
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#8411
PS: I've opened #10209 to solve the issue with Golang Code Coverage CI
checks.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
## Summary
This pull request is created by [Secure
Repo](https://app.stepsecurity.io/securerepo) at the request of @zwass.
Please merge the Pull Request to incorporate the requested changes.
Please tag @zwass on your message if you have any questions related to
the PR. You can also engage with the
[StepSecurity](https://github.com/step-security) team by tagging
@step-security-bot.
## Security Fixes
### Secure Dockerfiles
Pin image tags to digests in Dockerfiles. With the Docker v2 API
release, it became possible to use digests in place of tags when pulling
images or to use them in FROM lines in Dockerfiles.
- [The Open Source Security Foundation (OpenSSF) Security
Guide](https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies)
## Feedback
For bug reports, feature requests, and general feedback; please create
an issue in
[step-security/secure-repo](https://github.com/step-security/secure-repo).
To create such PRs, please visit https://app.stepsecurity.io/securerepo.
Signed-off-by: StepSecurity Bot <bot@stepsecurity.io>
---------
Signed-off-by: StepSecurity Bot <bot@stepsecurity.io>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
#9260
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#9253
- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
---------
Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
Update the default file carver block size to be compatible with MySQL 8
& S3.
Update surrounding docs.
Various other updates to references of MySQL versions (all terraform
deploys are now defaulted MySQL 8 in AWS)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
# Addresses #9365
# Implements
MDM enrollment modal that handles both automatic and manual enrollment
instructions:
- Automatic:
<img width="1181" alt="Screenshot 2023-01-20 at 4 33 50 PM"
src="https://user-images.githubusercontent.com/61553566/213829293-6d4a5053-9a3c-4f52-8cf8-a6607dc8df4e.png">
- Manual:
<img width="1158" alt="Screenshot 2023-01-20 at 4 35 04 PM"
src="https://user-images.githubusercontent.com/61553566/213829369-73ae779d-14a8-4aa7-9c6a-b97d046d0dc1.png">
- Also includes (by mistake, but might as well include them now) some
small bash scripts for use in MDM development
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`
- [x] Updated testing inventory
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
* Fix ingestion of MDM server for macOS and Windows hosts
* Return early if there are no mobile_device_management_solutions to clean up
* Run make dump-test-schema
* Fix TestHosts/DeleteHosts test
* Add test for host unenrolling from MDM
* Fix lint
* Fix typo
* Identify Fleet MDM
* Move logic to deduce MDM name out of Datastore
* Run make dump-test-schema
* Remove printfs from migration
* Update go to 1.19.4
* Comment out failing package test
* Comment out ALL the packaging tests for windows for the moment
* Update go to 1.19.4
* Comment out failing package test
* Comment out ALL the packaging tests for windows for the moment
* Update changelog
* Bump versions
* Update changelog to reflect this being a security release
* Remove email address for redudancy
Call to actions at the bottom of the page offer better forms of communication to join the community, trial Fleet, or ask questions.
* Remove email mention
* Remove email mention
* Remove email mention
Motivation: I often find myself digging through the XML in https://tuf.fleetctl.com to see if/when/what packages have been published in the different release channels, this tool aims to make the process less painful.
```
~/fleet $ go run tools/tuf/status/status.go --help
This is a CLI utility to fetch and filter the entries posted by a TUF repository.
-key-filter string
filter keys using a regular expression (default "stable")
-url string
URL of the TUF repository (default "https://tuf.fleetctl.com")
```
### Examples
- To filter all items on the edge channel use `--key-filter="edge"`
- To filter all items on version `1.3` including patches that run on Linux use `--key-filter="linux/1.3.*"`
- To filter Fleet Desktop items on `1.3.*`, `stable` and `edge` that run on macOS use `--key-filter="desktop/*.*/macos/(1.3.*|stable|edge)"`
### Example output
```
~/fleet $ go run tools/tuf/status/status.go --key-filter="desktop/*.*/macos/(1.3.*|stable|edge)"
Results filtered by "desktop/*.*/macos/(1.3.*|stable|edge)" and sorted by version, platform and key.
VERSION PLATFORM KEY LAST MODIFIED SIZE ETAG
1.3 macos targets/desktop/macos/1.3/desktop.app.tar.gz 2022-10-20T00:42:58.000Z 28.4 MB "ceb7a975880db6ab9aea7907915406e3-4"
1.3.0 macos targets/desktop/macos/1.3.0/desktop.app.tar.gz 2022-10-20T00:42:58.000Z 28.4 MB "ceb7a975880db6ab9aea7907915406e3-4"
1.3.1 macos targets/desktop/macos/1.3.1/desktop.app.tar.gz 2022-10-21T20:28:24.000Z 28.5 MB "3c6641a0df00009f3f076bd6b4fbc748-4"
edge macos targets/desktop/macos/edge/desktop.app.tar.gz 2022-10-21T20:28:37.000Z 28.5 MB "3c6641a0df00009f3f076bd6b4fbc748-4"
stable macos targets/desktop/macos/stable/desktop.app.tar.gz 2022-10-20T00:43:06.000Z 28.4 MB "ceb7a975880db6ab9aea7907915406e3-4"
```
* WIP
* Adding DEP functionality to Fleet
* Better organize additional MDM code
* Add cmdr.py and amend API paths
* Fix lint
* Add demo file
* Fix demo.md
* go mod tidy
* Add munki setup to Fleet
* Add diagram to demo.md
* Add fixes
* Update TODOs and demo.md
* Fix cmdr.py and add TODO
* Add endpoints to demo.md
* Add more Munki PoC/demo stuff
* WIP
* Remove proposals from PoC
* Replace prepare commands with fleetctl commands
* Update demo.md with current state
* Remove config field
* Amend demo
* Remove Munki setup from MVP-Dogfood
* Update demo.md
* Add apple mdm commands (#7769)
* fleetctl enqueue mdm command
* fix deps
* Fix build
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
* Add command to upload installers
* go mod tidy
* fix subcommands help
There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.
* Add support for installing apps
* Add a way to list enrolled devices
* Add dep listing
* Rearrange endpoints
* Move DEP routine to schedule
* Define paths globally
* Add a way to list enrollments and installers
* Parse device-ids as comma-separated string
* Remove unused types
* Add simple commands and nest under enqueue-command
* Fix simple commands
* Add help to enqueue-command
* merge apple_mdm database
* Fix commands
* update nanomdm
* Split nanomdm and nanodep schemas
* Set 512 MB in memory for upload
* Remove empty file
* Amend profile
* Add sample commands
* Add delete installers and fix bug in DEP profile assigning
* Add dogfood.md deployment guide
* Update schema.sql
* Dump schema with MySQL 5
* Set default value for authenticate_at
* add tokens to enrollment profiles
When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.
- Rename enrollments to enrollment profiles. Enrollments is used by nano
to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=
* remove mdm apple server url
* update docs
* make dump-test-schema
* Update nanomdm with missing prefix table
* Add docs and simplify changes
* Add changes file
* Add method docs
* Fix compile and revert prepare.go changes
* Revert migration status check change
* Amend comments
* Add more docs
* Clarify storage of installers
* Remove TODO
* Remove unused
* update dogfood.md
* remove cmdr.py
* Add authorization tests
* Add TODO comment
* use kitlog for nano logging
* Add yaml tags
* Remove unused flag
* Remove changes file
* Only run DEP routine if MDM is enabled
* Add docs to all new exported types
* Add docs
* more nano logging changes
* Fix unintentional removal
* more nano logging changes
* Fix compile test
* Use string for configs and fix config test
* Add docs and amend changes
* revert changes to basicAuthHandler
* remove exported BasicAuthHandler
* rename rego authz type
* Add more information to dep list
* add db tag
* update deps
* Fix schema
* Remove unimplemented
Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
Configuration and fixes for the Fleet server and frontend to add support
for https://github.com/Uptycs/kubequery.
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
* Bump go to 1.19.1
* Bump remaining go-version to the 1.19.1
* Add extra paths for test-go
* Oops, putting the right path in the right place
* gofmt file
* gofmt ALL THE THINGS
* Moar changes
* Actually, go.mod doesn't like minor versions
* Updated testing.md
Updated the name of testing.md to testing-and-local-development.md based off https://github.com/fleetdm/fleet/issues/4706
* add redirect for /docs/contributing/testing
Co-authored-by: Eric <eashaw@sailsjs.com>
This improves the installerstore CLI tool with:
- The ability to create tests buckets for local development (otherwise you have to interact with another CLI or the MinIO UI)
- Improved error handling and messaging.
Related to #6365 this adds a new tool to upload pre-built Orbit installers to a storage blob. It uses the same file conventions that the Fleet server expects, making it useful for local testing and infra envs alike.\
Usage and details in the README
Related to #6364 and #6363, this:
- Adds a new Docker image, `fleetdm/fleetctl` equipped with all necessary dependencies to build Fleet-osquery binaries for all platforms
- Modifies the package generation logic to special case this scenario via an environment variable `FLEETCTL_NATIVE_TOOLING`
- Adds a new GitHub workflow to test this
There are more details in the README, but part of the special-casing logic is in place to output the binaries to a folder named `build` when they are run with `FLEETCTL_NATIVE_TOOLING`, this is so we can persist the binary generated by the docker container via a bind mount:
```bash
docker run -v "$(pwd):/build" fleetdm/fleetctl package --type=msi
```
To test this changeset, I have generated packages for all platforms, both via the new Docker image and via the classic `fleetctl package`.
* fix old root dir in orbit
* add changes
* Add automation for orbit shell (with TUF)
* Fix workflow syntax
* Add logging to latest fleetctl preview action
* Add changes to fix workflow
* Use macOS host for TUF server and package generation
* Remove copy/paste if clause
* Fix orbit logs on macOS, Ubuntu
* Simplify TUF and generation of packages
* Set enroll secret instead of getting it
* Increase timeouts
* Add step id
* Fixes to the upload/download of artifacts
* Rearrange steps to not lose the downloads
* Fix copy/paste
* Add fleetctl login step
* Add missing config set
* Fix quotes on Windows
* Increase timeout
* Fix job termination
* Disable FLEET_DESKTOP for now
* Checkout repository on macOS
* Fix logs path
* Enable fleet desktop
* Use cancel, nitpick
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
* Prepare TUF scripts for CI and support different dev setups
* Allow only generating selected platforms for pkgs
* Force linux/amd64 on Dockerfile to support M1
