#9609
This PR also fixes#10777.
The issue is: We were using `svc.AppConfig` instead of
`svc.ds.AppConfig` to retrieve the SMTP credentials.
`svc.AppConfig` obfuscates credentials, whereas `svc.ds.AppConfig` does
not.
To help prevent this from happening again I've renamed `svc.AppConfig`
to `svc.AppConfigObfuscated`.
I've also added a new test SMTP server
(https://github.com/axllent/mailpit) that supports Basic Authentication
and tests that make use of it to catch these kind of bugs (the tests are
executed when running `go test` with `MAIL_TEST=1`).
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
* WIP
* Adding DEP functionality to Fleet
* Better organize additional MDM code
* Add cmdr.py and amend API paths
* Fix lint
* Add demo file
* Fix demo.md
* go mod tidy
* Add munki setup to Fleet
* Add diagram to demo.md
* Add fixes
* Update TODOs and demo.md
* Fix cmdr.py and add TODO
* Add endpoints to demo.md
* Add more Munki PoC/demo stuff
* WIP
* Remove proposals from PoC
* Replace prepare commands with fleetctl commands
* Update demo.md with current state
* Remove config field
* Amend demo
* Remove Munki setup from MVP-Dogfood
* Update demo.md
* Add apple mdm commands (#7769)
* fleetctl enqueue mdm command
* fix deps
* Fix build
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
* Add command to upload installers
* go mod tidy
* fix subcommands help
There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.
* Add support for installing apps
* Add a way to list enrolled devices
* Add dep listing
* Rearrange endpoints
* Move DEP routine to schedule
* Define paths globally
* Add a way to list enrollments and installers
* Parse device-ids as comma-separated string
* Remove unused types
* Add simple commands and nest under enqueue-command
* Fix simple commands
* Add help to enqueue-command
* merge apple_mdm database
* Fix commands
* update nanomdm
* Split nanomdm and nanodep schemas
* Set 512 MB in memory for upload
* Remove empty file
* Amend profile
* Add sample commands
* Add delete installers and fix bug in DEP profile assigning
* Add dogfood.md deployment guide
* Update schema.sql
* Dump schema with MySQL 5
* Set default value for authenticate_at
* add tokens to enrollment profiles
When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.
- Rename enrollments to enrollment profiles. Enrollments is used by nano
to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=
* remove mdm apple server url
* update docs
* make dump-test-schema
* Update nanomdm with missing prefix table
* Add docs and simplify changes
* Add changes file
* Add method docs
* Fix compile and revert prepare.go changes
* Revert migration status check change
* Amend comments
* Add more docs
* Clarify storage of installers
* Remove TODO
* Remove unused
* update dogfood.md
* remove cmdr.py
* Add authorization tests
* Add TODO comment
* use kitlog for nano logging
* Add yaml tags
* Remove unused flag
* Remove changes file
* Only run DEP routine if MDM is enabled
* Add docs to all new exported types
* Add docs
* more nano logging changes
* Fix unintentional removal
* more nano logging changes
* Fix compile test
* Use string for configs and fix config test
* Add docs and amend changes
* revert changes to basicAuthHandler
* remove exported BasicAuthHandler
* rename rego authz type
* Add more information to dep list
* add db tag
* update deps
* Fix schema
* Remove unimplemented
Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
* Make migrations compatible with GTID replication
Fixes an issue some deployments encountered when migrations used a
statement that is unsupported in GTID replication mode (#2462).
Local dev MySQL now enforces this consistency, so it should be easier to
maintain compatibility going forward.
* Update docker-compose formatting
* if exists
* Migrate all mysql tests to the new form
* Only dump sql if MYSQL_TEST is on
* Removing parallel until we get rid of this code
* Move TestMain to an actual _test file
* A little experiment with tmpfs to speed up the db
* Let's make sure the dump.sql file is also in ram
- Update names/roles of users in `make e2e-setup`.
- Update test SSO user info.
- Add Cypress commands for seeding users/Teams.
- Stub Cypress tests for team/tier matrix.
Instead of synchronously updating the seen_time column for a host on an update, batch these updates to be written together every 1 second.
This results in a ~33% reduction in MySQL CPU usage in a local test with 4,000 simulated hosts and MySQL running in Docker.
Improves MySQL test time (on my 2020 MBP) to ~18s from ~125s.
- Use separate databases for each test to allow parallelization.
- Run migrations only once at beginning of tests and then reload
generated schema.
- Add `--innodb-file-per-table=OFF` for ~20% additional speedup.
- Add --dev flag that will set default flag values. This simplifies the
invocation of Fleet in a development environment.
- Change defaults in docker-compose to use `fleet` in place of `kolide`.
- Skip prompt in `prepare db` when `--dev` specified.
- Update developer documentation.
Updates to MySQL configuration in docker-compose.yml may require
existing development containers and volumes to be deleted (this will
delete data in MySQL):
```shell
docker-compose rm -sf
docker volume rm fleet_mysql-persistent-volume
```
Closes#170
* Use YAML anchors to avoid repeating config blocks
* Use docker volumes to persist data for mysql
* Allow setting `FLEET_SERVER` (fixes#2127) when using the docker-compose file to spin up multiple osquery clients
A new datastore interface is needed for buffering incoming distributed query results to be sent to the client. This PR attempts to define and implement that interface.
It is intended that the ReadChannel() method be used by the goroutine that will push query results down a websocket to the client. Passing the results through this channel will allow that goroutine to perform a select on both the channel and the websocket, in order to properly handle IO.
* No more hard deletes
* scaffolding for password reset endpoint
* Ensure password reset state is accounted for in VC checks
* password reset endpoints and data structures
* ability to change password with reset token
* smtp server connection pool management
* stubbing out the sending of the email
* adding mailhog via docker
* HTML emails with confgurable host name
* fixing typo in the comments
* Fixing merge which undid DatabaseError replacement
* documentation in the readme
* webpack shortcut for components
* removing a sneaky merge line that snuck in
* temporary email content api
* tests for password reset flow
* fixing go vet
* comments and making all db use `&value` rather than `reference`
* more correct usage of the errors library and moving email sending to it's own method
* using the wrong error
* fixing email mock object error
* less incorrect error usage
* rebasing and merging
* http constants for status code
* using ParseAndValidateJSON instead of BindJSON
* validate instead of binding in struct tags
* NewFromError instead of New
This commit vendors in all of our dependencies using
[GoDep](https://github.com/tools/godep). We are forgoing using a vendor/ folder to avoid checking in deps into the repo.
Note: Never manually modify `Godeps/Godeps.json` this file is dynamically
by the godep CLI
Common Actions:
To add a new package foo/bar, do this:
1. Run `go get foo/bar`
1. Edit your code to import foo/bar.
1. Run `godep save` (or `godep save ./...`).
To update a package from your `$GOPATH`, do this:
1. Run `go get -u foo/bar`
1. Run `godep update foo/bar`. (You can use the `...` wildcard, for example
`godep update foo/...`).
This commit adds both a Dockerfile and updates the docker-compose.yml with local mounting so that you can standup a consistent dev environment. Please view the project README for more information.
