## Addresses #11272
- For requests with invalid list options (`page`, `per_page`,
`order_key`, `order_direction`), return `400` instead of `500`
<img width="957" alt="Screenshot 2023-05-10 at 2 28 56 PM"
src="https://github.com/fleetdm/fleet/assets/61553566/d4400a92-b158-4a41-9d00-9ba5170d48f6">
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#11531
How to use:
```
go run ./tools/mdm/apple/loadtest -api_token $API_TOKEN -fleet_url https://fleet.example.com -team_count 5 -team_extra_count 2 -loop_count 1
2023-05-15T16:05:06Z: 1. Creating 5 teams... (press enter to proceed)
2023-05-15T16:05:18Z: 1. Duration: 165.020742ms
2023-05-15T16:05:18Z: 2. Transfering one host to each team... (press enter to proceed)
2023-05-15T16:05:20Z: 2. Duration: 370.515741ms
2023-05-15T16:05:20Z: 3a. Add 10 profiles to all teams... (press enter to proceed)
2023-05-15T16:05:22Z: Applying profiles to team Team 0...
2023-05-15T16:05:22Z: Applying profiles to team Team 1...
2023-05-15T16:05:22Z: Applying profiles to team Team 2...
2023-05-15T16:05:22Z: Applying profiles to team Team 3...
2023-05-15T16:05:22Z: Applying profiles to team Team 4...
2023-05-15T16:05:22Z: 3a. Duration: 502.557741ms
2023-05-15T16:05:22Z: 3b. Waiting for all profiles to be applied on all teams...
2023-05-15T16:05:22Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:27Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:32Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:37Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:42Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:1 Pending:0 Failed:0} 43:{Verifying:1 Pending:0 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:48Z: 3b. Duration: 25.487993296s
2023-05-15T16:05:48Z: 4a. Modify a profile on all teams... (press enter to proceed)
2023-05-15T16:05:50Z: 4a. Duration: 654.712391ms
2023-05-15T16:05:50Z: 4b. Waiting for all profiles to be applied on all hosts of all teams...
2023-05-15T16:05:50Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:55Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:06:00Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:06:05Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:06:10Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:1 Pending:0 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:06:15Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:1 Pending:0 Failed:0} 43:{Verifying:1 Pending:0 Failed:0} 44:{Verifying:1 Pending:0 Failed:0}
45:{Verifying:1 Pending:0 Failed:0}]
2023-05-15T16:06:20Z: 4b. Duration: 30.48166655s
2023-05-15T16:06:20Z: 5. Creating extra 2 teams... (press enter to proceed)
2023-05-15T16:06:22Z: 5. Duration: 72.531251ms
2023-05-15T16:06:22Z: 6a. Moving one host to each new extra 2 teams... (press enter to proceed)
2023-05-15T16:07:23Z: 6a. Duration: 164.446788ms
2023-05-15T16:07:23Z: 6b. Waiting for all profiles to be applied on all hosts of the extra teams...
2023-05-15T16:07:23Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:29Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:34Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:39Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:44Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:1 Pending:0 Failed:0}]
2023-05-15T16:07:49Z: 6b. Duration: 25.210811603s
2023-05-15T16:07:49Z: 7a. Add 10 profiles to all extra 2 teams... (press enter to proceed)
2023-05-15T16:07:50Z: 7a. Duration: 175.694423ms
2023-05-15T16:07:50Z: 7b. Waiting for all profiles to be applied on all hosts of the extra teams...
2023-05-15T16:07:50Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:55Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:08:01Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:08:06Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:08:11Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:1 Pending:0 Failed:0}]
2023-05-15T16:08:16Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:1 Pending:0 Failed:0}]
2023-05-15T16:08:21Z: 7b. Duration: 30.219261468s
2023-05-15T16:08:21Z: 8. Destroy 2 extra teams... (press enter to proceed)
2023-05-15T16:08:31Z: 8. Duration: 134.372194ms
```
- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.~
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#11528
osquery-perf simulated hosts enroll and are identified as manually
enrolled. (Enrolling as DEP requires more work, e.g. a new mocked Apple
DEP endpoint).
Given that these are simulated MDM clients, they cannot be woken up with
push notifications. Instead, these check for new commands to execute
every 10 seconds (which is not realistic, but could serve as a good
loadtesting exercise).
I will now start setting up the loadtest environment with MDM enabled
and configured to test this.
- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#11447
- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#7970
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
#11089
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
Related to #10741, this adds a new key to app config named
`end_user_authentication`, which can be configured using the same keys
as the existing SSO feature.
Per the spec, if the feature is configured, it's implicitly enabled, at
least until we get to #10999.
Note that this only enables the SSO config, a second part of the ticket
with endpoints for the EULA will be tackled separately.
This test was failing because we can send other commands when a DEP
device is enrolling (eg: to install a bootstrap package) and the test
was assuming that the first command we'll always get is the command to
install `fleetd`.
We have discussed in the past (@gillespi314 suggested that is important)
increasing the accuracy of the timestamps in the `nano_commadns` table,
which is the proper fix for this.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
In #10338 we introduced logic to gate DEP profiles behind Okta auth
using the ROP flow.
We're not going to use that, and instead we're going to gate profiles
behind SSO, which can be used from multiple providers and supports SSO
(the initial motivation behind the ROP flow was to create a local user
account.)
This removes some of the old code, which was never used in
production/documented for the public to use.
At the moment I'm leaving the `mdm_idp_accounts` table and related
methods untouched, as it's unclear yet if we're going to need a similar
auxiliar table, and I would rather deal with the migrations all at once.
#10880
I was not able to reproduce other 500s in `/api/_version_/fleet/hosts`
other than the one fixed in the PR.
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
A question in form of PR:
Do we really need the following two entities in our
[policy.rego](https://github.com/fleetdm/fleet/blob/main/server/authz/policy.rego)
`1. (object=mdm_apple_command, action=read/write)` and `2. (object=host,
action=mdm_command)`? (Maybe mdm_command is a leftover action from the
PoC?)
Guess: `mdm_apple_command` (`fleet.MDMAppleCommandAuthz`) is what we
want: `action=write` means you can enqueue, `action=read` means you can
list commands and read their results.
PS: Found this while trying to add command execution permissions to the
new `GitOps` role.
