* Fix ingestion of MDM server for macOS and Windows hosts
* Return early if there are no mobile_device_management_solutions to clean up
* Run make dump-test-schema
* Fix TestHosts/DeleteHosts test
* Add test for host unenrolling from MDM
* Fix lint
* Fix typo
* Identify Fleet MDM
* Move logic to deduce MDM name out of Datastore
* Run make dump-test-schema
* Remove printfs from migration
* WIP
* Add more logging
* Check rate limit at end of action
* Add github client in more places
* Add new published firefox 93 vulnerabilities to tests
* Remove fmt printfs
* Restore CI check settings
* Readd newline
Related to #8167, from RFC5280:
> The serial number MUST be a positive integer assigned by the CA to
> each certificate. It MUST be unique for each certificate issued by a
> given CA (i.e., the issuer name and serial number identify a unique
> certificate).
Without this change, we're issuing two different certificates with the
same serial number (1, one) and issuer name:
1. One during `fleetctl generate mdm-apple`
1. One during the first SCEP enrollment
* Make activities type and auto-generate docs for each type
* Add pageOrderInSection to not break site
* Add do not edit note to generated file
* Add make generate-doc step
* Fix main merge
* Improve how we are logging failed queries and windows detail queries
* Update failing tests
* Remove unneeded table in this particular select
* Update tests now that we dont rely on release_id
* Update ingestion since we want to ingest failures for policies and live queries
* We also want labels with failures
* Fix test since we are logging errors slightly differently
* Improve logging and error handling
* Revert back test change
* Actually, we need to OR the booleans to maintain logic
This needed to be set specifically for the live query websocket dialer,
since that doesn't use the HTTP client that the rest of the API client
methods use.
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
* add premium feature critical policies
* update documentation
* add test for premium-only field
* update old change-file
* test policies more comprehensively
* also test team policies
* PATCH returns wrong timestamp, updating test
rel: https://github.com/fleetdm/fleet/issues/8768 thanks to @gillespi314 for the idea, the test consistently fails with:
```
=== RUN TestHosts/TestHostOrder
hosts_test.go:5392:
Error Trace: /home/runner/work/fleet/fleet/server/datastore/mysql/hosts_test.go:5392
/home/runner/work/fleet/fleet/server/datastore/mysql/hosts_test.go:5416
/home/runner/work/fleet/fleet/server/datastore/mysql/hosts_test.go:139
Error: "[]" should have 3 item(s), but has 0
Test: TestHosts/TestHostOrder
```
and the relevant code around line `5416`:
```go
_, err = ds.writer.Exec(`UPDATE hosts SET created_at = created_at + id`)
require.NoError(t, err)
hosts, err = ds.ListHosts(ctx, fleet.TeamFilter{User: test.UserAdmin}, fleet.HostListOptions{
ListOptions: fleet.ListOptions{
OrderKey: "created_at",
After: "2010-10-22T20:22:03Z",
OrderDirection: fleet.OrderAscending,
},
})
require.NoError(t, err)
chk(hosts, "0001", "0004", "0003")
```
This PR changes it to be `UPDATE hosts SET created_at = DATE_ADD(created_at, INTERVAL id DAY)` instead, which seems to fix the issue (so far 3 runs without issues)
This test was failing in Dec 2022, because in this line:
```go
_, err := sut.GetFeed((now.AddDate(0, 1, 0)).Month(), now.Year())
```
`(now.AddDate(0, 1, 0)).Month()` returns `"January"` , and `now.Year()` returns `2022` , so we were sending a date in the past.
The value of this attribute is used by macOS in "System Preferences >
Profiles" to display a note saying "This Mac is supervised and managed
by <xyz>".
When <PayloadOrganization> is not set, it uses the URL of the MDM server
for the message, which looks unpolished.
The OVAL analyzer falsely assumes that any vulnerabilities detected on a
host only come from OVAL. However, it is possible that NVD detects
vulnerabilities on these hosts even though it excludes software from
deb_packages and rpm_packages. For example, a python package twisted
v22.20 has a vulnerability CVE-2022-39348 detected by NVD. The OVAL
analyzer would delete this vulnerability, and it would be re-inserted by
the NVD scanner on the next run. This creates a loop.
The fix is to only delete vulnerabilities that are actually detected
using OVAL. We already store this in the source column in the
software_cve table.
This improves osquery-perf with support for a more realistic orbit + fleet desktop simulation as described in #8212
This was based on the work done by @sharvilshah in his branch.
Previously, the primary ip was determined using the interface with the most traffic. This had a few issues and required explicitly filtering out certain interfaces that we don't want eg loopback, docker.
Instead, use the default route to determine the primary (private) ip. If there are multiple, use the one with the lowest metric and prefer ipv4 addresses.
Related to https://github.com/fleetdm/fleet/issues/8010 and https://github.com/fleetdm/fleet/issues/8013 this prevents a bug that happens when:
1. A team doesn't have a `config.features` key in the JSON stored in the table or `config` is `NULL`
2. The team is edited from the UI
All `config.features` will default to `false`, which can be a problem if your global settings are `true` for both (which is the default)
related to #8031, this adds the following headers to HTML responses:
- Strict-Transport-Security: informs browsers that the site should only
be accessed using HTTPS, and that any future attempts to access it
using HTTP should automatically be converted to HTTPS.
- X-Frames-Options: disallows embedding the UI in other sites via
<frame>, <iframe>, <embed> or <object>, which can prevent attacks like
clickjacking.
- X-Content-Type-Options: prevents browsers from trying to guess the MIME
type which can cause browsers to transform non-executable content into
executable content.
- Referrer-Policy: prevents leaking the origin of the referrer in the
Referer.
additionally, this ensures we set `X-Content-Type-Options` for CSV and
installer responses.
Related to #7664, this cleans up all policy memberships for a host when its re-enrolled, afterwards only the relevant policy memberships for the host will be created.
* WIP
* Adding DEP functionality to Fleet
* Better organize additional MDM code
* Add cmdr.py and amend API paths
* Fix lint
* Add demo file
* Fix demo.md
* go mod tidy
* Add munki setup to Fleet
* Add diagram to demo.md
* Add fixes
* Update TODOs and demo.md
* Fix cmdr.py and add TODO
* Add endpoints to demo.md
* Add more Munki PoC/demo stuff
* WIP
* Remove proposals from PoC
* Replace prepare commands with fleetctl commands
* Update demo.md with current state
* Remove config field
* Amend demo
* Remove Munki setup from MVP-Dogfood
* Update demo.md
* Add apple mdm commands (#7769)
* fleetctl enqueue mdm command
* fix deps
* Fix build
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
* Add command to upload installers
* go mod tidy
* fix subcommands help
There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.
* Add support for installing apps
* Add a way to list enrolled devices
* Add dep listing
* Rearrange endpoints
* Move DEP routine to schedule
* Define paths globally
* Add a way to list enrollments and installers
* Parse device-ids as comma-separated string
* Remove unused types
* Add simple commands and nest under enqueue-command
* Fix simple commands
* Add help to enqueue-command
* merge apple_mdm database
* Fix commands
* update nanomdm
* Split nanomdm and nanodep schemas
* Set 512 MB in memory for upload
* Remove empty file
* Amend profile
* Add sample commands
* Add delete installers and fix bug in DEP profile assigning
* Add dogfood.md deployment guide
* Update schema.sql
* Dump schema with MySQL 5
* Set default value for authenticate_at
* add tokens to enrollment profiles
When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.
- Rename enrollments to enrollment profiles. Enrollments is used by nano
to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=
* remove mdm apple server url
* update docs
* make dump-test-schema
* Update nanomdm with missing prefix table
* Add docs and simplify changes
* Add changes file
* Add method docs
* Fix compile and revert prepare.go changes
* Revert migration status check change
* Amend comments
* Add more docs
* Clarify storage of installers
* Remove TODO
* Remove unused
* update dogfood.md
* remove cmdr.py
* Add authorization tests
* Add TODO comment
* use kitlog for nano logging
* Add yaml tags
* Remove unused flag
* Remove changes file
* Only run DEP routine if MDM is enabled
* Add docs to all new exported types
* Add docs
* more nano logging changes
* Fix unintentional removal
* more nano logging changes
* Fix compile test
* Use string for configs and fix config test
* Add docs and amend changes
* revert changes to basicAuthHandler
* remove exported BasicAuthHandler
* rename rego authz type
* Add more information to dep list
* add db tag
* update deps
* Fix schema
* Remove unimplemented
Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
* Support environments with revoked enroll secrets
* Add instructions on how to fix Orbit enroll
* Rename to last_recorded_error
* Add alternative instructions
related to https://github.com/fleetdm/fleet/issues/7199, this adds email validation to the `verifyCreateShared` which is used for user creation in the server.
validation messages come directly from Go's `net/mail` package.
```
~/fleet $ curl 'https://localhost:8080/api/latest/fleet/users/admin' -X POST -H 'Authorization: Bearer $TOKEN' --data-raw '{"email":"asdf","name":"asdf@asd.com","password":"as;lkdfjasdlk;fja3234@","global_role":"observer","teams":[]}'
{
"message": "Validation Failed",
"errors": [
{
"name": "email",
"reason": "mail: missing '@' or angle-addr"
}
]
}
```
This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop.
The general idea is to _always_ send a custom header of the form:
```
fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities
capabilities = capability * (,)
capability = string
```
Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: 8c0bbdd291
Also, the following applies:
- Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability
- The current capabilities endpoint will be removed
### Motivation
This solution is trying to solve the following problems:
- We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers.
- We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because:
- There are cases in which the server also needs to know which features are supported by its clients
- Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses.
- We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation.
Related to https://github.com/fleetdm/fleet/issues/7929
Configuration and fixes for the Fleet server and frontend to add support
for https://github.com/Uptycs/kubequery.
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
* Bump go to 1.19.1
* Bump remaining go-version to the 1.19.1
* Add extra paths for test-go
* Oops, putting the right path in the right place
* gofmt file
* gofmt ALL THE THINGS
* Moar changes
* Actually, go.mod doesn't like minor versions
- Add a new "Configuration for contributors" doc page. Move settings that are not recommended for production use
- Remove settings modified in the `config` YAML document from the deploying/configuration doc page
- Document all keys in `config` and `teams` YAML documents
- Add comments to several `.go` files and remove unused struct
This fixes an unreleased bug I introduced in eeefe2, as the config colum in the teams table is nulleable, it unmarshalls into nil and we can't dereference the variable
