fleet/server/service/orbit.go

337 lines
10 KiB
Go
Raw Normal View History

package service
import (
"context"
"encoding/json"
add headers denoting capabilities between fleet server / desktop / orbit (#7833) This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop. The general idea is to _always_ send a custom header of the form: ``` fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities capabilities = capability * (,) capability = string ``` Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: https://github.com/fleetdm/fleet/commit/8c0bbdd291f54e03e19766bcdfead0fb8067f60c Also, the following applies: - Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability - The current capabilities endpoint will be removed ### Motivation This solution is trying to solve the following problems: - We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers. - We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because: - There are cases in which the server also needs to know which features are supported by its clients - Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses. - We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation. Related to https://github.com/fleetdm/fleet/issues/7929
2022-09-26 10:53:53 +00:00
"fmt"
"net/http"
"github.com/fleetdm/fleet/v4/server"
"github.com/fleetdm/fleet/v4/server/contexts/ctxerr"
hostctx "github.com/fleetdm/fleet/v4/server/contexts/host"
"github.com/fleetdm/fleet/v4/server/contexts/logging"
"github.com/fleetdm/fleet/v4/server/fleet"
Orbit to set `--database_path` when invoking osquery to retrieve system info (#10308) #9132 The actual fix for the empty hosts is adding the `--database_path` argument in the initial `osqueryd -S` invocation when retrieving the UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not possible (which is what happens on some linux distributions), then it resorts to generating a new random UUID and storing it in the `osquery.db`. The issue was Orbit's first invocation of `osqueryd -S` was not using the same `osquery.db` as the main daemon invocation of `osqueryd`. I'm also adding a `hostname` + `platform` to the orbit enroll phase so that if there are any issues in the future we can avoid the "empty" host and have some information to help us troubleshoot. ## How to reproduce On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the hardware UUID. Some Linux distributions running on specific hardware or container environments do not have such file available. The way to reproduce on a Linux VM is to do the following: ```sh $ sudo su # chmod -r /sys/class/dmi/id/product_uuid ``` which will turn the file inaccessible to root. ## Checklist - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] Added/updated tests - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-03-13 21:54:18 +00:00
"github.com/go-kit/kit/log/level"
)
type setOrbitNodeKeyer interface {
setOrbitNodeKey(nodeKey string)
}
type orbitError struct {
message string
}
Orbit to set `--database_path` when invoking osquery to retrieve system info (#10308) #9132 The actual fix for the empty hosts is adding the `--database_path` argument in the initial `osqueryd -S` invocation when retrieving the UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not possible (which is what happens on some linux distributions), then it resorts to generating a new random UUID and storing it in the `osquery.db`. The issue was Orbit's first invocation of `osqueryd -S` was not using the same `osquery.db` as the main daemon invocation of `osqueryd`. I'm also adding a `hostname` + `platform` to the orbit enroll phase so that if there are any issues in the future we can avoid the "empty" host and have some information to help us troubleshoot. ## How to reproduce On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the hardware UUID. Some Linux distributions running on specific hardware or container environments do not have such file available. The way to reproduce on a Linux VM is to do the following: ```sh $ sudo su # chmod -r /sys/class/dmi/id/product_uuid ``` which will turn the file inaccessible to root. ## Checklist - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] Added/updated tests - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-03-13 21:54:18 +00:00
// EnrollOrbitRequest is the request Orbit instances use to enroll to Fleet.
type EnrollOrbitRequest struct {
Orbit to set `--database_path` when invoking osquery to retrieve system info (#10308) #9132 The actual fix for the empty hosts is adding the `--database_path` argument in the initial `osqueryd -S` invocation when retrieving the UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not possible (which is what happens on some linux distributions), then it resorts to generating a new random UUID and storing it in the `osquery.db`. The issue was Orbit's first invocation of `osqueryd -S` was not using the same `osquery.db` as the main daemon invocation of `osqueryd`. I'm also adding a `hostname` + `platform` to the orbit enroll phase so that if there are any issues in the future we can avoid the "empty" host and have some information to help us troubleshoot. ## How to reproduce On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the hardware UUID. Some Linux distributions running on specific hardware or container environments do not have such file available. The way to reproduce on a Linux VM is to do the following: ```sh $ sudo su # chmod -r /sys/class/dmi/id/product_uuid ``` which will turn the file inaccessible to root. ## Checklist - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] Added/updated tests - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-03-13 21:54:18 +00:00
// EnrollSecret is the secret to authenticate the enroll request.
EnrollSecret string `json:"enroll_secret"`
// HardwareUUID is the device's hardware UUID.
HardwareUUID string `json:"hardware_uuid"`
// HardwareSerial is the device's serial number.
HardwareSerial string `json:"hardware_serial"`
Orbit to set `--database_path` when invoking osquery to retrieve system info (#10308) #9132 The actual fix for the empty hosts is adding the `--database_path` argument in the initial `osqueryd -S` invocation when retrieving the UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not possible (which is what happens on some linux distributions), then it resorts to generating a new random UUID and storing it in the `osquery.db`. The issue was Orbit's first invocation of `osqueryd -S` was not using the same `osquery.db` as the main daemon invocation of `osqueryd`. I'm also adding a `hostname` + `platform` to the orbit enroll phase so that if there are any issues in the future we can avoid the "empty" host and have some information to help us troubleshoot. ## How to reproduce On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the hardware UUID. Some Linux distributions running on specific hardware or container environments do not have such file available. The way to reproduce on a Linux VM is to do the following: ```sh $ sudo su # chmod -r /sys/class/dmi/id/product_uuid ``` which will turn the file inaccessible to root. ## Checklist - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] Added/updated tests - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-03-13 21:54:18 +00:00
// Hostname is the device's hostname.
Hostname string `json:"hostname"`
// Platform is the device's platform as defined by osquery.
Platform string `json:"platform"`
}
type EnrollOrbitResponse struct {
OrbitNodeKey string `json:"orbit_node_key,omitempty"`
Err error `json:"error,omitempty"`
}
type orbitGetConfigRequest struct {
OrbitNodeKey string `json:"orbit_node_key"`
}
func (r *orbitGetConfigRequest) setOrbitNodeKey(nodeKey string) {
r.OrbitNodeKey = nodeKey
}
func (r *orbitGetConfigRequest) orbitHostNodeKey() string {
return r.OrbitNodeKey
}
type orbitGetConfigResponse struct {
fleet.OrbitConfig
Err error `json:"error,omitempty"`
}
func (r orbitGetConfigResponse) error() error { return r.Err }
func (e orbitError) Error() string {
return e.message
}
func (r EnrollOrbitResponse) error() error { return r.Err }
add headers denoting capabilities between fleet server / desktop / orbit (#7833) This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop. The general idea is to _always_ send a custom header of the form: ``` fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities capabilities = capability * (,) capability = string ``` Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: https://github.com/fleetdm/fleet/commit/8c0bbdd291f54e03e19766bcdfead0fb8067f60c Also, the following applies: - Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability - The current capabilities endpoint will be removed ### Motivation This solution is trying to solve the following problems: - We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers. - We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because: - There are cases in which the server also needs to know which features are supported by its clients - Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses. - We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation. Related to https://github.com/fleetdm/fleet/issues/7929
2022-09-26 10:53:53 +00:00
// hijackRender so we can add a header with the server capabilities in the
// response, allowing Orbit to know what features are available without the
// need to enroll.
func (r EnrollOrbitResponse) hijackRender(ctx context.Context, w http.ResponseWriter) {
add headers denoting capabilities between fleet server / desktop / orbit (#7833) This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop. The general idea is to _always_ send a custom header of the form: ``` fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities capabilities = capability * (,) capability = string ``` Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: https://github.com/fleetdm/fleet/commit/8c0bbdd291f54e03e19766bcdfead0fb8067f60c Also, the following applies: - Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability - The current capabilities endpoint will be removed ### Motivation This solution is trying to solve the following problems: - We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers. - We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because: - There are cases in which the server also needs to know which features are supported by its clients - Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses. - We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation. Related to https://github.com/fleetdm/fleet/issues/7929
2022-09-26 10:53:53 +00:00
writeCapabilitiesHeader(w, fleet.ServerOrbitCapabilities)
enc := json.NewEncoder(w)
enc.SetIndent("", " ")
if err := enc.Encode(r); err != nil {
Add UUID to Fleet errors and clean up error msgs (#10411) #8129 Apart from fixing the issue in #8129, this change also introduces UUIDs to Fleet errors. To be able to match a returned error from the API to a error in the Fleet logs. See https://fleetdm.slack.com/archives/C019WG4GH0A/p1677780622769939 for more context. Samples with the changes in this PR: ``` curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d '' { "message": "Bad request", "errors": [ { "name": "base", "reason": "Expected JSON Body" } ], "uuid": "a01f6e10-354c-4ff0-b96e-1f64adb500b0" } ``` ``` curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d 'asd' { "message": "Bad request", "errors": [ { "name": "base", "reason": "json decoder error" } ], "uuid": "5f716a64-7550-464b-a1dd-e6a505a9f89d" } ``` ``` curl -k -X GET -H "Authorization: Bearer badtoken" "https://localhost:8080/api/latest/fleet/teams" { "message": "Authentication required", "errors": [ { "name": "base", "reason": "Authentication required" } ], "uuid": "efe45bc0-f956-4bf9-ba4f-aa9020a9aaaf" } ``` ``` curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}' { "message": "Authorization header required", "errors": [ { "name": "base", "reason": "Authorization header required" } ], "uuid": "57f78cd0-4559-464f-9df7-36c9ef7c89b3" } ``` ``` curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}' { "message": "Permission Denied", "uuid": "7f0220ad-6de7-4faf-8b6c-8d7ff9d2ca06" } ``` - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [X] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md) - ~[ ] Documented any permissions changes~ - ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements)~ - ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features.~ - [X] Added/updated tests - [X] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [X] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - ~[ ] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-03-13 16:44:06 +00:00
encodeError(ctx, newOsqueryError(fmt.Sprintf("orbit enroll failed: %s", err)), w)
add headers denoting capabilities between fleet server / desktop / orbit (#7833) This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop. The general idea is to _always_ send a custom header of the form: ``` fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities capabilities = capability * (,) capability = string ``` Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: https://github.com/fleetdm/fleet/commit/8c0bbdd291f54e03e19766bcdfead0fb8067f60c Also, the following applies: - Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability - The current capabilities endpoint will be removed ### Motivation This solution is trying to solve the following problems: - We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers. - We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because: - There are cases in which the server also needs to know which features are supported by its clients - Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses. - We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation. Related to https://github.com/fleetdm/fleet/issues/7929
2022-09-26 10:53:53 +00:00
}
}
func enrollOrbitEndpoint(ctx context.Context, request interface{}, svc fleet.Service) (errorer, error) {
req := request.(*EnrollOrbitRequest)
Orbit to set `--database_path` when invoking osquery to retrieve system info (#10308) #9132 The actual fix for the empty hosts is adding the `--database_path` argument in the initial `osqueryd -S` invocation when retrieving the UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not possible (which is what happens on some linux distributions), then it resorts to generating a new random UUID and storing it in the `osquery.db`. The issue was Orbit's first invocation of `osqueryd -S` was not using the same `osquery.db` as the main daemon invocation of `osqueryd`. I'm also adding a `hostname` + `platform` to the orbit enroll phase so that if there are any issues in the future we can avoid the "empty" host and have some information to help us troubleshoot. ## How to reproduce On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the hardware UUID. Some Linux distributions running on specific hardware or container environments do not have such file available. The way to reproduce on a Linux VM is to do the following: ```sh $ sudo su # chmod -r /sys/class/dmi/id/product_uuid ``` which will turn the file inaccessible to root. ## Checklist - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] Added/updated tests - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-03-13 21:54:18 +00:00
nodeKey, err := svc.EnrollOrbit(ctx, fleet.OrbitHostInfo{
HardwareUUID: req.HardwareUUID,
HardwareSerial: req.HardwareSerial,
Hostname: req.Hostname,
Platform: req.Platform,
}, req.EnrollSecret)
if err != nil {
return EnrollOrbitResponse{Err: err}, nil
}
return EnrollOrbitResponse{OrbitNodeKey: nodeKey}, nil
}
func (svc *Service) AuthenticateOrbitHost(ctx context.Context, orbitNodeKey string) (*fleet.Host, bool, error) {
svc.authz.SkipAuthorization(ctx)
if orbitNodeKey == "" {
return nil, false, ctxerr.Wrap(ctx, fleet.NewAuthRequiredError("authentication error: missing orbit node key"))
}
host, err := svc.ds.LoadHostByOrbitNodeKey(ctx, orbitNodeKey)
switch {
case err == nil:
// OK
case fleet.IsNotFound(err):
return nil, false, ctxerr.Wrap(ctx, fleet.NewAuthRequiredError("authentication error: invalid orbit node key"))
default:
return nil, false, ctxerr.Wrap(ctx, err, "authentication error orbit")
}
return host, svc.debugEnabledForHost(ctx, host.ID), nil
}
Orbit to set `--database_path` when invoking osquery to retrieve system info (#10308) #9132 The actual fix for the empty hosts is adding the `--database_path` argument in the initial `osqueryd -S` invocation when retrieving the UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not possible (which is what happens on some linux distributions), then it resorts to generating a new random UUID and storing it in the `osquery.db`. The issue was Orbit's first invocation of `osqueryd -S` was not using the same `osquery.db` as the main daemon invocation of `osqueryd`. I'm also adding a `hostname` + `platform` to the orbit enroll phase so that if there are any issues in the future we can avoid the "empty" host and have some information to help us troubleshoot. ## How to reproduce On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the hardware UUID. Some Linux distributions running on specific hardware or container environments do not have such file available. The way to reproduce on a Linux VM is to do the following: ```sh $ sudo su # chmod -r /sys/class/dmi/id/product_uuid ``` which will turn the file inaccessible to root. ## Checklist - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] Added/updated tests - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-03-13 21:54:18 +00:00
// EnrollOrbit enrolls an Orbit instance to Fleet and returns the orbit node key.
func (svc *Service) EnrollOrbit(ctx context.Context, hostInfo fleet.OrbitHostInfo, enrollSecret string) (string, error) {
// this is not a user-authenticated endpoint
svc.authz.SkipAuthorization(ctx)
Orbit to set `--database_path` when invoking osquery to retrieve system info (#10308) #9132 The actual fix for the empty hosts is adding the `--database_path` argument in the initial `osqueryd -S` invocation when retrieving the UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not possible (which is what happens on some linux distributions), then it resorts to generating a new random UUID and storing it in the `osquery.db`. The issue was Orbit's first invocation of `osqueryd -S` was not using the same `osquery.db` as the main daemon invocation of `osqueryd`. I'm also adding a `hostname` + `platform` to the orbit enroll phase so that if there are any issues in the future we can avoid the "empty" host and have some information to help us troubleshoot. ## How to reproduce On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the hardware UUID. Some Linux distributions running on specific hardware or container environments do not have such file available. The way to reproduce on a Linux VM is to do the following: ```sh $ sudo su # chmod -r /sys/class/dmi/id/product_uuid ``` which will turn the file inaccessible to root. ## Checklist - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] Added/updated tests - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-03-13 21:54:18 +00:00
logging.WithLevel(
logging.WithExtras(ctx,
"hardware_uuid", hostInfo.HardwareUUID,
"hardware_serial", hostInfo.HardwareSerial,
"hostname", hostInfo.Hostname,
"platform", hostInfo.Platform,
),
level.Info,
)
secret, err := svc.ds.VerifyEnrollSecret(ctx, enrollSecret)
if err != nil {
if fleet.IsNotFound(err) {
// OK - This can happen if the following sequence of events take place:
// 1. User deletes global/team enroll secret.
// 2. User deletes the host in Fleet.
// 3. Orbit tries to re-enroll using old secret.
return "", fleet.NewAuthFailedError("invalid secret")
}
return "", orbitError{message: err.Error()}
}
orbitNodeKey, err := server.GenerateRandomText(svc.config.Osquery.NodeKeySize)
if err != nil {
return "", orbitError{message: "failed to generate orbit node key: " + err.Error()}
}
appConfig, err := svc.ds.AppConfig(ctx)
if err != nil {
return "", orbitError{message: "app config load failed: " + err.Error()}
}
Orbit to set `--database_path` when invoking osquery to retrieve system info (#10308) #9132 The actual fix for the empty hosts is adding the `--database_path` argument in the initial `osqueryd -S` invocation when retrieving the UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not possible (which is what happens on some linux distributions), then it resorts to generating a new random UUID and storing it in the `osquery.db`. The issue was Orbit's first invocation of `osqueryd -S` was not using the same `osquery.db` as the main daemon invocation of `osqueryd`. I'm also adding a `hostname` + `platform` to the orbit enroll phase so that if there are any issues in the future we can avoid the "empty" host and have some information to help us troubleshoot. ## How to reproduce On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the hardware UUID. Some Linux distributions running on specific hardware or container environments do not have such file available. The way to reproduce on a Linux VM is to do the following: ```sh $ sudo su # chmod -r /sys/class/dmi/id/product_uuid ``` which will turn the file inaccessible to root. ## Checklist - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] Added/updated tests - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-03-13 21:54:18 +00:00
_, err = svc.ds.EnrollOrbit(ctx, appConfig.MDM.EnabledAndConfigured, hostInfo, orbitNodeKey, secret.TeamID)
if err != nil {
return "", orbitError{message: "failed to enroll " + err.Error()}
}
return orbitNodeKey, nil
}
func getOrbitConfigEndpoint(ctx context.Context, request interface{}, svc fleet.Service) (errorer, error) {
cfg, err := svc.GetOrbitConfig(ctx)
if err != nil {
return orbitGetConfigResponse{Err: err}, nil
}
return orbitGetConfigResponse{OrbitConfig: cfg}, nil
}
func (svc *Service) GetOrbitConfig(ctx context.Context) (fleet.OrbitConfig, error) {
// this is not a user-authenticated endpoint
svc.authz.SkipAuthorization(ctx)
var notifs fleet.OrbitConfigNotifications
host, ok := hostctx.FromContext(ctx)
if !ok {
return fleet.OrbitConfig{Notifications: notifs}, orbitError{message: "internal error: missing host from request context"}
}
config, err := svc.ds.AppConfig(ctx)
if err != nil {
return fleet.OrbitConfig{Notifications: notifs}, err
}
// set the host's orbit notifications
if config.MDM.EnabledAndConfigured && host.IsOsqueryEnrolled() {
if host.NeedsDEPEnrollment() {
notifs.RenewEnrollmentProfile = true
}
if config.MDM.MacOSMigration.Enable &&
host.IsElegibleForDEPMigration() {
notifs.NeedsMDMMigration = true
}
if host.DiskEncryptionResetRequested != nil && *host.DiskEncryptionResetRequested {
notifs.RotateDiskEncryptionKey = true
// Since this is an user initiated action, we disable
// the flag when we deliver the notification to Orbit
if err := svc.ds.SetDiskEncryptionResetStatus(ctx, host.ID, false); err != nil {
return fleet.OrbitConfig{Notifications: notifs}, err
}
}
}
// team ID is not nil, get team specific flags and options
if host.TeamID != nil {
teamAgentOptions, err := svc.ds.TeamAgentOptions(ctx, *host.TeamID)
if err != nil {
return fleet.OrbitConfig{Notifications: notifs}, err
}
var opts fleet.AgentOptions
if teamAgentOptions != nil && len(*teamAgentOptions) > 0 {
if err := json.Unmarshal(*teamAgentOptions, &opts); err != nil {
return fleet.OrbitConfig{Notifications: notifs}, err
}
}
mdmConfig, err := svc.ds.TeamMDMConfig(ctx, *host.TeamID)
if err != nil {
return fleet.OrbitConfig{Notifications: notifs}, err
}
var nudgeConfig *fleet.NudgeConfig
if mdmConfig != nil &&
mdmConfig.MacOSUpdates.Deadline.Value != "" &&
mdmConfig.MacOSUpdates.MinimumVersion.Value != "" {
nudgeConfig, err = fleet.NewNudgeConfig(mdmConfig.MacOSUpdates)
if err != nil {
return fleet.OrbitConfig{Notifications: notifs}, err
}
}
return fleet.OrbitConfig{
Flags: opts.CommandLineStartUpFlags,
Extensions: opts.Extensions,
Notifications: notifs,
NudgeConfig: nudgeConfig,
}, nil
}
// team ID is nil, get global flags and options
var opts fleet.AgentOptions
if config.AgentOptions != nil {
if err := json.Unmarshal(*config.AgentOptions, &opts); err != nil {
return fleet.OrbitConfig{Notifications: notifs}, err
}
}
var nudgeConfig *fleet.NudgeConfig
if config.MDM.MacOSUpdates.Deadline.Value != "" &&
config.MDM.MacOSUpdates.MinimumVersion.Value != "" {
nudgeConfig, err = fleet.NewNudgeConfig(config.MDM.MacOSUpdates)
if err != nil {
return fleet.OrbitConfig{Notifications: notifs}, err
}
}
return fleet.OrbitConfig{
Flags: opts.CommandLineStartUpFlags,
Extensions: opts.Extensions,
Notifications: notifs,
NudgeConfig: nudgeConfig,
}, nil
}
add headers denoting capabilities between fleet server / desktop / orbit (#7833) This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop. The general idea is to _always_ send a custom header of the form: ``` fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities capabilities = capability * (,) capability = string ``` Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: https://github.com/fleetdm/fleet/commit/8c0bbdd291f54e03e19766bcdfead0fb8067f60c Also, the following applies: - Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability - The current capabilities endpoint will be removed ### Motivation This solution is trying to solve the following problems: - We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers. - We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because: - There are cases in which the server also needs to know which features are supported by its clients - Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses. - We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation. Related to https://github.com/fleetdm/fleet/issues/7929
2022-09-26 10:53:53 +00:00
/////////////////////////////////////////////////////////////////////////////////
// Ping orbit endpoint
/////////////////////////////////////////////////////////////////////////////////
type orbitPingRequest struct{}
type orbitPingResponse struct{}
func (r orbitPingResponse) hijackRender(ctx context.Context, w http.ResponseWriter) {
writeCapabilitiesHeader(w, fleet.ServerOrbitCapabilities)
}
func (r orbitPingResponse) error() error { return nil }
add headers denoting capabilities between fleet server / desktop / orbit (#7833) This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop. The general idea is to _always_ send a custom header of the form: ``` fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities capabilities = capability * (,) capability = string ``` Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: https://github.com/fleetdm/fleet/commit/8c0bbdd291f54e03e19766bcdfead0fb8067f60c Also, the following applies: - Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability - The current capabilities endpoint will be removed ### Motivation This solution is trying to solve the following problems: - We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers. - We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because: - There are cases in which the server also needs to know which features are supported by its clients - Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses. - We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation. Related to https://github.com/fleetdm/fleet/issues/7929
2022-09-26 10:53:53 +00:00
// NOTE: we're intentionally not reading the capabilities header in this
// endpoint as is unauthenticated and we don't want to trust whatever comes in
// there.
func orbitPingEndpoint(ctx context.Context, request interface{}, svc fleet.Service) (errorer, error) {
add headers denoting capabilities between fleet server / desktop / orbit (#7833) This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop. The general idea is to _always_ send a custom header of the form: ``` fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities capabilities = capability * (,) capability = string ``` Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: https://github.com/fleetdm/fleet/commit/8c0bbdd291f54e03e19766bcdfead0fb8067f60c Also, the following applies: - Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability - The current capabilities endpoint will be removed ### Motivation This solution is trying to solve the following problems: - We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers. - We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because: - There are cases in which the server also needs to know which features are supported by its clients - Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses. - We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation. Related to https://github.com/fleetdm/fleet/issues/7929
2022-09-26 10:53:53 +00:00
svc.DisableAuthForPing(ctx)
return orbitPingResponse{}, nil
}
/////////////////////////////////////////////////////////////////////////////////
// SetOrUpdateDeviceToken endpoint
/////////////////////////////////////////////////////////////////////////////////
type setOrUpdateDeviceTokenRequest struct {
OrbitNodeKey string `json:"orbit_node_key"`
DeviceAuthToken string `json:"device_auth_token"`
}
func (r *setOrUpdateDeviceTokenRequest) setOrbitNodeKey(nodeKey string) {
r.OrbitNodeKey = nodeKey
}
func (r *setOrUpdateDeviceTokenRequest) orbitHostNodeKey() string {
return r.OrbitNodeKey
}
type setOrUpdateDeviceTokenResponse struct {
Err error `json:"error,omitempty"`
}
func (r setOrUpdateDeviceTokenResponse) error() error { return r.Err }
func setOrUpdateDeviceTokenEndpoint(ctx context.Context, request interface{}, svc fleet.Service) (errorer, error) {
req := request.(*setOrUpdateDeviceTokenRequest)
if err := svc.SetOrUpdateDeviceAuthToken(ctx, req.DeviceAuthToken); err != nil {
return setOrUpdateDeviceTokenResponse{Err: err}, nil
}
return setOrUpdateDeviceTokenResponse{}, nil
}
func (svc *Service) SetOrUpdateDeviceAuthToken(ctx context.Context, deviceAuthToken string) error {
// this is not a user-authenticated endpoint
svc.authz.SkipAuthorization(ctx)
host, ok := hostctx.FromContext(ctx)
if !ok {
Add UUID to Fleet errors and clean up error msgs (#10411) #8129 Apart from fixing the issue in #8129, this change also introduces UUIDs to Fleet errors. To be able to match a returned error from the API to a error in the Fleet logs. See https://fleetdm.slack.com/archives/C019WG4GH0A/p1677780622769939 for more context. Samples with the changes in this PR: ``` curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d '' { "message": "Bad request", "errors": [ { "name": "base", "reason": "Expected JSON Body" } ], "uuid": "a01f6e10-354c-4ff0-b96e-1f64adb500b0" } ``` ``` curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d 'asd' { "message": "Bad request", "errors": [ { "name": "base", "reason": "json decoder error" } ], "uuid": "5f716a64-7550-464b-a1dd-e6a505a9f89d" } ``` ``` curl -k -X GET -H "Authorization: Bearer badtoken" "https://localhost:8080/api/latest/fleet/teams" { "message": "Authentication required", "errors": [ { "name": "base", "reason": "Authentication required" } ], "uuid": "efe45bc0-f956-4bf9-ba4f-aa9020a9aaaf" } ``` ``` curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}' { "message": "Authorization header required", "errors": [ { "name": "base", "reason": "Authorization header required" } ], "uuid": "57f78cd0-4559-464f-9df7-36c9ef7c89b3" } ``` ``` curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}' { "message": "Permission Denied", "uuid": "7f0220ad-6de7-4faf-8b6c-8d7ff9d2ca06" } ``` - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [X] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md) - ~[ ] Documented any permissions changes~ - ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements)~ - ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features.~ - [X] Added/updated tests - [X] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [X] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - ~[ ] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-03-13 16:44:06 +00:00
return newOsqueryError("internal error: missing host from request context")
}
if err := svc.ds.SetOrUpdateDeviceAuthToken(ctx, host.ID, deviceAuthToken); err != nil {
Add UUID to Fleet errors and clean up error msgs (#10411) #8129 Apart from fixing the issue in #8129, this change also introduces UUIDs to Fleet errors. To be able to match a returned error from the API to a error in the Fleet logs. See https://fleetdm.slack.com/archives/C019WG4GH0A/p1677780622769939 for more context. Samples with the changes in this PR: ``` curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d '' { "message": "Bad request", "errors": [ { "name": "base", "reason": "Expected JSON Body" } ], "uuid": "a01f6e10-354c-4ff0-b96e-1f64adb500b0" } ``` ``` curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d 'asd' { "message": "Bad request", "errors": [ { "name": "base", "reason": "json decoder error" } ], "uuid": "5f716a64-7550-464b-a1dd-e6a505a9f89d" } ``` ``` curl -k -X GET -H "Authorization: Bearer badtoken" "https://localhost:8080/api/latest/fleet/teams" { "message": "Authentication required", "errors": [ { "name": "base", "reason": "Authentication required" } ], "uuid": "efe45bc0-f956-4bf9-ba4f-aa9020a9aaaf" } ``` ``` curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}' { "message": "Authorization header required", "errors": [ { "name": "base", "reason": "Authorization header required" } ], "uuid": "57f78cd0-4559-464f-9df7-36c9ef7c89b3" } ``` ``` curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}' { "message": "Permission Denied", "uuid": "7f0220ad-6de7-4faf-8b6c-8d7ff9d2ca06" } ``` - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [X] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md) - ~[ ] Documented any permissions changes~ - ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements)~ - ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features.~ - [X] Added/updated tests - [X] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [X] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - ~[ ] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-03-13 16:44:06 +00:00
return newOsqueryError(fmt.Sprintf("internal error: failed to set or update device auth token: %e", err))
}
return nil
}