Commit Graph

804 Commits

Author SHA1 Message Date
Roberto Dip
d0cde1aaaa
Use the same HTTP server config as the production server in tests (#8254)
* Use the same HTTP server config as the production server in tests

This abstracts the default config we use to run the server into a
function so it can be used in tests to run an HTTP server using the same
configuration.

Additionally, this fixes a data race in tests, as an HTTP server
configuration can't be changed once you call `server.Start()`[1]

[1]:
https://cs.opensource.google/go/go/+/refs/tags/go1.19.2:src/net/http/httptest/server.go;l=40;drc=19309779ac5e2f5a2fd3cbb34421dafb2855ac21
2022-10-19 07:42:21 -03:00
Frank Sievertsen
7c3d9f007a
return http status 400 for bad json request body (#8287) 2022-10-18 14:43:16 +02:00
Martin Angers
323620bf39
Deep-clone the appconfig when getting from cache (#8194) 2022-10-17 15:03:49 -04:00
Frank Sievertsen
a2c47cf4f4
add host_display_name to live query rows (#8251) 2022-10-17 15:52:25 +02:00
Roberto Dip
e4cd25f4aa
fix a bug causing features to be false (#8240)
Related to https://github.com/fleetdm/fleet/issues/8010 and https://github.com/fleetdm/fleet/issues/8013 this prevents a bug that happens when:

1. A team doesn't have a `config.features` key in the JSON stored in the table or `config` is `NULL`
2. The team is edited from the UI

All `config.features` will default to `false`, which can be a problem if your global settings are `true` for both (which is the default)
2022-10-14 19:27:37 -03:00
Lucas Manuel Rodriguez
03096ee546
Fix race in tests (modifying already started server config) (#8227) 2022-10-14 12:00:16 -03:00
Frank Sievertsen
e54f066266
Add display_name to more results (#8204)
* Make HostResponses with one dedicated function only
* more display_name for Hosts
2022-10-14 11:14:18 -03:00
Martin Angers
fae8e4ca2c
Translate the AppConfig and Team validation messages to be more user-friendly (#8171) 2022-10-12 17:10:50 -04:00
Juan Fernandez
6939af045d
Fleet desktop should use lightweight endpoint for getting failing policies count (#8159)
Fleet desktop should use lightweight endpoint for getting failing policies count
2022-10-12 17:13:43 -03:00
Juan Fernandez
72cfdac634
Only ingest installed deb_packages (#8160)
Only ingest deb_packages with status = 'install ok installed'
2022-10-12 14:27:07 -04:00
Roberto Dip
4042f8d826
add browser-related security headers to HTML responses (#8180)
related to #8031, this adds the following headers to HTML responses:

- Strict-Transport-Security: informs browsers that the site should only
  be accessed using HTTPS, and that any future attempts to access it
  using HTTP should automatically be converted to HTTPS.
- X-Frames-Options: disallows embedding the UI in other sites via
  <frame>, <iframe>, <embed> or <object>, which can prevent attacks like
  clickjacking.
- X-Content-Type-Options: prevents browsers from trying to guess the MIME
  type which can cause browsers to transform non-executable content into
  executable content.
- Referrer-Policy: prevents leaking the origin of the referrer in the
  Referer.

additionally, this ensures we set `X-Content-Type-Options` for CSV and
installer responses.
2022-10-12 10:19:21 -03:00
Martin Angers
d321cfc68e
Add inherited policies to the team's list policies response payload (#8068) 2022-10-12 08:35:36 -04:00
Lucas Manuel Rodriguez
42c47a6fa7
Add missing return to ingestKubequeryInfo (#8178)
* Add missing return to ingestKubequeryInfo

* No need to log error twice
2022-10-12 09:00:49 -03:00
Frank Sievertsen
23199c3d83
Add missing display_name to getHostEndpoint (#8174) 2022-10-11 17:00:32 -04:00
Tomas Touceda
d912376f02
Handle tcp read timeouts (#8163)
* Handle tcp read timeouts properly

* Add changes file

* Fix bad mini refactor after nailing the test

* Update based on review

* Update comment
2022-10-11 13:58:52 -03:00
Roberto Dip
e630fabf89
token rotation for fleet desktop (#7779)
This implements what's described in detail here https://github.com/fleetdm/fleet/blob/main/proposals/fleet-desktop-token-rotation.md
2022-10-10 17:15:35 -03:00
gillespi314
ca379e7459
Fix directIngestOSWindows error in query ingestion for non-Windows hosts (#8148) 2022-10-10 13:27:15 -05:00
Frank Sievertsen
1eacecf637
Add new missing value to existing status URL parameter at the GET /hosts endpoint (#7916) 2022-10-10 07:45:39 -04:00
Frank Sievertsen
e9f7066d87
7135 host display name (#7873) 2022-10-08 08:57:46 -04:00
Roberto Dip
bfe698d090
cleanup all policy memberships for a host on re-enrollment (#8120)
Related to #7664, this cleans up all policy memberships for a host when its re-enrolled, afterwards only the relevant policy memberships for the host will be created.
2022-10-07 11:36:17 -03:00
Lucas Manuel Rodriguez
9191f4ce66
Add Apple MDM functionality (#7940)
* WIP

* Adding DEP functionality to Fleet

* Better organize additional MDM code

* Add cmdr.py and amend API paths

* Fix lint

* Add demo file

* Fix demo.md

* go mod tidy

* Add munki setup to Fleet

* Add diagram to demo.md

* Add fixes

* Update TODOs and demo.md

* Fix cmdr.py and add TODO

* Add endpoints to demo.md

* Add more Munki PoC/demo stuff

* WIP

* Remove proposals from PoC

* Replace prepare commands with fleetctl commands

* Update demo.md with current state

* Remove config field

* Amend demo

* Remove Munki setup from MVP-Dogfood

* Update demo.md

* Add apple mdm commands (#7769)

* fleetctl enqueue mdm command

* fix deps

* Fix build

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>

* Add command to upload installers

* go mod tidy

* fix subcommands help

There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.

* Add support for installing apps

* Add a way to list enrolled devices

* Add dep listing

* Rearrange endpoints

* Move DEP routine to schedule

* Define paths globally

* Add a way to list enrollments and installers

* Parse device-ids as comma-separated string

* Remove unused types

* Add simple commands and nest under enqueue-command

* Fix simple commands

* Add help to enqueue-command

* merge apple_mdm database

* Fix commands

* update nanomdm

* Split nanomdm and nanodep schemas

* Set 512 MB in memory for upload

* Remove empty file

* Amend profile

* Add sample commands

* Add delete installers and fix bug in DEP profile assigning

* Add dogfood.md deployment guide

* Update schema.sql

* Dump schema with MySQL 5

* Set default value for authenticate_at

* add tokens to enrollment profiles

When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.

- Rename enrollments to enrollment profiles. Enrollments is used by nano
  to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=

* remove mdm apple server url

* update docs

* make dump-test-schema

* Update nanomdm with missing prefix table

* Add docs and simplify changes

* Add changes file

* Add method docs

* Fix compile and revert prepare.go changes

* Revert migration status check change

* Amend comments

* Add more docs

* Clarify storage of installers

* Remove TODO

* Remove unused

* update dogfood.md

* remove cmdr.py

* Add authorization tests

* Add TODO comment

* use kitlog for nano logging

* Add yaml tags

* Remove unused flag

* Remove changes file

* Only run DEP routine if MDM is enabled

* Add docs to all new exported types

* Add docs

* more nano logging changes

* Fix unintentional removal

* more nano logging changes

* Fix compile test

* Use string for configs and fix config test

* Add docs and amend changes

* revert changes to basicAuthHandler

* remove exported BasicAuthHandler

* rename rego authz type

* Add more information to dep list

* add db tag

* update deps

* Fix schema

* Remove unimplemented

Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 19:53:54 -03:00
RachelElysia
3054450065
Improve developer documentation: Update seed data documentation (#7904) 2022-10-05 12:42:45 -04:00
Martin Angers
c23a15394e
Keep created_at timestamp of existing enroll secrets (#8062) 2022-10-05 08:35:36 -04:00
Martin Angers
ec75fb10b2
Fix CI tests for Go (#8079) 2022-10-04 15:48:14 -04:00
gillespi314
e3590beaf7
Ingest ReleaseId from Windows registry (#8030) 2022-10-04 10:34:08 -05:00
Lucas Manuel Rodriguez
a4d7e81475
Orbit to support environments with revoked enroll secrets (#8056)
* Support environments with revoked enroll secrets

* Add instructions on how to fix Orbit enroll

* Rename to last_recorded_error

* Add alternative instructions
2022-10-03 17:28:19 -03:00
Roberto Dip
cd2ab6b17c
add server side validation for emails (#8022)
related to https://github.com/fleetdm/fleet/issues/7199, this adds email validation to the `verifyCreateShared` which is used for user creation in the server.

validation messages come directly from Go's `net/mail` package.

```
~/fleet $ curl 'https://localhost:8080/api/latest/fleet/users/admin' -X POST -H 'Authorization: Bearer $TOKEN' --data-raw '{"email":"asdf","name":"asdf@asd.com","password":"as;lkdfjasdlk;fja3234@","global_role":"observer","teams":[]}'
{
  "message": "Validation Failed",
  "errors": [
    {
      "name": "email",
      "reason": "mail: missing '@' or angle-addr"
    }
  ]
}
```
2022-10-03 13:29:01 -03:00
Martin Angers
8eee0b35f3
Add validations for agent options' command-line flags section. (#7979) 2022-10-03 08:29:41 -04:00
gillespi314
2a6895b015
Log user email upon successful SSO login (#8012) 2022-09-29 09:25:45 -05:00
Martin Angers
8023193ea8
Fix integration logger test (#8014) 2022-09-28 16:35:54 -04:00
gillespi314
749ff9ec2b
Add logging to capture user email upon successful login (#7927)
* Log user email upon successful login

* Add user email to logger context

* Use logging.With Extras for login email
2022-09-27 16:32:46 -03:00
Roberto Dip
b9927eade4
make orbit and device ping paths unversioned (#7933) 2022-09-26 14:39:56 -03:00
Roberto Dip
a509bdd0ac
add backwards compatiblity to orbit enrollment via the API (#7921) 2022-09-26 11:44:09 -03:00
Roberto Dip
2fcb27ed3f
add headers denoting capabilities between fleet server / desktop / orbit (#7833)
This adds a new mechanism to allow us to handle compatibility issues between Orbit, Fleet Server and Fleet Desktop.

The general idea is to _always_ send a custom header of the form:

```
fleet-capabilities-header = "X-Fleet-Capabilities:" capabilities
capabilities              = capability * (,)
capability                = string
```

Both from the server to the clients (Orbit, Fleet Desktop) and vice-versa. For an example, see: 8c0bbdd291

Also, the following applies:

- Backwards compat: if the header is not present, assume that orbit/fleet doesn't have the capability
- The current capabilities endpoint will be removed

### Motivation

This solution is trying to solve the following problems:

- We have three independent processes communicating with each other (Fleet Desktop, Orbit and Fleet Server). Each process can be updated independently, and therefore we need a way for each process to know what features are supported by its peers.
- We originally implemented a dedicated API endpoint in the server that returned a list of the capabilities (or "features") enabled, we found this, and any other server-only solution (like API versioning) to be insufficient because:
  - There are cases in which the server also needs to know which features are supported by its clients
  - Clients needed to poll for changes to detect if the capabilities supported by the server change, by sending the capabilities on each request we have a much cleaner way to handling different responses.
- We are also introducing an unauthenticated endpoint to get the server features, this gives us flexibility if we need to implement different authentication mechanisms, and was one of the pitfalls of the first implementation.

Related to https://github.com/fleetdm/fleet/issues/7929
2022-09-26 07:53:53 -03:00
Juan Fernandez
d7ca8fcd66
Reverted changes made to Fleet Desktop - should not use the lightweight 'desktop' endpoint (#7919)
Reverted changes made to Fleet Desktop. Desktop should not use the lightweight 'desktop' endpoint
2022-09-23 15:18:19 -04:00
Sharvil Shah
7d4e2e2b4b
Orbit remote management for flags (#7246)
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2022-09-24 00:30:23 +05:30
Martin Angers
5477acb688
Add low_disk_space_count property to GET /host_summary (#7872) 2022-09-21 15:56:17 -04:00
Martin Angers
db24cf32fa
Add new low_disk_space URL parameter to the GET /hosts endpoint (and GET /hosts/count) (#7853) 2022-09-21 15:16:31 -04:00
gillespi314
34688f531a
Refactor webhooks cron to new schedule package (#7840) 2022-09-20 14:26:36 -05:00
Martin Angers
478b4d3f69
Validate team and appconfig payloads, with dry-run and force modes (#7731) 2022-09-19 13:53:44 -04:00
gillespi314
6a3d9959fc
Refactor vulnerabilities cron to scheduler package (#7650) 2022-09-16 10:08:51 -05:00
Juan Fernandez
4ef883b311
Feature 6946: Fleet Desktop should use minimal api end-point for data (#7536)
Updated desktop client to use new EE desktop endpoint.
2022-09-15 12:12:50 -04:00
Zach Wasserman
486b67caca
Add initial support for kubequery (#6863)
Configuration and fixes for the Fleet server and frontend to add support
for https://github.com/Uptycs/kubequery.

Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-09-15 10:02:17 -06:00
Tomas Touceda
8457e55b53
Bump go to 1.19.1 (#7690)
* Bump go to 1.19.1

* Bump remaining go-version to the 1.19.1

* Add extra paths for test-go

* Oops, putting the right path in the right place

* gofmt file

* gofmt ALL THE THINGS

* Moar changes

* Actually, go.mod doesn't like minor versions
2022-09-12 20:32:43 -03:00
Juan Fernandez
b60d535d4a
Feature 7084: Add new EE endpoint for Fleet Desktop (#7530)
Added new EE endpoint, that is meant to be used by Fleet Desktop only. The new endpoint will return the number of failed policies.
2022-09-12 15:37:38 -04:00
Martin Angers
988f50fa34
Document all keys in config and team YAML documents (#7449)
- Add a new "Configuration for contributors" doc page. Move settings that are not recommended for production use
- Remove settings modified in the `config` YAML document from the deploying/configuration doc page
- Document all keys in `config` and `teams` YAML documents
- Add comments to several `.go` files and remove unused struct
2022-09-08 17:57:38 -04:00
Roberto Dip
740aafd817
Revert token rotation (#7628)
This reverts all changes related to token rotation.
2022-09-08 15:04:02 -03:00
Roberto Dip
9174c7c711
token rotation for Fleet Desktop (#7517)
See https://github.com/fleetdm/fleet/issues/6348 for a very detailed rundown

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2022-09-06 20:25:29 -03:00
Martin Angers
aa0102d6b6
Remove aggregate host counts from GET /hosts responses (#7510) 2022-09-06 10:34:06 -04:00
Michal Nicpon
0709d1bc5c
improve vuln cpe matching on macos (#6985)
* add cpe translations
* fix matching on target_sw
2022-09-01 10:02:07 -06:00