Commit Graph

1173 Commits

Author SHA1 Message Date
Roberto Dip
6faee84f57
allow to configure a default global enroll secret (#6609)
Related to #6365 this adds a new config to set a global enroll token that will be used by the server.
2022-07-12 19:12:10 -03:00
Tomas Touceda
7f8099db57
Add sandbox fleet serve config (#6619)
* Add sandbox fleet serve config

* Update docs

* Maybe fix lint issues

* Add lint dep
2022-07-12 18:21:15 -03:00
Tomas Touceda
af0cf9b703
Add rate limits for device authed endpoints (#6529)
* Add rate limits for device authed endpoints

* Fix lint

* Add missing test

* Fix test

* Increase the quota for desktop endpoints

* Add comment about quota
2022-07-11 10:49:05 -03:00
Roberto Dip
1ed8efacfa
increase the length of host_batteries.health (#6560)
Rel to #6559, this increases the size of host_batteries.health for cases like "Check Battery" that exceed 10 chars
2022-07-08 21:06:50 +00:00
Juan Fernandez
ab8cde6371
Feature 6454: Add software_id column to software_cve table (#6489)
Added software_id to software_cve in preparation for the removal of the cpe_id column.
2022-07-07 10:22:10 -04:00
Lucas Manuel Rodriguez
700bb9aa98
Fix Datastore.HostByIdentifier to set SeenTime (#6484)
* Fix Datastore.HostByIdentifier to set SeenTime

* Add changes file
2022-07-05 08:08:43 -03:00
Zach Wasserman
03734a37aa
Add server support for Fleet Sandbox demo login (#6387)
* Add server support for Fleet Sandbox demo login

This adds an endpoint `/api/latest/fleet/demologin` that provides a
redirect for the fleetdm.com portion of Fleet Sandbox to automatically
log in a user. The username and password must be provided as form
values. The endpoint is only enabled if `FLEET_DEMO=1` is set in the
server environment.

This was tested locally with the following HTML served by `python3 -m
http.server`, and the Fleet server running with `FLEET_DEMO=1
./build/fleet serve --dev`:

```
<!DOCTYPE html>
  <body>
    <form
      method="post"
      action="https://localhost:8080/api/latest/fleet/demologin"
      id="demologin"
    >
      <input type="hidden" name="email" value="admin@example.com" />
      <input type="hidden" name="password" value="admin123123#" />
      <input type="submit"/>
    </form>
    <script type="text/javascript">
      document.forms["demologin"].submit();
    </script>
  </body>
</html>

```

For Fleet sandbox purposes, the `action` should be set to the correct
hostname for the sandbox instance, while the `email` and `password`
should be set to the same credentials that were provided when creating
the instance.

* lucas comments

* Add integration tests

* Fix status codes and add comments

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2022-07-01 16:52:55 -03:00
Zach Wasserman
db22f68c88
Separate health checks for MySQL and Redis (#6468)
This required a bit of refactoring of some mocking due to how the code
generation does not handle having the same function in different types.
2022-07-01 08:08:03 -03:00
Martin Angers
b9930930f5
Document the device-authenticated endpoints in API for Contributors (#6406) 2022-06-29 08:12:20 -04:00
Roberto Dip
b2d07e56b5
report error analytics (#6341)
Related to #5898, this reports an anonymized summary of errors stored in Redis into the analytics payload.

For each error stored, this includes:

- A `count` attribute with the number of occurrences of the error
- A `loc` attribute with the 3 topmost lines in the stack trace. Note that stack traces only contain package name + line number (example: github.com/fleetdm/fleet/server.go:12

This also includes a minor refactor around error types.
2022-06-28 16:31:14 -03:00
Martin Angers
539be8ee09
Add battery info in host details response (#6394) 2022-06-28 14:11:49 -04:00
Juan Fernandez
79bf51b03c
Feature 6242: Use oval to detect vulnerabilities on Fedora hosts (#6330)
* Feature 6242: Scan Fedora hosts using OVAL definitions
2022-06-24 11:02:51 -04:00
Juan Fernandez
9d01ba33c6
Feature 6096: Scan RHEL/CentOS hosts using OVAL definitions (#6241)
Extended the OVAL parser/analyzer so that we can scan RHEL based systems.
2022-06-23 16:44:45 -04:00
gillespi314
15de4f3e65
Update Fleet host detail query so os_version for Ubuntu hosts reflects accurate patch number in point release (#6360) 2022-06-23 15:24:18 -05:00
Lucas Manuel Rodriguez
fc7650c4f8
Fix mock tests set test name (#6345)
* Set mock test on live query mocked tests

* Use MySQL 5.7 as default in docker-compose.yml
2022-06-23 11:34:52 -03:00
Martin Angers
8f66a50f2b
Add hosts enrolled by operating system to anonymous usage stats (#6329) 2022-06-22 16:38:11 -04:00
Roberto Dip
2948e112f6
ensure software_host_counts is cleaned when software is deleted (#6270)
Related to #5982, this ensures we clean up software_host_counts rows referencing software that is not longer present in the software table.
2022-06-22 17:35:53 -03:00
Martin Angers
dc9e158153
Add weekly active users count in anonymous usage stats (#6317) 2022-06-22 15:20:57 -04:00
Lucas Manuel Rodriguez
9b210fc6bd
Add support for CA root certificate to Fleet Desktop (fleetctl package's --fleet-certificate flag) (#6312)
* Orbit to pass the value of `--fleet-certificate` to Fleet Desktop

* Add changes for testing
2022-06-21 16:25:36 -03:00
Roberto Dip
4a867d53dc
use a single context for background jobs and HTTP handlers (#6313) 2022-06-21 15:09:00 -03:00
Martin Angers
7bfe93f5d7
Include an error code as query string in /sso/callback response in case of failure (#6286) 2022-06-21 09:04:50 -04:00
Roberto Dip
984605f630
store count alongside unique error instances in Redis (#6232)
Related to #4972, this adds a new key, error:<hash>:count to Redis in order to keep track of how many times each instance of an error occurred.
2022-06-21 08:43:53 -03:00
Martin Angers
1d60d6fdc2
Add test for no host in failing policy trigger (#6279) 2022-06-20 11:41:45 -04:00
Aaron
75f093e802
Use nanoseconds for campaign IDs (#6216)
Fixes #4806 by adding resolution to the generated IDs.
2022-06-14 15:46:09 +00:00
gillespi314
ed4ae18602
Handle transparency url for downgraded license (#6207) 2022-06-13 16:03:51 -05:00
Martin Angers
81f0e0ccfa
Track active hosts count and enforce limit (#6099) 2022-06-13 16:29:32 -04:00
Roberto Dip
19c5e3545b
add a dedicated endpoint that redirects to fleet_desktop.transparency_url (#6204)
As part of https://github.com/fleetdm/fleet/issues/5947, and in order to have a simplified workflow in Fleet Desktop, we defined https://github.com/fleetdm/fleet/issues/6200 to add a new endpoint that redirects to the transparency url as defined in the config (for premium users only)

```
~/projects/fleet $ curl -v -s https://localhost:8080/api/latest/fleet/device/bf34ab98-23b0-48bc-8e82-8c0143cba11c/transparency
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 307
< content-type: application/json; charset=utf-8
< location: https://fleetdm.com/transparency
< content-length: 0
< date: Mon, 13 Jun 2022 18:09:29 GMT
<
* Connection #0 to host localhost left intact
```
2022-06-13 16:07:08 -03:00
Martin Angers
7f9bb6431e
Update team integrations to reference global integrations (part of failing policies automation support) (#6156) 2022-06-13 10:04:47 -04:00
Eng Zer Jun
abe33f1d8d
test: use T.TempDir to create temporary test directory (#6080)
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-06-13 10:20:38 -03:00
Lucas Manuel Rodriguez
11af33e9a1
Allow troubleshooting of mocked live query store (#6197) 2022-06-13 10:18:03 -03:00
Lucas Manuel Rodriguez
515454e47f
Fix no such table errors for mdm & munki_info in vanilla osquery macOS hosts (#6170)
* Add discovery queries for mdm and munki_info

* Add changes file

* Amend discovery table tests
2022-06-13 08:52:33 -03:00
Lucas Manuel Rodriguez
4cfeaa1580
Do not use golangci action for better reproducibility (use make lint-go) (#6175)
* Do not use golangci action for better reproducibility

* Add fix to trigger build

* Fix all reported issues

* fix more lint errors

* Add missing import

* Remove unused method

* Remove change not necessary
2022-06-10 18:52:24 -03:00
gillespi314
44f902218c
Improve live query UX (#5749) 2022-06-10 13:29:45 -05:00
gillespi314
a3ab5646f5
Add new fleet_desktop property to config object (#6151) 2022-06-10 10:39:02 -05:00
Roberto Dip
3bcd4e4ca6
Revert "pin api version in DeviceClient.ListDevicePolicies (#6139)" (#6163)
This reverts commit 4d184ef583.
2022-06-09 20:35:40 -03:00
Tomas Touceda
a15101601c
Use the improved users query everywhere (#6126)
* Use the improved users query everywhere

* Filter out users without a home dir
2022-06-09 17:18:28 -03:00
Lucas Manuel Rodriguez
3dd89bc0a1
Fix flaky test TestVulnerabilityDataStream (#6162) 2022-06-09 10:36:37 -03:00
Tomas Touceda
fc215610a4
Allow users to customize detail queries for troubleshooting purposes (#6150)
* Allow users to customize detail queries for troubleshooting purposes

* Address review comments
2022-06-09 10:33:49 -03:00
Roberto Dip
a6cf9b3d8d
add endpoint in devices API to query for api features (#6152)
Related to #6063, this adds a new device API to get an object with boolean values that we can use as feature flags to manage backwards compatibility in Fleet Desktop.
2022-06-09 10:17:55 -03:00
Michal Nicpon
bac168e3fd
fix mem usage cve sync (#6131) 2022-06-08 13:15:44 -06:00
Lucas Manuel Rodriguez
99833a3b92
Rearrange migrations (#6140)
Co-authored-by: Juan Fernandez <juan@fleetdm.com>
2022-06-08 16:03:23 -03:00
Juan Fernandez
337734fce1
Bug: Issue while parsing dpkg objects with inline names (#6146)
* Bug: Issue while parsing dpkg objects with inline names

Dpkg objects can also define their names inline, not only using variable references.
2022-06-08 14:29:58 -04:00
Roberto Dip
4d184ef583
pin api version in DeviceClient.ListDevicePolicies (#6139) 2022-06-08 11:34:40 -03:00
Juan Fernandez
ef73039559
Improve vulnerability detection for Ubuntu (#6102)
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts

To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
2022-06-07 21:09:47 -04:00
Martin Angers
0b5fcd8e27
Add support for custom headers in fleetctl (#6118) 2022-06-07 16:00:09 -04:00
Roberto Dip
19f995d3b5
only include policies in device endpoints for premium users (#6077)
This removes policy information from `GET /api/_version_/fleet/device/{token}` from non-premium Fleet instances.

Starting the server with `./build/fleet serve --dev --dev_license`

```bash
$ curl -s https://localhost:8080/api/latest/fleet/device/1804e808-171f-4dda-9bec-f695b2f2371a | jq '.host.policies // "not present"'
[
  {
    "id": 3,
    "name": "Antivirus healthy (Linux)",
    "query": "SELECT score FROM (SELECT case when COUNT(*) = 2 then 1 ELSE 0 END AS score FROM processes WHERE (name = 'clamd') OR (name = 'freshclam')) WHERE score == 1;",
    "description": "Checks that both ClamAV's daemon and its updater service (freshclam) are running.",
    "author_id": 1,
    "author_name": "Roberto",
    "author_email": "test@example.com",
    "team_id": null,
    "resolution": "Ensure ClamAV and Freshclam are installed and running.",
    "platform": "darwin,linux",
    "created_at": "2022-05-23T20:53:36Z",
    "updated_at": "2022-06-03T13:17:42Z",
    "response": ""
  }
]
```

Starting the server with `./build/fleet serve --dev`

```bash
$ curl -s https://localhost:8080/api/latest/fleet/device/1804e808-171f-4dda-9bec-f695b2f2371a | jq '.host.policies // "not present"'
"not present"
```
2022-06-07 13:27:13 -03:00
Martin Angers
a23e0c41ff
Support failing policies integrations (#5973) 2022-06-06 10:41:51 -04:00
Michal Nicpon
1044182b7b
fix epss probability (#6083) 2022-06-03 11:37:47 -06:00
Roberto Dip
842ebbb2ae
add Go client to consume device endpoints (#5987)
This adds a new API client named DeviceClient to server/service, meant to consume device endpoints and be used from Fleet Desktop.

Some of the logic to make requests and parse responses was very repetitive, so I introduced a private baseClient type and moved some of the logic of the existent Client there.

Related to #5685 and #5697
2022-06-01 20:05:05 -03:00
Michal Nicpon
30a864c8d2
Sync CVE scores periodically (#5838) 2022-06-01 10:06:57 -06:00