Commit Graph

747 Commits

Author SHA1 Message Date
Jacob Shandling
70e0182e4d Merge branch 'main' into 7766-main-merge 2023-10-11 12:56:52 -07:00
Roberto Dip
9ffa11c25d
Feat: saved scripts (#14409)
For #9537
2023-10-10 19:00:45 -03:00
Jacob Shandling
fb494ca3cb Merge branch '7766-frontend' into 7766-backend-frontend 2023-10-10 13:21:19 -07:00
Jahziel Villasana-Espinoza
0435431ae9
feat: can delete via cron job if global flag set (#14398)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added/updated tests

---------

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2023-10-10 10:06:54 -03:00
Tim Lee
fa55d72118
Save Query Results to DB (#14335)
# Checklist for submitter

#13486 

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Eric <eashaw@sailsjs.com>
Co-authored-by: Sampfluger88 <108141731+Sampfluger88@users.noreply.github.com>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Roberto Dip <me@roperzh.com>
Co-authored-by: Tyler Diderich <15862572+tdiderich@users.noreply.github.com>
Co-authored-by: Dave Herder <27025660+dherder@users.noreply.github.com>
Co-authored-by: Rachael Shaw <r@rachael.wtf>
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
Co-authored-by: Nathanael Holliday <100959072+hollidayn@users.noreply.github.com>
Co-authored-by: Katheryn Satterlee <me@ksatter.com>
Co-authored-by: Mo Zhu <mo@fleetdm.com>
Co-authored-by: Mo Zhu <mozhu888@gmail.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Robert Fairburn <8029478+rfairburn@users.noreply.github.com>
Co-authored-by: Sabrina Coy <13890648+sabrinabuckets@users.noreply.github.com>
Co-authored-by: Isabell Reedy <113355639+ireedy@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Alex Mitchell <105945793+alexmitchelliii@users.noreply.github.com>
2023-10-10 09:44:03 -03:00
Jahziel Villasana-Espinoza
5c868c9d3d
feat: deletion for query results (#14302)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
2023-10-09 17:43:17 -04:00
Jacob Shandling
eb327faabf
maintenance merge of main into feature branch (#14393)
maintenance merge of `main` into feature branch
2023-10-09 14:28:35 -07:00
Marcos Oviedo
f0d77ab3db
Merging Bitlocker feature branch (#14350)
This relates to #12577

---------

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2023-10-06 19:04:33 -03:00
Tim Lee
ccd6746633
7766 backmerge main (#14324) 2023-10-05 11:39:48 -06:00
Tim Lee
4b50cf811b
Revert Backmerge from Main 2023-10-04 20:56:17 -06:00
Jacob Shandling
ab50f0f59d
Maintenance merge of main into feature branch (#14308) 2023-10-04 15:59:36 -07:00
Tim Lee
38af9678f4
Backmerge from main (#14298) 2023-10-04 14:24:40 -06:00
Noah Talerman
9c7079f50c
Windows MDM docs: turn on Windows MDM (#13943)
- Add new "Windows setup" page to "Device Management" section of docs
- Rename "MDM setup"  page to "macOS setup." Update links and add redirect
- Rename existing "macOS setup" page to "macOS setup experience." Update links. Did not add redirect because of conflict with "macOS setup" page
- Remove "MDM" from all MDM doc page titles
2023-10-04 15:39:09 -04:00
Lucas Manuel Rodriguez
71a237042a
Add continues to not crash with panics on connection errors (#14223)
I found a few panics in osquery-perf while working in load testing
#13287.

These panic happened due to e.g. network errors.
2023-10-04 09:10:16 -03:00
Robert Henry
81a6c76bed
fix typo in help message in serve.go (#14037) 2023-09-26 13:06:17 -03:00
Jacob Shandling
1993c02b3a
Move code for --local-wix-dir into ee directory (#14104) 2023-09-25 10:38:03 -07:00
Jacob Shandling
b6f072775f
Add ability for fleetctl package to use local WiX v3 binaries when generating installer .msi (#14033) 2023-09-22 08:49:01 -07:00
Tim Lee
e80ea50240
Flock vulnerability false positive (#13827) 2023-09-14 10:38:36 -06:00
Roberto Dip
541cc638e4
Add an activity log when an script execution is requested (#13655)
For #9553
2023-09-05 20:09:29 -03:00
gillespi314
37fb4b0dab
Add fleetctl run-script command (#13622) 2023-09-05 14:14:09 -05:00
Martin Angers
cbc3f32e9d
Adjust response payload, messages and validations for /scripts/run/* endpoints. (#13607) 2023-08-31 09:08:50 -05:00
Lucas Manuel Rodriguez
9142c5de79
Prevent thundering herd when applying large number of policies on large number of hosts (#13552)
#13527

(Adding @mna to double check the changes in the async implementation of
policy result storage)

This PR also adds the osquery-perf changes needed to define the count of
macOS and Windows hosts.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~

Test with 80k hosts: 70k simulated macOS, 10k simulated Windows.
Apply Windows policies first, then apply macOS policies:
```
fleetctl apply -f ee/cis/win-10/cis-policy-queries.yml

# Leave running for some time

fleetctl apply -f ee/cis/macos-13/cis-policy-queries.yml
```

After applying CIS policies previous to these changes:
![Screenshot 2023-08-23 at 11 36
18](https://github.com/fleetdm/fleet/assets/2073526/72c1dc7d-e601-4248-be35-93c85b749f5d)

After applying these changes and applying the same policies:
![Screenshot 2023-08-28 at 15 42
57](https://github.com/fleetdm/fleet/assets/2073526/6b6d76b8-6acb-4893-a913-bf603a68f1a4)
2023-08-31 10:58:50 -03:00
Martin Angers
090b142c49
Implement script execution on the fleetd agent (disabled by default) (#13569) 2023-08-30 14:02:44 -04:00
Roberto Dip
b50e1939db
Allow to configure fleetd for script execution (#13564)
Related to #13310 and #13304 this adds two ways to enable script
execution in `fleetd` (the orbit component)

- By building a package with `--enable-scripts`
- By providing a setting via a configuration profile (macOS only)

Due to how the profile assignment works, this change automatically
updates the `com.fleetdm.fleetd.config` for hosts that already have the
profile installed.

> [!NOTE]
> Documentation is in
[#13577](https://github.com/fleetdm/fleet/pull/13577) to decouple
reviews.
2023-08-30 10:18:34 -03:00
Martin Angers
edf4a4d02f
Add script execution simulation to osquery-perf in preparation for load testing (part 3 of ticket) (#13456) 2023-08-23 18:31:47 -04:00
Tim Lee
3b61adf7a4
Add validation for policy specs (#13294) 2023-08-21 10:22:07 -06:00
Jacob Shandling
a7f1501054
Update fleetctl convert for schedulable queries (#13125) 2023-08-08 10:39:20 -07:00
Martin Angers
37ba43d404
Add fleetctl upgrade-packs command to migrate 2017 packs to queries (#13078) 2023-08-08 08:21:57 -04:00
Martin Angers
554e024f7b
Fix gitops access when using --dry-run with fleetctl apply (#13178) 2023-08-07 13:51:11 -04:00
Juan Fernandez
63c7c8a3ae
If no global queries, show proper message at the end (#13006)
If no global queries, show proper message at the end
2023-07-27 16:55:07 -04:00
Juan Fernandez
1dde10d5c3
Print inherited message if no queries found (#12998)
If the team has no queries, then fleetctl output should also show the number of inherited queries.
2023-07-27 15:08:40 -04:00
Juan Fernandez
e4cc0c3098
Fixed format issues with fleetctl get queries (#12983)
Output from `fleetctl get queries` should include the team the query is in and also the scheduling information.
2023-07-27 09:29:09 -04:00
Martin Angers
6f77911ffe
Fix performance regression found in load testing (#12981) 2023-07-26 17:13:27 -04:00
Lucas Manuel Rodriguez
2afbd24021
Combine Schedules and Queries: API changes (#12778)
Combining schedules and queries API changes.
2023-07-24 20:17:20 -04:00
Juan Fernandez
6b664a2a82
Merge branch 'main' into 7765-combined-schedules-and-queries 2023-07-21 13:53:13 -04:00
Lucas Manuel Rodriguez
b803bbe5bf
osquery-perf changes needed for load testing with simulated Windows hosts (#12754)
Changes in osquery-perf to allow for testing of Windows hosts in
loadtest environments.
2023-07-14 13:06:34 -03:00
Gabriel Hernandez
9aa7c0c714
add dark and light background logo colors and show them on mdm migrat… (#12681) 2023-07-13 19:35:25 +01:00
Roberto Dip
666ae8d787
ensure ds.TeamByName returns a 4xx response if no team is found (#12620)
this helps consumer of the datastore method handle the not found
scenario better and ensures we always return a 4xx code by default if we
can't find a matching team.

seems like calls to this method were special-cased everywhere except in
the apply user roles endpoint, where we returned a `500` status code if
we couldn't find a team.
2023-07-13 11:55:05 -03:00
Juan Fernandez
390e0565d0
Updated delete method on the DB layer 2023-07-07 07:31:36 -04:00
Juan Fernandez
807b2e35d3
Updated QueryByName DB access method 2023-07-06 19:37:08 -04:00
Marcos Oviedo
96449dd47b
Adding support for RequestSecurityToken messages - Windows MDM enroll endpoint (#12555)
This relates to #12263 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests

---------

Co-authored-by: Roberto Dip <me@roperzh.com>
2023-07-05 10:06:37 -03:00
gillespi314
410cbc3972
Add certificate management for Microsoft MDM (WSTEP) (#12543)
Issue #12261

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-06-29 19:31:53 -03:00
Marcos Oviedo
821f6b064f
Adding support for GetPolicies message (#12477)
This relates to #12262 

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2023-06-27 12:59:33 -03:00
Marcos Oviedo
22bb16bf2e
Pushing initial support for MS-MDE2 Discovery message (#12387)
This PR requires the Windows MDM configuration changes - This will be
updated next week

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Documented any permissions changes
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
2023-06-22 17:31:17 -03:00
Juan Fernandez
d08cc18111
Don't analyze non-windows OS using the MSRC scanner (#12421)
Don't analyze non-windows OS using the MSRC scanner
2023-06-21 14:19:55 -04:00
Martin Angers
1c249b60da
Add support to configure and enable Windows MDM, notify elegible hosts (#12340) 2023-06-20 14:06:45 -04:00
Lucas Manuel Rodriguez
bac309c978
Provide feedback to user in fleetctl login when using env vars (#12371)
Issue reported by @jarodreyes.

The user was not informed that `fleetctl login` was using the PASSWORD
environment variable:

`main`:
```sh
export PASSWORD=wrong

fleetctl login
Log in using the standard Fleet credentials.
Email: a@b.c
Error: Login failed: login received status 401 Authentication failed: Authentication failed
```
And with the changes on this PR:
```
export PASSWORD=wrong

fleetctl login
Log in using the standard Fleet credentials.
Email: a@b.c
Using value of environment variable $PASSWORD as password.
Error: Login failed: login received status 401 Authentication failed: Authentication failed
```
2023-06-16 11:42:04 -03:00
Juan Fernandez
7226b7f087
Warnings in fleetctl should use Stderr not Stdout (#12316)
Fixed issue were the expired license banner was being sent to Stdout instead of Stderr
2023-06-15 13:13:41 -04:00
Martin Angers
68fa60c54d
Add a transferred_hosts activity when hosts are transferred to a new team (#12287) 2023-06-14 08:15:05 -04:00
Lucas Manuel Rodriguez
2a532ede94
Do not return empty SSO and SMTP settings for non-global-admins (#12180)
#11266

PS: I first attempted a serialization trick by introducing a new
`appConfigResponse` and implementing `json.Marshal` to exclude these
fields but it was too hacky and hard to maintain moving forward, so I'm
bitting the bullet now. Happy to hear other ideas.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-06-07 16:06:36 -03:00
Martin Angers
f390055847
Fix table formatting of fleetctl get mdm-command-results output (#12197) 2023-06-07 14:35:48 -04:00
Martin Angers
68ddaafac0
Fix bug preventing gitops role from fleetctl applying macos setup assistant (and bootstrap package) (#12193) 2023-06-07 13:29:36 -04:00
Martin Angers
f27fcddd55
Prevent clearing macos updates settings when applying/modifying a team without those settings (#12160) 2023-06-06 14:31:33 -04:00
Martin Angers
48774876ea
Move post-DEP-enrollment processing to a worker job (#12017) 2023-06-05 11:58:23 -04:00
Lucas Manuel Rodriguez
2d21b2be29
Provide more feedback to the user when there's a Redis connection issue when running live queries (#11947)
This could help future users to detect this issue: #10957
It also adds an error log in Fleet that prints the actual error.

The error is displayed if I kill Redis during a live session or if I set
`client-output-buffer-limit` to something real low like `CONFIG SET
"client-output-buffer-limit" "pubsub 100kb 50kb 60"`:
![Screenshot 2023-05-25 at 09 08
08](https://github.com/fleetdm/fleet/assets/2073526/f021a77a-3a22-4b48-8073-bae9c6e21a11)

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-06-01 16:11:55 -03:00
Martin Angers
4322a28f5a
Implement preassign endpoint as first step to match profiles and hosts to teams (#12046) 2023-05-31 09:24:22 -04:00
Lucas Manuel Rodriguez
f0dbb981a3
Fix small fleetctl query bug when running with --exit flag (#11894)
Bug found while working on #10957.

Can be reproduced in our dogfood environment:
```
fleetctl query --context dogfood --query "SELECT * from osquery_info;" --hosts dogfood-centos-box --exit
⠋ %
```

With the changes in this PR:
```
fleetctl query --context dogfood --query "SELECT * from osquery_info;" --hosts dogfood-centos-box --exit
{"host":"dogfood-centos-box","rows":[{"build_distro":"centos7","build_platform":"linux","config_hash":"e3832343af2f8dc3e5ab62e709c78d3c3ef32b86","config_valid":"1","extensions":"active","host_display_name":"dogfood-centos-box","host_hostname":"dogfood-centos-box","instance_id":"9f0f6433-fbcf-4f15-8f1b-4dedc669ee2d","pid":"2760450","platform_mask":"9","start_time":"1684821735","uuid":"911CBDBA-7B3A-4B96-88F7-B28CECBEF400","version":"5.8.2","watcher":"2760447"}]}
⠦ 0% responded (0% online) | 0/1 targeted hosts (0/1 online) %
```
2023-05-25 08:12:45 -03:00
Lucas Manuel Rodriguez
3f05208192
Improve osquery-perf support for live queries (simulate no results and/or query failures) (#11883)
These osquery-perf changes were useful to test/troubleshoot #10957.
It basically allows osquery-perf to simulate no results and/or failures
when running live queries on the devices.

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-05-25 08:12:10 -03:00
Lucas Manuel Rodriguez
6415f062c6
Reduce size of DistributedQueryResult to improve live query performance (#11882)
This was found while working on #10957.

When running a live query, a lot of unused host data is stored in Redis
and sent on every live query result message via websockets. The frontend
and fleetctl just need `id`, `hostname` and `display_name`. (This
becomes worse every time we add new fields to the `Host` struct.)

Sample of one websocket message result when running `SELECT * from
osquery_info;`:

size in `main`: 2234 bytes
```
a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":57,\"host\":
{\"created_at\":\"2023-05-22T12:14:11Z\",\"updated_at\":\"2023-05-23T12:31:51Z\",
\"software_updated_at\":\"0001-01-01T00:00:00Z\",\"id\":106,\"detail_updated_at\":\"2023-05-23T11:50:04Z\",
\"label_updated_at\":\"2023-05-23T11:50:04Z\",\"policy_updated_at\":\"1970-01-02T00:00:00Z\",
\"last_enrolled_at\":\"2023-05-22T12:14:12Z\",
\"seen_time\":\"2023-05-23T09:52:23.876311-03:00\",\"refetch_requested\":false,
\"hostname\":\"lucass-macbook-pro.local\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"platform\":\"darwin\",\"osquery_version\":\"5.8.2\",\"os_version\":\"macOS 13.3.1\",\"build\":\"22E261\",\"platform_like\":\"darwin\",\"code_name\":\"\",
\"uptime\":91125000000000,\"memory\":34359738368,\"cpu_type\":\"x86_64h\",\"cpu_subtype\":\"Intel x86-64h Haswell\",\"cpu_brand\":\"Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz\",\"cpu_physical_cores\":4,\"cpu_logical_cores\":8,\"hardware_vendor\":\"Apple Inc.\",\"hardware_model\":\"MacBookPro16,2\",\"hardware_version\":\"1.0\",
\"hardware_serial\":\"0DPQR4HMD1FZ\",
\"computer_name\":\"Lucas’s MacBook Pro\",\"public_ip\":\"\",
\"primary_ip\":\"192.168.0.230\",\"primary_mac\":\"68:2f:67:8e:b6:1f\",
\"distributed_interval\":1,\"config_tls_refresh\":60,\"logger_tls_period\":10,\"team_id\":null,
\"pack_stats\":null,\"team_name\":null,
\"gigs_disk_space_available\":386.23,\"percent_disk_space_available\":40,
\"issues\":{\"total_issues_count\":0,\"failing_policies_count\":0},
\"mdm\":{\"enrollment_status\":null,\"server_url\":null,\"name\":\"\",\"encryption_key_available\":false},
\"status\":\"online\",\"display_text\":\"lucass-macbook-pro.local\",\"display_name\":\"Lucas’s MacBook Pro\"},
\"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\",
\"config_hash\":\"b7ee9363a7c686e76e99ffb122e9c5241a791e69\",\"config_valid\":\"1\",
\"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\",
\"host_hostname\":\"lucass-macbook-pro.local\",\"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\",\"start_time\":\"1684757652\",
\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",
\"version\":\"5.8.2\",\"watcher\":\"8364\"}],\"error\":null}}"]
```

vs. size of the message result on this branch: 675 bytes
```
a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":59,
\"host\":{\"id\":106,\"hostname\":\"lucass-macbook-pro.local\",
\"display_name\":\"Lucas’s MacBook Pro\"},
\"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\",
\"config_hash\":\"f80dee827635db39077a458243379b3ad63311fd\",
\"config_valid\":\"1\",\"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\",
\"host_hostname\":\"lucass-macbook-pro.local\",
\"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\",
\"start_time\":\"1684757652\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"version\":\"5.8.2\",
\"watcher\":\"8364\"}]}}"]
```

Manual tests included running with an old fleetctl running with a new
fleet server, and vice-versa, a new fleetctl running against an old
fleet server.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-05-25 08:11:53 -03:00
Roberto Dip
d0658868dc
adjust the UI to show a refetching spinner on critical queries (#11868)
A possible solution for https://github.com/fleetdm/fleet/issues/11860,
this updates the "My Device" page to show the spinner rotating if
critical queries are being fetched:



https://github.com/fleetdm/fleet/assets/4419992/0a84508a-4df2-4da3-b9bc-b3e75bab5e82


# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Manual QA for all new/changed functionality
2023-05-23 12:43:50 -03:00
Roberto Dip
dcc22f436b
allow to set await_device_configured in DEP profile (#11789)
#10744
2023-05-19 07:08:35 -03:00
Juan Fernandez
009a87d33e
Feature 10196: Add filepath to end-points and third party integrations (#11285)
Adds the software installed path property to the proper end-points and third party integrations (webhook, Zendesk and Jira).
2023-05-17 16:53:15 -04:00
Juan Fernandez
7f83135aa1
Feature: Store installed file path when ingesting software (#11214)
Store software installed paths into the host_software_installed_paths table when ingesting osquery software data.
2023-05-17 14:49:09 -04:00
Roberto Dip
4dd127d577
base logic to show/hide the new Migrate to Fleet FD menu (#11679)
Related to #11670
2023-05-15 17:00:52 -03:00
Martin Angers
7b1b392627
Implement worker jobs that update/re-assign setup assistants on changes (#11630) 2023-05-15 14:06:09 -04:00
gillespi314
ceeb4c1ed5
Add mdm.macos_migration to app config endpoints (#11694) 2023-05-15 11:50:07 -05:00
Lucas Manuel Rodriguez
bb3b21b574
Add TestMDMClient to simulate MDM clients in osquery-perf (#11672)
#11528

osquery-perf simulated hosts enroll and are identified as manually
enrolled. (Enrolling as DEP requires more work, e.g. a new mocked Apple
DEP endpoint).

Given that these are simulated MDM clients, they cannot be woken up with
push notifications. Instead, these check for new commands to execute
every 10 seconds (which is not realistic, but could serve as a good
loadtesting exercise).

I will now start setting up the loadtest environment with MDM enabled
and configured to test this.

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-05-12 13:50:20 -03:00
gillespi314
a9584dc32f
Allow end user authentication during automatic MDM enrollment to be enabled on a per-team basis (#11566) 2023-05-10 15:22:08 -05:00
gillespi314
434cc5938c
Improve error handling for fleetctl mdm run-comand (#11588) 2023-05-09 13:43:15 -05:00
Martin Angers
b3993ebda4
Allow "not_before" timestamp for worker jobs, schedule more quickly (#11512) 2023-05-03 16:25:36 -04:00
gillespi314
8fba5b87c7
Revise validation errors for fleetctl mdm run-command (#11464) 2023-05-03 10:56:25 -05:00
gillespi314
de987de674
Add license check and update testing for client-side bootstrap package upload flow via fleetctl apply (#11459) 2023-05-02 14:03:10 -05:00
Roberto Dip
a23d208b1d
gate DEP enrollment behind SSO when configured (#11309)
#10739

Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
2023-04-27 09:43:20 -03:00
Lucas Manuel Rodriguez
7dadec3ecf
Add mTLS support to fleetd (#11319)
#7970

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-04-27 08:44:39 -03:00
gillespi314
003e208e4d
Update CLI flow to manage adding and deleting MDM bootstrap packages by applying config and team specs (#11349) 2023-04-26 16:09:21 -05:00
Lucas Manuel Rodriguez
b9e6a84f24
Filter out non-observer_can_run queries for observers in fleetctl get queries command to match the UI. (#11251)
#11089

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-04-26 11:38:20 -03:00
Martin Angers
77855a5e1d
Add activities for macOS setup assistant. (#11310) 2023-04-26 10:37:03 -04:00
Roberto Dip
9068faf38f
Allow to configure SSO settings for MDM end user authentication (#11270)
Related to #10741, this adds a new key to app config named
`end_user_authentication`, which can be configured using the same keys
as the existing SSO feature.

Per the spec, if the feature is configured, it's implicitly enabled, at
least until we get to #10999.

Note that this only enables the SSO config, a second part of the ticket
with endpoints for the EULA will be tackled separately.
2023-04-25 12:16:33 -03:00
Martin Angers
582e85c876
Add support for the mdm.macos_setup.macos_setup_assistant key in fleetctl, API (#11296) 2023-04-25 09:36:01 -04:00
Zachary Winnerman
2ec19979c2
APM Improvements (#11103)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-04-24 01:13:15 -04:00
Roberto Dip
5c487890ca
add an endpoint to get an aggregate summary of bootstrap packages (#11156)
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2023-04-22 10:23:38 -05:00
Juan Fernandez
d353a3deb4
Handle flaky vulnerability tests (#11262)
- Refactored some of the vulnerabilities tests to help with flakiness.
- Don't load NVD assets if local assets have a timestamp of today.
2023-04-21 19:37:29 -04:00
gillespi314
bb2fbbdd38
Add apple_bm_enabled_and_configured to app config responses (#11255) 2023-04-21 11:08:09 -05:00
Martin Angers
c1d3f67e6f
Add fleetctl get mdm-commands command and supporting API endpoint (#11163) 2023-04-17 11:45:16 -04:00
Lucas Manuel Rodriguez
5aa5f8aae3
Add MDM configuration permissions to GitOps (#11207)
#8593 

Adding new MDM functionality to GitOps.

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-04-17 12:08:55 -03:00
Benjamin Edwards
03cc60a3db
remove code accidentally left in from previous PR (#11193) 2023-04-13 17:43:32 -04:00
Lucas Manuel Rodriguez
1ebfbb14eb
New gitops role (#10850)
#8593

This PR adds a new role `gitops` to Fleet.
MDM capabilities for the role coming on a separate PR. We need this
merged ASAP so that we can unblock the UI work for this.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [X] Documented any permissions changes
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-04-12 16:11:04 -03:00
Roberto Dip
77e5c004f4
implement bootstrap packages during DEP enrollment (#11052)
#10213
2023-04-07 17:31:02 -03:00
Benjamin Edwards
6f836d60cb
add support for AWS SES email backend (#10847) 2023-04-06 13:21:07 -05:00
Roberto Dip
40c5bb1c25
install fleetd on DEP enrolled hosts during enrollment (#10971)
https://github.com/fleetdm/fleet/issues/9459
2023-04-05 20:52:26 -03:00
Roberto Dip
1516caad46
explicitly enable orbit to read config from the system (#10980)
in #10134 we added a silent mechanism to try to read configuration
values from macOS configuration profiles if --fleet-url and
--enroll-secret weren't present.

while using this logic to test #9459 I have found that there's a race
condition where sometimes `fleetd` is installed before the configuration
profile with the values delivered by Fleet, causing orbit to get stuck
forever.

I added logic to loop every 30 seconds and try to fetch the values again
if none are found, but I didn't felt comfortable adding this logic
without also adding an extra flag to explicitly enable this behavior.
2023-04-05 15:02:18 -03:00
Lucas Manuel Rodriguez
2f38f2e76a
Uninstalling software in a host also updates software table (#10540)
https://github.com/fleetdm/confidential/issues/1968

It's ready for review but I still need to load test this.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-04-05 13:53:43 -03:00
Martin Angers
4ed50185c0
Add clearer message when mdm-command-results fails due to authz (#10997) 2023-04-05 11:57:31 -04:00
Martin Angers
741a7aa5d0
Finalize MDM commands part 3: add the fleetctl get mdm-command-results command (#10964) 2023-04-05 10:50:36 -04:00
Martin Angers
e0e547f1a2
Finalize MDM commands part 2: implement fleetctl mdm run-command (#10866) 2023-04-03 14:25:49 -04:00
Juan Fernandez
4c2ddba2e4
Clean out-of-date NVD results. (#10514)
Keep the vulnerabilities detected via NVD and stored in the DB in sync. with the results from the NVD vulnerability process.
2023-04-03 13:45:18 -04:00
Roberto Dip
729c1e4042
automatically create DEP JSON profiles if none is set. (#10871)
#9569
2023-03-30 14:25:30 -03:00
Martin Angers
0e2c9bb873
finalize mdm commands part 1: support fleetctl get hosts --mdm and --mdm-pending (#10796) 2023-03-29 08:30:49 -04:00
Juan Fernandez
aecc2fed75
Feature 9834: Add published date to vulnerability object (#10434)
This only applies to Premium users, we want to show the vulnerabilities' published date anywhere vulnerabilities are shown including API endpoints and third party integrations.
2023-03-28 16:11:31 -04:00
Noah Talerman
4372b5b2b5
Update broken links to MDM docs (#10789)
- At some point we updated how we generate anchor links on
fleetdm.com/docs. PR is here: #10657
  - For example:
-
`https://fleetdm.com/docs/using-fleet/mdm-setup#apple-push-notification-service-ap-ns`
is now
`https://fleetdm.com/docs/using-fleet/mdm-setup#apple-push-notification-service-apns`
(apns doesn't have a hyphen)
2023-03-27 17:14:20 -07:00
Roberto Dip
98845f03a6
remove feature flags to enable MDM (#10746)
https://github.com/fleetdm/fleet/issues/10025
2023-03-27 16:30:29 -03:00
Martin Angers
b6e10eb6da
Update host MDM profile status to pending in response to triggering events (#10443) 2023-03-27 14:43:01 -04:00
Lucas Manuel Rodriguez
e0d8dcad21
Allow prometheus /metrics endpoint with http basic auth disabled (#10634)
#8957

To test this feature, build+run Fleet and then visit:
`https://localhost:8080/metrics`.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [ ] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-03-23 19:00:11 -03:00
Martin Angers
2fb5aa629d
Normalize the naming of mdm settings, update docs and document missing ones (#10681)
#10408
2023-03-23 07:30:28 -03:00
Mo Zhu
e632725a44
Break up MDM docs to be smaller articles for improved readability. (#10538)
- Broke up the single MDM doc into multiple ones organized by category
- Changed any links to point to the new docs

---------

Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
2023-03-17 12:34:35 -07:00
Lucas Manuel Rodriguez
6d918d1366
Add enable_jit_role_sync config (#10478)
#8411

We decided to only update roles for existing accounts if enabled by a
new setting (disabled by default) `sso_settings.enable_jit_role_sync`.

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-03-14 17:17:08 -03:00
Lucas Manuel Rodriguez
5ec4fab440
Orbit to set --database_path when invoking osquery to retrieve system info (#10308)
#9132

The actual fix for the empty hosts is adding the `--database_path`
argument in the initial `osqueryd -S` invocation when retrieving the
UUID. Osquery attempts to retrieve the UUID from OS files/APIs, when not
possible (which is what happens on some linux distributions), then it
resorts to generating a new random UUID and storing it in the
`osquery.db`. The issue was Orbit's first invocation of `osqueryd -S`
was not using the same `osquery.db` as the main daemon invocation of
`osqueryd`.

I'm also adding a `hostname` + `platform` to the orbit enroll phase so
that if there are any issues in the future we can avoid the "empty" host
and have some information to help us troubleshoot.

## How to reproduce

On Linux, osquery reads `/sys/class/dmi/id/product_uuid` to load the
hardware UUID.
Some Linux distributions running on specific hardware or container
environments do not have such file available.
The way to reproduce on a Linux VM is to do the following:
```sh
$ sudo su
# chmod -r /sys/class/dmi/id/product_uuid
```
which will turn the file inaccessible to root.

## Checklist

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-03-13 18:54:18 -03:00
Roberto Dip
61f276fe44
allow to start Fleet MDM without configuring Apple BM (#10453)
Related to https://github.com/fleetdm/fleet/issues/10299, this allows to
start the Fleet server with MDM enabled without having to provide ABM
configs.

I have tested this with:

1. Premium account, no ABM config: the server starts normally, but
without ABM features
2. Premium account, invalid ABM config: error starting the server
3. Premium account, valid ABM config: ABM features enabled
4. Free account, no ABM config: the server starts normally
5. Free account, any ABM config: error due to invalid license
2023-03-13 18:16:11 -03:00
Lucas Manuel Rodriguez
b0475d998e
Run cleanup of cron_stats outside of the schedule package to prevent outages from breaking cron jobs (#10439)
#9486

Now cron jobs should recover from a Fleet outage after ~ two hours.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-03-13 16:15:30 -03:00
Lucas Manuel Rodriguez
93e150666b
Add instanceID to schedule logging (#10413)
#9515

Sample output after running `fleetctl trigger --name
cleanups_then_aggregation`:
```sh
./build/fleet serve --dev --dev_license 2>&1 | tee ~/fleet.txt
level=info ts=2023-03-09T19:27:17.324691Z component=redis mode=standalone
level=info ts=2023-03-09T19:27:17.360565Z instanceID="V9mArnX3lPhlIS0enyFau9eWi/dpjUPmOzJ3rwQUkX+l2aU1AMM4lQfdaDFZfeyJSHBwrIt/km1ghmRcyhdWqA=="
level=info ts=2023-03-09T19:27:17.372767Z msg="started cron schedules: automations, cleanups_then_aggregation, integrations, usage_statistics, vulnerabilities"
ts=2023-03-09T19:27:17.391404Z transport=https address=0.0.0.0:8080 msg=listening
level=error ts=2023-03-09T19:27:19.973841Z query=fleet_detail_query_software_macos message="distributed query is denylisted" hostID=58
level=info ts=2023-03-09T19:27:21.262799Z cron=cleanups_then_aggregation schedule=cleanups_then_aggregation instanceID="V9mArnX3lPhlIS0enyFau9eWi/dpjUPmOzJ3rwQUkX+l2aU1AMM4lQfdaDFZfeyJSHBwrIt/km1ghmRcyhdWqA==" status=pending
ts=2023-03-09T19:27:22.218129Z inf="skipping verification of encryption keys as MDM is not fully configured"
level=info ts=2023-03-09T19:27:22.224179Z cron=cleanups_then_aggregation schedule=cleanups_then_aggregation instanceID="V9mArnX3lPhlIS0enyFau9eWi/dpjUPmOzJ3rwQUkX+l2aU1AMM4lQfdaDFZfeyJSHBwrIt/km1ghmRcyhdWqA==" status=completed
```

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-03-13 15:37:03 -03:00
Lucas Manuel Rodriguez
3757aace08
Add UUID to Fleet errors and clean up error msgs (#10411)
#8129 

Apart from fixing the issue in #8129, this change also introduces UUIDs
to Fleet errors. To be able to match a returned error from the API to a
error in the Fleet logs. See
https://fleetdm.slack.com/archives/C019WG4GH0A/p1677780622769939 for
more context.

Samples with the changes in this PR:
```
curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d ''
{
  "message": "Bad request",
  "errors": [
    {
      "name": "base",
      "reason": "Expected JSON Body"
    }
  ],
  "uuid": "a01f6e10-354c-4ff0-b96e-1f64adb500b0"
}
```
```
curl -k -H "Authorization: Bearer $TEST_TOKEN" -H 'Content-Type:application/json' "https://localhost:8080/api/v1/fleet/sso" -d 'asd'
{
  "message": "Bad request",
  "errors": [
    {
      "name": "base",
      "reason": "json decoder error"
    }
  ],
  "uuid": "5f716a64-7550-464b-a1dd-e6a505a9f89d"
}
```
```
curl -k -X GET -H "Authorization: Bearer badtoken" "https://localhost:8080/api/latest/fleet/teams"
{
  "message": "Authentication required",
  "errors": [
    {
      "name": "base",
      "reason": "Authentication required"
    }
  ],
  "uuid": "efe45bc0-f956-4bf9-ba4f-aa9020a9aaaf"
}
```
```
curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}'
{
  "message": "Authorization header required",
  "errors": [
    {
      "name": "base",
      "reason": "Authorization header required"
    }
  ],
  "uuid": "57f78cd0-4559-464f-9df7-36c9ef7c89b3"
}
```
```
curl -k -X PATCH -H "Authorization: Bearer $TEST_TOKEN" "https://localhost:8080/api/latest/fleet/users/14" -d '{"name": "Manuel2", "password": "what", "new_password": "p4ssw0rd.12345"}'
{
  "message": "Permission Denied",
  "uuid": "7f0220ad-6de7-4faf-8b6c-8d7ff9d2ca06"
}
```

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-03-13 13:44:06 -03:00
Roberto Dip
a1ca172c95
allow to set up a DEP flow gated by Okta auth (#10338)
#10271
2023-03-13 10:33:32 -03:00
Juan Fernandez
6ff2c449cf
Improve test coverage and bug fix (#10301)
Addresses https://github.com/fleetdm/fleet/issues/10112

Improved test coverage, also fixed a bug related to vulnerability processing for mac office apps.
2023-03-06 15:07:27 -04:00
Benjamin Edwards
dc210d594b
missed config translation (#10294)
missed setting the new rotation configs when wiring up all the
dependencies in fleet serve
2023-03-03 15:28:58 -05:00
Zach Wasserman
ca2e30e59c
Fix error writing coverage when running tests (#10278)
Intended to fix this error we are seeing in CI:

```
error generating coverage report: write |1: file already closed
```

It seems like perhaps a change in the way the test coverage is reported
in a recent Go version has interacted with the closing of stdout in
these tests.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Added/updated tests
2023-03-02 17:11:15 -08:00
Martin Angers
4593c49ec4
Add disk_encryption option to config and team YAML (#10185) 2023-02-28 15:34:46 -05:00
Martin Angers
e3ddb5f3ce
Support matching a host in orbit enrollment using the serial number (#9612) 2023-02-28 12:55:04 -05:00
gillespi314
6fec539fbf
Update API responses for hosts and labels endpoints to include host mdm info (#10141)
Issue #10126 

- Add mdm solution name to host mdm inf
- Add host mdm info in labels API response;
2023-02-27 18:40:34 -03:00
Juan Fernandez
7e366272c0
Feature 9386: Parse the Mac Office release notes for vulnerability processing (#9993)
This PR adds the capability of parsing the release notes posted in https://learn.microsoft.com/en-us/officeupdates/release-notes-office-for-mac into a JSON metadata file (to be released in the NVD repo) and use it for detecting vulnerabilities on Mac Office apps.
2023-02-24 14:18:25 -04:00
Benjamin Edwards
f3b0e4da72
add configuration parameters for filesystem logging file rotation (#10048) 2023-02-24 07:44:56 -05:00
Benjamin Edwards
6f8abe3bc3
refactor vuln process command (#10088)
propagate error and properly defer store unlock
2023-02-23 23:22:36 -05:00
Benjamin Edwards
1158e62e44
use the same lock id between lock and unlock (#10039)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-02-22 17:48:46 -05:00
gillespi314
e31fc889f1
Add MDM profiles to host detail in API responses (#10034)
Issue #9599 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-02-22 16:26:06 -06:00
Roberto Dip
8284274c3b
incomplete implementation of device wipe and lock (#9947) 2023-02-22 17:11:44 -03:00
Roberto Dip
262e9870e3
add a cron job to reconcile profiles (#9946)
https://github.com/fleetdm/fleet/issues/9590
2023-02-22 14:49:06 -03:00
Lucas Manuel Rodriguez
407d05eab9
Workaround to set policy specs on a team (#9978)
For the CIS benchmark feature, we need a way to import a group of
policies (spec yml) into a team.
This PR adds a flag to `apply -f` to allow setting a team name to a
group of policies.

Sample:
```sh
fleetctl apply --context dogfood --policies-team "📊 CIS Benchmarks" -f ee/cis/macos-13/cis-policy-queries.yml
```

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [ ] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-02-22 13:14:53 -03:00
Roberto Dip
f6b86a55d9
Use dynamic serial numbers in osquery-perf (#9904)
This helps with MDM local testing as serial numbers play a big role.
2023-02-17 17:10:49 -03:00
Benjamin Edwards
0db432f7e3
vuln processing distinct command (#9813)
closes https://github.com/fleetdm/fleet/issues/3723

Add new vuln processing command, configs, and documentation on how to utilize.
2023-02-17 10:00:57 -05:00
Martin Angers
33f33163a9
Add macos custom profiles support via fleetctl apply (#9824) 2023-02-15 13:01:44 -05:00
Roberto Dip
7cd581866a
add API endpoint to see disk encryption key (#9713)
https://github.com/fleetdm/fleet/issues/8708
2023-02-08 20:20:23 -03:00
Roberto Dip
046401d190
Ingest file vault recovery keys in macOS (#9712)
Related + details at https://github.com/fleetdm/fleet/issues/8708
2023-02-08 11:49:42 -03:00
Lucas Manuel Rodriguez
dfe6b53963
Make fleetctl get teams --yaml output compatible with fleetctl apply -f (#9626)
#9535

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-02-02 14:46:50 -03:00
Roberto Dip
e06b00df11
Add readonly MDM.EnabledAndConfigured to app config and device responses (#9575)
Related to #9571, this adds a new value to both responses which is
calculated when the Fleet server is started, and only set to `true` if
the server is properly configured for MDM.

This helps the UI to determine wether or not we should show certain UI
elements that we only want to show to servers with MDM enabled.
2023-02-01 14:47:52 -03:00
Roberto Dip
ffed7f8ebe
return 422 status code if fleetdm.com returns any 4xx status for CSR (#9610)
Related to https://github.com/fleetdm/fleet/issues/9588, we now handle 4xx responses from the fleetdm.com server and forward those to the client.

At the time of this commit, the only 4xx response that wasn't already handled by the server is because of an invalid email domain, so we assume that, but we should look into establishing a pattern of error messages with the website instead.
2023-02-01 12:50:22 -03:00
Roberto Dip
4c4c114e96
add mocks + tests and move things around (#9574)
#8948

- Add more go:generate commands for MDM mocks
- Add unit and integration tests for MDM code
- Move interfaces from their PoC location to match existing patterns
2023-01-31 11:46:01 -03:00
Martin Angers
8a137e2b5b
Move host details mdm properties to new mdm object (#9505) 2023-01-30 16:40:11 -06:00
Martin Angers
d0e6891d10
Add endpoint to trigger CSR request for APNs on fleetdm.com (#9494) 2023-01-25 14:44:29 -05:00
Martin Angers
84717de461
Document APNs and ABM setup and renewal (#9471) 2023-01-24 15:20:41 -05:00
Roberto Dip
2d25a3f48d
add mdm root key and macos_updates to app and team configs (#9442)
Related to https://github.com/fleetdm/fleet/issues/9345,
https://github.com/fleetdm/fleet/issues/9358 and
https://github.com/fleetdm/fleet/issues/9346 this adds:

1. The ability to configure `mdm.macos_updates` via `PATCH /config` and
`PATCH /teams/{id}`
3. The ability to configure `mdm.macos_updates` by using `fleetctl apply
-f` for teams and global config.
2023-01-24 13:20:02 -03:00
Gabriel Hernandez
7d4653baaa
add attribute to GET /activities endpoint with pagination metadata (#9279)
relates to https://github.com/fleetdm/fleet/issues/8928

This adds a new `meta` attribute to the "GET /activities" endpoint that
includes pagination metadata. This can allow clients to know if there
are additional items to request.


- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-01-18 12:57:11 +00:00
gillespi314
fba3607c4a
Add mdm status and server url to host endpoint responses (#9320) 2023-01-16 17:08:24 -06:00
Roberto Dip
2447a371b0
use nanomdm's multi service + integration tests and fixes (#9269)
**Use nano's multi service**

This allows us to integrate more seamlessly with nano and to run our
custom MDM logic _after_ the request was handled by nano, which gives us
more flexibility (for example: now we can issue commands after a
TokenUpdate message)

From nano's code:

> MultiService executes multiple services for the same service calls.
> The first service returns values or errors to the caller. We give the
> first service a chance to alter any 'core' request data (say, the
> Enrollment ID) by waiting for it to finish then we run the remaining
> services' calls in parallel.

**Integration tests + fixes**

- Move some of the service logic from `cmd/` to `server/service`
- Add integration tests for the MDM enrollment flow, including SCEP
authentication.
- Fixed a bug that set `host_mdm.mdm_id = 0`  during MDM enrollment due
  to how MySQL reports the last insert id when `ON DUPLICATE KEY` is
  used.
- Completely remove the host row from `host_mdm` when a device is
  unenrolled from MDM to match the behavior of how we ingest MDM data
  from osquery

Related to https://github.com/fleetdm/fleet/issues/8708,
https://github.com/fleetdm/fleet/issues/9034
2023-01-16 17:06:30 -03:00
Tomas Touceda
398bb38da7
Add software_updated_at field to host (#9116)
#9012

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Added/updated tests

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2023-01-09 08:55:43 -03:00
Martin Angers
656e5bfc70
Flag when the Apple BM terms have expired (#9091)
#8862 

Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
2023-01-06 17:44:20 -03:00
Lucas Manuel Rodriguez
ba6b6e12cc
Fix ingestion of MDM server for macOS and Windows hosts (#9133)
* Fix ingestion of MDM server for macOS and Windows hosts

* Return early if there are no mobile_device_management_solutions to clean up

* Run make dump-test-schema

* Fix TestHosts/DeleteHosts test

* Add test for host unenrolling from MDM

* Fix lint

* Fix typo

* Identify Fleet MDM

* Move logic to deduce MDM name out of Datastore

* Run make dump-test-schema

* Remove printfs from migration
2023-01-04 10:29:48 -03:00
Frank Sievertsen
91c90b4caa
Add new configuration option to set default team for Apple Business Manager (#9062) 2023-01-03 16:14:18 -06:00
Lucas Manuel Rodriguez
ac22aadc13
Fleet server and tooling to use NETWORK_TEST_GITHUB_TOKEN when environment variable is set. (#9143)
* WIP

* Add more logging

* Check rate limit at end of action

* Add github client in more places

* Add new published firefox 93 vulnerabilities to tests

* Remove fmt printfs

* Restore CI check settings

* Readd newline
2023-01-03 14:56:11 -03:00
gillespi314
b990121fdc
Add mdm enrollment pending filter to API endpoints (#9137) 2023-01-03 11:26:23 -06:00
Roberto Dip
1b47f9e700
add activities when a host is enrolled/unenrolled from MDM (#9127)
#8996
2022-12-28 16:41:18 -03:00
gillespi314
94dd1c3745
Ingest pending MDM hosts (#9065)
Co-authored-by @roperzh
2022-12-26 15:32:39 -06:00
Lucas Manuel Rodriguez
e1bbcfcfda
Generate audit logs for activities (#9001)
* Generate audit logs for activities

* Fix config tests

* Fix TestGetConfig/IncludeServerConfig

* Fix use of AddAttributes in results only

* Stream activities asynchronously

* Fix index and add logging

* Revert change

* Documentation fixes
2022-12-23 19:04:13 -03:00
Lucas Manuel Rodriguez
39f1029390
Make activities typed and auto-generate docs for each type (#9069)
* Make activities type and auto-generate docs for each type

* Add pageOrderInSection to not break site

* Add do not edit note to generated file

* Add make generate-doc step

* Fix main merge
2022-12-23 13:05:16 -03:00
Frank Sievertsen
8e65ec67b5
Add new activity that records create/edit/delete user roles (#9086) 2022-12-21 18:30:19 +01:00
Tomas Touceda
68bd8661e9
Simplify disk encryption query in linux and filter at ingestion (#9037)
* Simplify disk encryption query in linux and filter at ingestion

* Join with mounts to detect whatever is encrypting /
2022-12-19 10:01:59 -03:00
Frank Sievertsen
7ebb97cd70
Fire automations for hosts that failed before automation enabled (#9028) 2022-12-16 22:00:54 +01:00
Michal Nicpon
37f454c0be
Add dep key pair endpoint (#9007) 2022-12-16 19:39:36 +01:00
Stephan Miehe
b6a409fa3b
Support redis username (#9018) 2022-12-14 08:53:38 -05:00
Michal Nicpon
3c9dd27d75
add fleectl generate mdm-apple-bm command (#8979) 2022-12-13 16:24:50 +01:00
Martin Angers
f18a320bd5
Add command get mdm-apple-bm to fleetctl (#8949) 2022-12-12 15:45:53 -05:00
Roberto Dip
e68535d468
report errors that can occur during file carving (#8972)
related to https://github.com/fleetdm/fleet/issues/8117
2022-12-09 13:21:30 -03:00
Michal Nicpon
10b3179b63
Add fleetctl generate mdm-apple (#8812) 2022-12-07 18:24:42 +01:00
Martin Angers
d4a3730610
Add new configuration options for Apple Business Manager (#8886) 2022-12-06 15:50:56 -05:00
gillespi314
43b1241a0c
Add fleetctl trigger command (#8877) 2022-12-06 09:56:54 -06:00
Frank Sievertsen
e0766d21a3
Add ability to mark policy as critical. (#8896) 2022-12-06 15:59:20 +01:00
gillespi314
6fb3a87ae9
Enable errcheck linter for golangci-lint (#8899) 2022-12-05 16:50:49 -06:00
Martin Angers
866829dbfa
Implement fleetctl get mdm-apple (#8786) 2022-12-05 11:35:45 -05:00
Martin Angers
f5fc316ad3
Add options for configuring APNS and SCEP keys/certs in Fleet server (#8730) 2022-12-05 10:22:56 -05:00
gillespi314
d5c096fa02
Implement schedule triggers (#8747) 2022-11-28 13:28:06 -06:00
gillespi314
267aaf0dbe
Add holdLock and releaseLock methods to schedule package (#8464) 2022-11-16 15:14:38 -06:00
Martin Angers
7aab706175
Document the osquery-perf os_templates flag (#8745) 2022-11-16 16:12:24 -05:00
Martin Angers
ca13659fe0
Fix broken test (#8715) 2022-11-15 09:52:47 -05:00
Martin Angers
472c8bafb3
Refactor license so it is stored in the context (#8544) 2022-11-15 09:08:05 -05:00
Martin Angers
f3c7a37813
Support Windows and Ubuntu in osquery-perf (#8616) 2022-11-15 08:24:40 -05:00
Martin Angers
54bb5448a0
Fix missing valid properties in validation of FIM and YARA agent options (#8707) 2022-11-14 15:50:41 -05:00
Michal Nicpon
9ad1721efd
fix issue with duplicate vulns detected using nvd (#8613)
The OVAL analyzer falsely assumes that any vulnerabilities detected on a
host only come from OVAL. However, it is possible that NVD detects
vulnerabilities on these hosts even though it excludes software from
deb_packages and rpm_packages. For example, a python package twisted
v22.20 has a vulnerability CVE-2022-39348 detected by NVD. The OVAL
analyzer would delete this vulnerability, and it would be re-inserted by
the NVD scanner on the next run. This creates a loop.

The fix is to only delete vulnerabilities that are actually detected
using OVAL. We already store this in the source column in the
software_cve table.
2022-11-10 10:28:00 -07:00
Martin Angers
7fbe394b94
Ingest disk encryption flag for hosts, return in GET /hosts/{id} endpoint (#8526) 2022-11-02 15:44:02 -04:00
Martin Angers
8f21e026e3
Fix bug with fleetctl apply for teams, clear agent options only if key is present (#8508) 2022-11-01 15:22:45 -04:00
Roberto Dip
c51927e873
enhance support for orbit and fleet desktop in osquery-perf (#8217)
This improves osquery-perf with support for a more realistic orbit + fleet desktop simulation as described in #8212

This was based on the work done by @sharvilshah in his branch.
2022-10-28 14:27:21 -03:00
Juan Fernandez
53e112d264
Feature 7494: Use the MSRC security bulletin artifacts for detecting Win OS vulnerabilities (#7889)
Use the MSRC security bulletin artifacts for detecting Win OS vulnerabilities
2022-10-28 11:12:21 -04:00
Martin Angers
49e910270b
Add vulnerability scores to Jira/Zendesk tickets for premium users (#8346) 2022-10-26 10:42:09 -04:00
gillespi314
51e6369653
Fix password reset request expiration (#8246) 2022-10-25 09:46:41 -05:00
gillespi314
881194b3b4
Add orbit version and osquery version usage statistics (#8229) 2022-10-24 11:12:56 -05:00
Martin Angers
e19191161e
Validate team spec unknown keys (#8265) 2022-10-24 08:49:44 -04:00
Roberto Dip
d0cde1aaaa
Use the same HTTP server config as the production server in tests (#8254)
* Use the same HTTP server config as the production server in tests

This abstracts the default config we use to run the server into a
function so it can be used in tests to run an HTTP server using the same
configuration.

Additionally, this fixes a data race in tests, as an HTTP server
configuration can't be changed once you call `server.Start()`[1]

[1]:
https://cs.opensource.google/go/go/+/refs/tags/go1.19.2:src/net/http/httptest/server.go;l=40;drc=19309779ac5e2f5a2fd3cbb34421dafb2855ac21
2022-10-19 07:42:21 -03:00
Martin Angers
323620bf39
Deep-clone the appconfig when getting from cache (#8194) 2022-10-17 15:03:49 -04:00
Frank Sievertsen
a2c47cf4f4
add host_display_name to live query rows (#8251) 2022-10-17 15:52:25 +02:00
gillespi314
59876d37ea
Add usage statistics to measure policy violations (#8199) 2022-10-14 13:55:37 -05:00
Frank Sievertsen
e54f066266
Add display_name to more results (#8204)
* Make HostResponses with one dedicated function only
* more display_name for Hosts
2022-10-14 11:14:18 -03:00
Martin Angers
fae8e4ca2c
Translate the AppConfig and Team validation messages to be more user-friendly (#8171) 2022-10-12 17:10:50 -04:00
Juan Fernandez
91ea7bf244
Vulnerabilities not detected if NVD CVE feed is unavailable or rate limited (#8191)
If there is any problem when synching NVD source, proceed with vuln scan
2022-10-12 15:09:18 -04:00
Frank Sievertsen
23199c3d83
Add missing display_name to getHostEndpoint (#8174) 2022-10-11 17:00:32 -04:00
Roberto Dip
e630fabf89
token rotation for fleet desktop (#7779)
This implements what's described in detail here https://github.com/fleetdm/fleet/blob/main/proposals/fleet-desktop-token-rotation.md
2022-10-10 17:15:35 -03:00
Frank Sievertsen
e9f7066d87
7135 host display name (#7873) 2022-10-08 08:57:46 -04:00
Lucas Manuel Rodriguez
9191f4ce66
Add Apple MDM functionality (#7940)
* WIP

* Adding DEP functionality to Fleet

* Better organize additional MDM code

* Add cmdr.py and amend API paths

* Fix lint

* Add demo file

* Fix demo.md

* go mod tidy

* Add munki setup to Fleet

* Add diagram to demo.md

* Add fixes

* Update TODOs and demo.md

* Fix cmdr.py and add TODO

* Add endpoints to demo.md

* Add more Munki PoC/demo stuff

* WIP

* Remove proposals from PoC

* Replace prepare commands with fleetctl commands

* Update demo.md with current state

* Remove config field

* Amend demo

* Remove Munki setup from MVP-Dogfood

* Update demo.md

* Add apple mdm commands (#7769)

* fleetctl enqueue mdm command

* fix deps

* Fix build

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>

* Add command to upload installers

* go mod tidy

* fix subcommands help

There is a bug in urfave/cli where help text is not generated properly when subcommands
are nested too deep.

* Add support for installing apps

* Add a way to list enrolled devices

* Add dep listing

* Rearrange endpoints

* Move DEP routine to schedule

* Define paths globally

* Add a way to list enrollments and installers

* Parse device-ids as comma-separated string

* Remove unused types

* Add simple commands and nest under enqueue-command

* Fix simple commands

* Add help to enqueue-command

* merge apple_mdm database

* Fix commands

* update nanomdm

* Split nanomdm and nanodep schemas

* Set 512 MB in memory for upload

* Remove empty file

* Amend profile

* Add sample commands

* Add delete installers and fix bug in DEP profile assigning

* Add dogfood.md deployment guide

* Update schema.sql

* Dump schema with MySQL 5

* Set default value for authenticate_at

* add tokens to enrollment profiles

When a device downloads an MDM enrollment profile, verify the token passed
as a query parameter. This ensures untrusted devices don't enroll with
our MDM server.

- Rename enrollments to enrollment profiles. Enrollments is used by nano
  to refer to devices that are enrolled with MDM
- Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles
- Generate a token for authentication when creating an enrollment profile
- Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token=

* remove mdm apple server url

* update docs

* make dump-test-schema

* Update nanomdm with missing prefix table

* Add docs and simplify changes

* Add changes file

* Add method docs

* Fix compile and revert prepare.go changes

* Revert migration status check change

* Amend comments

* Add more docs

* Clarify storage of installers

* Remove TODO

* Remove unused

* update dogfood.md

* remove cmdr.py

* Add authorization tests

* Add TODO comment

* use kitlog for nano logging

* Add yaml tags

* Remove unused flag

* Remove changes file

* Only run DEP routine if MDM is enabled

* Add docs to all new exported types

* Add docs

* more nano logging changes

* Fix unintentional removal

* more nano logging changes

* Fix compile test

* Use string for configs and fix config test

* Add docs and amend changes

* revert changes to basicAuthHandler

* remove exported BasicAuthHandler

* rename rego authz type

* Add more information to dep list

* add db tag

* update deps

* Fix schema

* Remove unimplemented

Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com>
Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 19:53:54 -03:00
Juan Fernandez
812d3c85de
Fixes various bugs with NVD vulnerability detection (#7963)
- Improved NVD CPE matching process.
- Fixed bug with the 'software/<id>' endpoint not showing the generated_cpe value.
2022-10-04 07:04:48 -04:00
Martin Angers
8eee0b35f3
Add validations for agent options' command-line flags section. (#7979) 2022-10-03 08:29:41 -04:00
Roberto Dip
3a349bb07b
fix data race in TestAutomationsSchedule (#7952) 2022-09-26 17:19:38 -03:00
Martin Angers
1fd6844bb0
Make osquery-perf aware of kubequery so that hosts are not reported as such erroneously (#7949) 2022-09-26 15:39:39 -04:00
gillespi314
639b85e47b
Refactor cron worker to schedule package (#7886) 2022-09-23 11:45:48 -05:00
Martin Angers
db24cf32fa
Add new low_disk_space URL parameter to the GET /hosts endpoint (and GET /hosts/count) (#7853) 2022-09-21 15:16:31 -04:00