Commit Graph

2159 Commits

Author SHA1 Message Date
Roberto Dip
d261762e83
add base types and handlers for DDM (#17657)
This includes the base types and the request handlers to reduce the
chances of conflicts.
2024-03-15 10:51:47 -03:00
Roberto Dip
dc87ac2271
add schema for DDM (#17636)
#17405
2024-03-14 18:08:19 -03:00
Lucas Manuel Rodriguez
cf64d85deb
Add visual studio extensions to software inventory (#17501)
#17003

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-03-14 16:33:12 -03:00
Sarah Gillespie
c2a7c670fa
Handle null case in datastore method to get host disk encryption status (#17541) 2024-03-14 10:01:20 -05:00
Jahziel Villasana-Espinoza
2b9487197c
feat: script content cleanup and column removal (#17374)
> Related issue: #17374

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Manual QA for all new/changed functionality
2024-03-13 15:53:08 -04:00
Tim Lee
8be1d4766f
MS Edge Translation (#17171) 2024-03-13 12:47:31 -06:00
Martin Angers
0f8192348b
Add missing index to MDM Windows enrollments table (#17559) 2024-03-13 11:27:37 -04:00
Martin Angers
2522cc5ef6
Recategorize MDM endpoints to new mdm-less paths (#17372) 2024-03-13 10:27:29 -04:00
Martin Angers
c358bde87b
osquery-perf: add support for Windows MDM enrollment and session management. (#17522) 2024-03-13 09:29:25 -04:00
Victor Lyuboslavsky
ad5c0a90be
Team user should not access OS version on another team. (#17347)
#17117 
For `fleet/os_versions` and `/fleet/os_versions/[id]`, team users can no
longer access os versions on hosts from other teams.

### Team admin /os_versions - only returns os versions for the user's
team(s)
GET https://localhost:8080/api/v1/fleet/os_versions

### Team admin /os_versions/:id on 'No Team' - 403
GET https://localhost:8080/api/v1/fleet/os_versions/5

### Global admin /os_versions/:id?team_id does not exist anywhere - 404
GET https://localhost:8080/api/v1/fleet/os_versions/999999?team_id=1

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-13 08:20:00 -05:00
Martin Angers
8d8181eb0d
Fix flaky test due to package-level test variable set in a separate test (#17393) 2024-03-13 09:12:19 -04:00
Victor Lyuboslavsky
1052b6b350
Reducing the number of deadlock errors. (#17290)
#16562

Based on local testing, these changes reduce the number of 'Deadlock
found' errors when multiple hosts share the same UUID.

The performance issues with vulnerability processing will be fixed as
part of issue https://github.com/fleetdm/fleet/issues/16858

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-13 06:58:29 -05:00
Lucas Manuel Rodriguez
e9a464e0cf
Add exponential backoff to orbit enroll retries (#17368)
#16594

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-03-13 07:57:00 -03:00
Jahziel Villasana-Espinoza
9e147494a3
fix: process Add commands in Windows MDM profiles (#17570)
> Related issue: #17563

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2024-03-12 15:21:17 -04:00
Tim Lee
c4750d61f5
bugfix: add counts_updated_at (#17543) 2024-03-11 14:14:29 -06:00
Tim Lee
d4ba30cfa1
17257 validate bulk ops request filters (#17514) 2024-03-11 10:02:51 -06:00
Victor Lyuboslavsky
b174a476a2
Fixing unreleased spec bug in team host status webhook feature. (#17502)
Fixing unreleased spec bug in team host status webhook feature #17094.
Bug #17498
# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
  - Not needed. Part of new feature.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-08 15:09:33 -06:00
Roberto Dip
525233efea
fix unreleased bug with ADE profile assignment (#17509)
for unreleased #17508
2024-03-08 16:45:18 -03:00
Jacob Shandling
2584f6794a
Add integration test for change password endpoint (#17319)
## Addresses #16863 

- [x] Added/updated tests

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-03-08 08:28:00 -08:00
Roberto Dip
a39932b013
ammendment to properly fix #17489 (#17499)
for #17489, found an edge case during extra testing.
2024-03-08 13:15:04 -03:00
Lucas Manuel Rodriguez
37885d0932
Remove filtering of private IPs in public IP field (#17352)
#11102

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-08 12:57:18 -03:00
Roberto Dip
1be3aa2ffb
improve MDM detection with enrollment state checks (#17494)
for #17489
2024-03-08 12:21:07 -03:00
Tim Lee
2e67ef61d4
Bugfix: Revert host filters fix (#17390) (#17464)
This reverts commit 4b2ebdc8dc.
2024-03-07 12:57:27 -07:00
Victor Lyuboslavsky
a173be8f52
For fleetctl gitops, when MDM configs are not explicitly defined in gitops yml file, they are now set to default values. (#17223)
For fleetctl gitops, when MDM configs are not explicitly defined in
gitops yml file, they are now set to default values.
#17209

Gitops role can now read org config/settings. This is used to determine
whether license is Premium.
Doc changes for permission access:
https://github.com/fleetdm/fleet/pull/17238

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-07 13:20:14 -06:00
Roberto Dip
e8b7ce81dc
don't run SCEP test in parallel to avoid race in dependency (#17457)
The https://pkg.go.dev/go.mozilla.org/pkcs7 package uses an unguarded
package-level variable:


33d05740a3/ber.go (L19-L21)

This was causing this data race during tests:

```
==================
WARNING: DATA RACE
Read at 0x0000009b89b0 by goroutine 22:
  go.mozilla.org/pkcs7.asn1Structured.EncodeTo()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/ber.go:21 +0x56
  go.mozilla.org/pkcs7.(*asn1Structured).EncodeTo()
      <autogenerated>:1 +0xa5
  go.mozilla.org/pkcs7.ber2der()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/ber.go:68 +0xdb
  go.mozilla.org/pkcs7.Parse()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/pkcs7.go:160 +0x12f
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep.ParsePKIMessage()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep.go:234 +0x1d2
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.testParsePKIMessage()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:21 +0x56
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.TestNewCSRRequest.func1()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:198 +0x7e5
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x44

Previous write at 0x0000009b89b0 by goroutine 23:
  go.mozilla.org/pkcs7.asn1Structured.EncodeTo()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/ber.go:21 +0x6e
  go.mozilla.org/pkcs7.(*asn1Structured).EncodeTo()
      <autogenerated>:1 +0xa5
  go.mozilla.org/pkcs7.ber2der()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/ber.go:68 +0xdb
  go.mozilla.org/pkcs7.Parse()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/pkcs7.go:160 +0x12f
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep.(*PKIMessage).DecryptPKIEnvelope()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep.go:344 +0xde
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.TestNewCSRRequest.func1()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:199 +0x7fa
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x44

Goroutine 22 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x845
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.TestNewCSRRequest()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:165 +0x3ac
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x44

Goroutine 23 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x845
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.TestNewCSRRequest()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:165 +0x3ac
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x44
==================
    testing.go:1465: race detected during execution of test
=== NAME  TestNewCSRRequest/KeyEncipherment_not_set_with_NOP_certificates_selector
```
2024-03-07 12:01:52 -03:00
Jacob Shandling
e957292746
Update reference in mdm integration test (#17452)
Add one missed update

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-03-06 23:01:21 -03:00
Roberto Dip
9c5b59df0d
prevent redundant ADE profile assignment (#17427)
For #17291, this prevent re-assigning profiles to ABM hosts that already
have the right one.

This was happening very frequently for hosts that are in the last page
of the `/sync` request, as there's no indication that the cursor was
exhausted and we keept on assigning profiles to those hosts.

This caused profile assignment to eventually fail, presumably due to
rate limiting.
2024-03-06 18:16:10 -03:00
Tim Lee
4b2ebdc8dc
17257 host filters fix (#17390) 2024-03-06 14:15:53 -07:00
Jahziel Villasana-Espinoza
babf4e17c8
fix: return bad request error during MDM migration when appropriate (#16551)
> Related issue: https://github.com/fleetdm/confidential/issues/5138

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-06 15:38:44 -05:00
Jahziel Villasana-Espinoza
c808bba438
fix: don't show windows hosts when filtering by bootstrap package status (#17386)
> Related issue: #17341

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-06 15:07:48 -05:00
Sarah Gillespie
ba211437c6
Update integration tests for run script by name feature (#17381) 2024-03-05 15:12:52 -06:00
Martin Angers
0bb6d45596
Merge branch 'main' into feat-scripts-char-limit 2024-03-05 10:10:04 -05:00
Sarah Gillespie
c29f0abf92
Update API and CLI to enable running scripts by name and team id (#17322)
TODO:
- Integration tests

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-03-05 08:53:17 -06:00
Martin Angers
16ba308039 Store script contents for wipe host via script 2024-03-05 08:51:57 -05:00
Roberto Dip
7a7b043014
allow gitops role to read hosts by identifier and profiles (#16688)
for #15337
2024-03-05 10:43:49 -03:00
Martin Angers
88eeb9c1a4 Reorder database migrations 2024-03-05 08:22:11 -05:00
Martin Angers
98bc2dc29a Fix conflicts 2024-03-05 08:11:28 -05:00
Victor Lyuboslavsky
7bbb4a64bb
Reduced vulnerability processing time by grouping vulnerability dictionary by vendor. (#17298)
Reduced vulnerability processing time by grouping vulnerability
dictionary by vendor.
#16858 

Notice `cvefeed.(*Cache).Get`
Before:
<img width="1997" alt="image"
src="https://github.com/fleetdm/fleet/assets/2685025/a3cc5571-cee2-40a6-8b2d-cfed1323d96c">
After:

![image](https://github.com/fleetdm/fleet/assets/2685025/89484f87-a4e9-4c5c-943f-138e1c27a2c6)


# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-04 16:16:32 -06:00
Victor Lyuboslavsky
05ff1ca52a
Adding backend support for running live queries with team_id=0 (No team) (#17258)
- API endpoint GET fleet/targets/count can target 'No team' with
team_id=0
- API endpoint POST fleet/queries/run (for async live queries) can
target 'No team' with team_id=0
#16350

API doc changes PR: https://github.com/fleetdm/fleet/pull/17267

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-04 16:08:10 -06:00
RachelElysia
316321e889
[released bug] Fleet Full stack: Allow teams filter when transferring hosts by filters (#17241) 2024-03-04 15:48:19 -05:00
Roberto Dip
e3712fb263
Don't run cleanup tasks or enqueue commands on SCEP renewals (#17303)
for #17232
2024-03-04 16:53:16 -03:00
Martin Angers
0858f5a6f4
Scripts char limit: improve database migration performance and memory usage (#17338) 2024-03-04 13:51:32 -05:00
Victor Lyuboslavsky
592a7450e3
Enabling setting host status webhook at the team level via REST API and fleetctl apply/gitops. (#17186)
Enabling setting host status webhook at the team level via REST API and
fleetctl apply/gitops.
#14916

Example payload:
```json
{
    "data": {
        "days_unseen": 3,
        "host_ids": [
            10724,
            10726,
            10738,
            10739,
            10740,
            10741,
            10742,
            10744,
            10745,
            10746,
            10747,
            10748,
            10749
        ],
        "team_id": 3,
        "total_hosts": 15,
        "unseen_hosts": 13
    },
    "text": "More than 86.67% of your hosts have not checked into Fleet for more than 3 days. You've been sent this message because the Host status webhook is enabled in your Fleet instance."
}
```

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-04 12:35:27 -06:00
Lucas Manuel Rodriguez
4acb713bf1
osquery-perf changes for Fleet downtime load test (#17310)
Including all the osquery-perf changes needed to perform #16423.
2024-03-04 15:10:10 -03:00
Jahziel Villasana-Espinoza
33a0324ebb
feat: adding, reading, and deleting flows for scripts updated to use new table (#17305)
> Related issues: #16842, #16843

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Manual QA for all new/changed functionality
2024-03-04 11:00:08 -05:00
Jahziel Villasana-Espinoza
f3754e2674
fix: use correct copy in fleetctl error when applying profiles (#17318)
> Related issue: #16335, fixes issue QA found here:
https://github.com/fleetdm/fleet/issues/16335#issuecomment-1971560894

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-01 18:22:30 -05:00
Roberto Dip
730f8850ff
Add mdm wipe host feature (#17272)
relates to #9951

This adds the mdm host wipe feature to fleet. This includes:

1. adding wipe functionality in the fleet web UI
2. adding wipe functionality in the fleetctl CLI
3. adding API endpoints to wipe a host
4. Implementing wipe functionality on the fleet server.
2024-03-01 18:02:00 -03:00
Roberto Dip
152edb838b
fix flaky MDM test (#17307)
this one seems to be specially flaky and is blocking a couple of PRs
from being merged.
2024-03-01 17:54:42 -03:00
Gabriel Hernandez
e495144023 update schema file 2024-03-01 17:38:08 +00:00
Gabriel Hernandez
306133ae3b udpate migration timestamp for add fleet platform to host mdm action migration 2024-03-01 17:34:05 +00:00