Commit Graph

632 Commits

Author SHA1 Message Date
Sarah Gillespie
0e468b4981
Update software titles cron to include browser field (#15491) 2023-12-07 17:43:37 -06:00
Martin Angers
8b7d1a7e47
Remove Windows MDM feature flag (#15167)
For #14959 

---------

Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
2023-12-07 14:24:32 -03:00
Martin Angers
e3d225ade7
Update fleetctl get software to list titles and versions. (#15444) 2023-12-06 16:07:03 -05:00
Sarah Gillespie
f19dc8abe0
Add GET software/versions and GET software/versions/:id endpoints (#15450) 2023-12-06 08:30:49 -06:00
Sarah Gillespie
0db59a7a25
Add cron for software titles feature (#15422) 2023-12-04 10:09:23 -06:00
Martin Angers
f5f66ebe63
Unify profiles database identifier to profile_uuid for macOS and Windows profiles. (#15297) 2023-12-04 10:04:06 -05:00
Victor Lyuboslavsky
a7ae828581
Adding extension_id and browser columns to software table. (#15285)
https://github.com/fleetdm/confidential/issues/2118
API doc changes are at: https://github.com/fleetdm/fleet/pull/15092

This change affects Chrome-based and Firefox browsers. I tried Safari,
but it appears that latest Safari extensions simply show up as regular
apps on macOS.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - Tested detail query changes on macOS, Windows, Ubuntu, and ChromeOS
  - Tested Chrome, Edge, Brave, and Firefox browsers

Testing notes:
1. Install some browser extensions.
2. Update fleet to this change.
3. Refresh hosts, and trigger vulnerabilities job (which updates host
software).
4. Do something like `GET
https://localhost:8080/api/v1/fleet/software?query=Password%20Manager`
to see `extension_id` and `browser` for the hosts that have updated.
2023-11-30 19:06:17 -06:00
Roberto Dip
b07fbdc1d7
implement profile verification for windows (#15348)
for #14426.

In order to prevent import cycles and be able to use some type
definitions/constants I followed the same pattern we did for macOS by
creating a new package named `syncml`. This makes the changelog look
bigger than it actually is, so I split it into two commits to make it
easier to review:

-
[d7c233d](d7c233d54c)
moves the relevant bits to this new package
-
[7531a07](7531a0742b)
implements profile verification
2023-11-30 09:17:07 -03:00
Martin Angers
2f927df4f0
Implement Windows OS Updates (feature branch). (#15359) 2023-11-29 11:07:24 -05:00
Martin Angers
0b5eedb801
Fix edge case of AppConfig changes getting lost in cached mysql. (#15352) 2023-11-29 10:09:37 -05:00
Sarah Gillespie
1535b72305
Add hardware serial to host by identifier search (#15353) 2023-11-29 08:39:35 -06:00
Roberto Dip
f439432fae
feature: windows profiles (#15349) 2023-11-29 09:32:42 -05:00
Roberto Dip
9c7b740b03
disable windows profiles feature in the back-end (#15278)
per discussion in slack, reverting relevant user-facing changes for
#13281
2023-11-27 16:08:52 -03:00
Tim Lee
c5ea7eac4f
Cache policy counts (#15244) 2023-11-21 12:52:06 -07:00
Lucas Manuel Rodriguez
4194c44131
Use NVD API 2.0 to download CVE information (#15102)
#14888

@getvictor This is ready for review, but keeping as draft as there are
probably many tests that need amending.

I used the new version of the `./tools/nvd/nvdvuln/nvdvuln.go` to
compare the current vulnerabilities found in our dogfood environment
with the vulnerabilities found by the code in this PR and both results
match:
```
go run -race -tags fts5 ./tools/nvd/nvdvuln/nvdvuln.go --debug --db_dir ./local --software_from_url <dogfood URL> --software_from_api_token <API_TOKEN> --sync 2>&1 | tee out.txt
[...]
CVEs found and expected matched!
```

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: Victor Lyuboslavsky <victor@fleetdm.com>
Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
2023-11-21 12:30:07 -06:00
Victor Lyuboslavsky
eada583ff1
Updating CPE generator to use new NVD API. (#15018)
Loom explaining changes (hit 5 min limit):
https://www.loom.com/share/e59b63bf638e4d9cad7984ef589b878d?sid=111fff75-115a-4a44-ae4f-6f25fede0d51

#14887

- [x] Need to merge fleetdm/nvd PR
https://github.com/fleetdm/nvd/pull/25 before this one.

# Checklist for submitter

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- Manually tested (with corresponding fleetdm/fleet changes) in my
personal fork: https://github.com/getvictor/nvd/releases

# QA Plan (must be done before merging this PR, and after merging the
nvd PR)
- [ ] Fork https://github.com/fleetdm/nvd and point `generate.yml` to
this branch.
[example](9d8e54930b/.github/workflows/generate.yml (L26))
- [ ] Add NVD_API_KEY to nvd secrets, and run the the nvd generate
GitHub action. Get key:
https://nvd.nist.gov/developers/request-an-api-key
- [ ] Compare the generated `cpe-###.sqlite.gz` to the previous one. One
way is to open it up with sqlite3 and `select * from cpe_2 order by
cpe23;` and dump results to a CSV file. Known differences are:
   - New file has ~2,500 more records
- Backslashes are handled differently for `Backpack\CRUD` and `Philips
In.Sight B120\37` products -- not a new issue since we do not support
those products right now
- `cpe:2.3🅰️moodle:moodle:4.2.0:*:*:*:*:*:*:*` -- this appears OK.
Also, it is a PHP plugin, and we don't support these currently.
- [ ] Record the existing vulnerabilities of current hosts.
- [ ] Stop any running fleet server. Delete `/tmp/vulndbs/cpe.sqlite`.
Can also delete other files there, or not delete this file -- it should
be overwritten by the new file. Also delete all rows in software_cpe and
software_cve DB tables. (Or can just spin up a fresh fleet server with
fresh DB, and re-enroll hosts (after setting the new env variable
below))
- [ ] Find the path to the generated `cpe-###.sqlite.gz` file
- [ ] Set `FLEET_VULNERABILITIES_CPE_DATABASE_URL` environment variable
to the above path, and start fleet server.
- [ ] After server's vulnerabilities cron job runs, the new
vulnerabilities should match the previous vulnerabilities
2023-11-20 16:10:00 -06:00
Sarah Gillespie
57df2f250c
Add Windows MDM profiles to host details API response (#15210) 2023-11-20 14:34:57 -06:00
Roberto Dip
420dfe1cd0
batch set profiles as pending on profile-related actions (#15179)
final step of #14364
2023-11-20 11:16:02 -03:00
Sharon Katz
c67d9eb3a5
Hackathon add metrics from Fleet servers. (#14216)
Adding a few stats to the metrics Fleet Servers send to Heroku.
2023-11-16 10:25:19 -05:00
Jahziel Villasana-Espinoza
1fca8b1e38
fix: sort order for Last restarted (#14878)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/REST API/rest-api.md or
docs/Contributing/API-for-contributors.md)
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Rachael Shaw <r@rachael.wtf>
2023-11-15 16:42:57 -05:00
Roberto Dip
f82cfa307d
allow fleetctl to configure windows mdm profiles for teams and "no team" (#15161)
final step of #14361 , this integrates the work of the other PRs.

currently branched off `14361-windows-custom-settings-configs`

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2023-11-15 18:04:24 -03:00
Roberto Dip
319b64f368
allow to set mdm.windows_settings.custom_settings in configs (#15145)
for #14361 this adds the bits related to saving a slice of strings with
paths to configuration profiles.

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Marko Lisica <83164494+marko-lisica@users.noreply.github.com>
2023-11-15 13:58:46 -03:00
Roberto Dip
8478171256
add a cron job to deliver windows MDM profiles (#15065)
for #14364 , this implements the delivery mechanism for windows hosts.

I will follow up in another PR with logic to update the profile status
when we get responses from the device.
2023-11-10 11:05:10 -03:00
Martin Angers
b2ab4553cc
Validate that WSTEP is configured before enabling Windows MDM (#14858)
#14446 

~~Note that the fix requires a frontend change too, so this should not
be merged before the frontend is also ready.~~ Frontend
[PR](https://github.com/fleetdm/fleet/pull/14971) is ready.
2023-11-09 10:08:54 -03:00
Roberto Dip
33db665d63
show full formatted results for windows commands in fleetctl (#14922)
for #14912 this adds the full results to the "RESULTS" column of
`fleetctl get mdm-command-results`.

Additionally I included formatting of the XML output to improve
readability.
2023-11-03 12:01:43 -03:00
Roberto Dip
b04b20fcf0
fix fleetctl to set different CmdUID for Windows (#14895)
for #14891
2023-11-02 15:06:37 -03:00
Roberto Dip
9cf30a9131
Feat windows msmdm (#14837)
for #13069

---------

Co-authored-by: Marcos Oviedo <marcos@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
2023-11-01 11:13:12 -03:00
guangwu
33858d7301
chore: remove refs to deprecated io/ioutil (#14485)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-27 15:28:54 -03:00
Jahziel Villasana-Espinoza
c570ef1e21
feat: make analytics always true for premium customers (#14607)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-10-27 11:41:18 -04:00
Roberto Dip
5dfc72b2a1
allow to use --context in fleetctl mdm run-command (#14665)
for #13712
2023-10-26 19:09:31 -03:00
Lucas Manuel Rodriguez
fd99fd4121
Add optimizations for query reports at scale (#14710)
#7766

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.~
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~

---------

Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
2023-10-25 19:20:27 -03:00
Lucas Manuel Rodriguez
5d7ee58a87
Do not leak ticks and fix race condition (#14667)
This PR introduces the following fixes:
- Fixes a race condition by protecting the `scheduledQueries` and
`scheduledQueryData` fields with a `sync.Mutex` (*).
- Adds some more information about request counts for /log, /config and
/distributed/read requests (and uptime).
- Fixes the resource leaks around creating time.Ticks on every for loop
iteration.

(*) Sample of the race condition when running with `-race`:
```
==================
WARNING: DATA RACE
Read at 0x00c000604800 by goroutine 20:
  main.(*agent).runLoop()
      /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:525 +0x71b
  main.main.func2()
      /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:1737 +0x4e

Previous write at 0x00c000604800 by goroutine 40:
  main.(*agent).config()
      /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:915 +0xb30
  main.(*agent).runLoop.func2()
      /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:512 +0x37
```
2023-10-20 10:29:59 -03:00
Lucas Manuel Rodriguez
22bba274b7
Run logger, distributed and config in separate goroutines (#14584) 2023-10-17 11:30:59 -03:00
Lucas Manuel Rodriguez
8c6cefc4f4
Fix panic in osquery-perf (#14534)
This is safe to merge during freeze as this is an osquery-perf only
change.
```
panic: runtime error: index out of range [1] with length 1

goroutine 14 [running]:
main.(*agent).config(0xc0001e5b80)
        /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:847 +0xb74
main.(*agent).runLoop(0xc0001e5b80, 0x0?, 0x0?)
        /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:463 +0x4d2
created by main.main in goroutine 1
        /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:1666 +0x1285
```
2023-10-13 14:26:46 -03:00
Jahziel Villasana-Espinoza
f92695fda5
osquery-perf: can send scheduled query results (#14502) 2023-10-12 20:41:04 -06:00
Jacob Shandling
70e0182e4d Merge branch 'main' into 7766-main-merge 2023-10-11 12:56:52 -07:00
Roberto Dip
9ffa11c25d
Feat: saved scripts (#14409)
For #9537
2023-10-10 19:00:45 -03:00
Jacob Shandling
fb494ca3cb Merge branch '7766-frontend' into 7766-backend-frontend 2023-10-10 13:21:19 -07:00
Jahziel Villasana-Espinoza
0435431ae9
feat: can delete via cron job if global flag set (#14398)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added/updated tests

---------

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2023-10-10 10:06:54 -03:00
Tim Lee
fa55d72118
Save Query Results to DB (#14335)
# Checklist for submitter

#13486 

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Eric <eashaw@sailsjs.com>
Co-authored-by: Sampfluger88 <108141731+Sampfluger88@users.noreply.github.com>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Roberto Dip <me@roperzh.com>
Co-authored-by: Tyler Diderich <15862572+tdiderich@users.noreply.github.com>
Co-authored-by: Dave Herder <27025660+dherder@users.noreply.github.com>
Co-authored-by: Rachael Shaw <r@rachael.wtf>
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
Co-authored-by: Nathanael Holliday <100959072+hollidayn@users.noreply.github.com>
Co-authored-by: Katheryn Satterlee <me@ksatter.com>
Co-authored-by: Mo Zhu <mo@fleetdm.com>
Co-authored-by: Mo Zhu <mozhu888@gmail.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Robert Fairburn <8029478+rfairburn@users.noreply.github.com>
Co-authored-by: Sabrina Coy <13890648+sabrinabuckets@users.noreply.github.com>
Co-authored-by: Isabell Reedy <113355639+ireedy@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Alex Mitchell <105945793+alexmitchelliii@users.noreply.github.com>
2023-10-10 09:44:03 -03:00
Jahziel Villasana-Espinoza
5c868c9d3d
feat: deletion for query results (#14302)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
2023-10-09 17:43:17 -04:00
Jacob Shandling
eb327faabf
maintenance merge of main into feature branch (#14393)
maintenance merge of `main` into feature branch
2023-10-09 14:28:35 -07:00
Marcos Oviedo
f0d77ab3db
Merging Bitlocker feature branch (#14350)
This relates to #12577

---------

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2023-10-06 19:04:33 -03:00
Tim Lee
ccd6746633
7766 backmerge main (#14324) 2023-10-05 11:39:48 -06:00
Tim Lee
4b50cf811b
Revert Backmerge from Main 2023-10-04 20:56:17 -06:00
Jacob Shandling
ab50f0f59d
Maintenance merge of main into feature branch (#14308) 2023-10-04 15:59:36 -07:00
Tim Lee
38af9678f4
Backmerge from main (#14298) 2023-10-04 14:24:40 -06:00
Noah Talerman
9c7079f50c
Windows MDM docs: turn on Windows MDM (#13943)
- Add new "Windows setup" page to "Device Management" section of docs
- Rename "MDM setup"  page to "macOS setup." Update links and add redirect
- Rename existing "macOS setup" page to "macOS setup experience." Update links. Did not add redirect because of conflict with "macOS setup" page
- Remove "MDM" from all MDM doc page titles
2023-10-04 15:39:09 -04:00
Lucas Manuel Rodriguez
71a237042a
Add continues to not crash with panics on connection errors (#14223)
I found a few panics in osquery-perf while working in load testing
#13287.

These panic happened due to e.g. network errors.
2023-10-04 09:10:16 -03:00
Robert Henry
81a6c76bed
fix typo in help message in serve.go (#14037) 2023-09-26 13:06:17 -03:00