Commit Graph

608 Commits

Author SHA1 Message Date
Roberto Dip
33db665d63
show full formatted results for windows commands in fleetctl (#14922)
for #14912 this adds the full results to the "RESULTS" column of
`fleetctl get mdm-command-results`.

Additionally I included formatting of the XML output to improve
readability.
2023-11-03 12:01:43 -03:00
Roberto Dip
b04b20fcf0
fix fleetctl to set different CmdUID for Windows (#14895)
for #14891
2023-11-02 15:06:37 -03:00
Roberto Dip
9cf30a9131
Feat windows msmdm (#14837)
for #13069

---------

Co-authored-by: Marcos Oviedo <marcos@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
2023-11-01 11:13:12 -03:00
guangwu
33858d7301
chore: remove refs to deprecated io/ioutil (#14485)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-27 15:28:54 -03:00
Jahziel Villasana-Espinoza
c570ef1e21
feat: make analytics always true for premium customers (#14607)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-10-27 11:41:18 -04:00
Roberto Dip
5dfc72b2a1
allow to use --context in fleetctl mdm run-command (#14665)
for #13712
2023-10-26 19:09:31 -03:00
Lucas Manuel Rodriguez
fd99fd4121
Add optimizations for query reports at scale (#14710)
#7766

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.~
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~

---------

Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
2023-10-25 19:20:27 -03:00
Lucas Manuel Rodriguez
5d7ee58a87
Do not leak ticks and fix race condition (#14667)
This PR introduces the following fixes:
- Fixes a race condition by protecting the `scheduledQueries` and
`scheduledQueryData` fields with a `sync.Mutex` (*).
- Adds some more information about request counts for /log, /config and
/distributed/read requests (and uptime).
- Fixes the resource leaks around creating time.Ticks on every for loop
iteration.

(*) Sample of the race condition when running with `-race`:
```
==================
WARNING: DATA RACE
Read at 0x00c000604800 by goroutine 20:
  main.(*agent).runLoop()
      /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:525 +0x71b
  main.main.func2()
      /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:1737 +0x4e

Previous write at 0x00c000604800 by goroutine 40:
  main.(*agent).config()
      /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:915 +0xb30
  main.(*agent).runLoop.func2()
      /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:512 +0x37
```
2023-10-20 10:29:59 -03:00
Lucas Manuel Rodriguez
22bba274b7
Run logger, distributed and config in separate goroutines (#14584) 2023-10-17 11:30:59 -03:00
Lucas Manuel Rodriguez
8c6cefc4f4
Fix panic in osquery-perf (#14534)
This is safe to merge during freeze as this is an osquery-perf only
change.
```
panic: runtime error: index out of range [1] with length 1

goroutine 14 [running]:
main.(*agent).config(0xc0001e5b80)
        /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:847 +0xb74
main.(*agent).runLoop(0xc0001e5b80, 0x0?, 0x0?)
        /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:463 +0x4d2
created by main.main in goroutine 1
        /Users/luk/fleetdm/git/fleet/cmd/osquery-perf/agent.go:1666 +0x1285
```
2023-10-13 14:26:46 -03:00
Jahziel Villasana-Espinoza
f92695fda5
osquery-perf: can send scheduled query results (#14502) 2023-10-12 20:41:04 -06:00
Jacob Shandling
70e0182e4d Merge branch 'main' into 7766-main-merge 2023-10-11 12:56:52 -07:00
Roberto Dip
9ffa11c25d
Feat: saved scripts (#14409)
For #9537
2023-10-10 19:00:45 -03:00
Jacob Shandling
fb494ca3cb Merge branch '7766-frontend' into 7766-backend-frontend 2023-10-10 13:21:19 -07:00
Jahziel Villasana-Espinoza
0435431ae9
feat: can delete via cron job if global flag set (#14398)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added/updated tests

---------

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2023-10-10 10:06:54 -03:00
Tim Lee
fa55d72118
Save Query Results to DB (#14335)
# Checklist for submitter

#13486 

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Eric <eashaw@sailsjs.com>
Co-authored-by: Sampfluger88 <108141731+Sampfluger88@users.noreply.github.com>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Roberto Dip <me@roperzh.com>
Co-authored-by: Tyler Diderich <15862572+tdiderich@users.noreply.github.com>
Co-authored-by: Dave Herder <27025660+dherder@users.noreply.github.com>
Co-authored-by: Rachael Shaw <r@rachael.wtf>
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
Co-authored-by: Nathanael Holliday <100959072+hollidayn@users.noreply.github.com>
Co-authored-by: Katheryn Satterlee <me@ksatter.com>
Co-authored-by: Mo Zhu <mo@fleetdm.com>
Co-authored-by: Mo Zhu <mozhu888@gmail.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Robert Fairburn <8029478+rfairburn@users.noreply.github.com>
Co-authored-by: Sabrina Coy <13890648+sabrinabuckets@users.noreply.github.com>
Co-authored-by: Isabell Reedy <113355639+ireedy@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Alex Mitchell <105945793+alexmitchelliii@users.noreply.github.com>
2023-10-10 09:44:03 -03:00
Jahziel Villasana-Espinoza
5c868c9d3d
feat: deletion for query results (#14302)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
2023-10-09 17:43:17 -04:00
Jacob Shandling
eb327faabf
maintenance merge of main into feature branch (#14393)
maintenance merge of `main` into feature branch
2023-10-09 14:28:35 -07:00
Marcos Oviedo
f0d77ab3db
Merging Bitlocker feature branch (#14350)
This relates to #12577

---------

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2023-10-06 19:04:33 -03:00
Tim Lee
ccd6746633
7766 backmerge main (#14324) 2023-10-05 11:39:48 -06:00
Tim Lee
4b50cf811b
Revert Backmerge from Main 2023-10-04 20:56:17 -06:00
Jacob Shandling
ab50f0f59d
Maintenance merge of main into feature branch (#14308) 2023-10-04 15:59:36 -07:00
Tim Lee
38af9678f4
Backmerge from main (#14298) 2023-10-04 14:24:40 -06:00
Noah Talerman
9c7079f50c
Windows MDM docs: turn on Windows MDM (#13943)
- Add new "Windows setup" page to "Device Management" section of docs
- Rename "MDM setup"  page to "macOS setup." Update links and add redirect
- Rename existing "macOS setup" page to "macOS setup experience." Update links. Did not add redirect because of conflict with "macOS setup" page
- Remove "MDM" from all MDM doc page titles
2023-10-04 15:39:09 -04:00
Lucas Manuel Rodriguez
71a237042a
Add continues to not crash with panics on connection errors (#14223)
I found a few panics in osquery-perf while working in load testing
#13287.

These panic happened due to e.g. network errors.
2023-10-04 09:10:16 -03:00
Robert Henry
81a6c76bed
fix typo in help message in serve.go (#14037) 2023-09-26 13:06:17 -03:00
Jacob Shandling
1993c02b3a
Move code for --local-wix-dir into ee directory (#14104) 2023-09-25 10:38:03 -07:00
Jacob Shandling
b6f072775f
Add ability for fleetctl package to use local WiX v3 binaries when generating installer .msi (#14033) 2023-09-22 08:49:01 -07:00
Tim Lee
e80ea50240
Flock vulnerability false positive (#13827) 2023-09-14 10:38:36 -06:00
Roberto Dip
541cc638e4
Add an activity log when an script execution is requested (#13655)
For #9553
2023-09-05 20:09:29 -03:00
gillespi314
37fb4b0dab
Add fleetctl run-script command (#13622) 2023-09-05 14:14:09 -05:00
Martin Angers
cbc3f32e9d
Adjust response payload, messages and validations for /scripts/run/* endpoints. (#13607) 2023-08-31 09:08:50 -05:00
Lucas Manuel Rodriguez
9142c5de79
Prevent thundering herd when applying large number of policies on large number of hosts (#13552)
#13527

(Adding @mna to double check the changes in the async implementation of
policy result storage)

This PR also adds the osquery-perf changes needed to define the count of
macOS and Windows hosts.

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)~
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~

Test with 80k hosts: 70k simulated macOS, 10k simulated Windows.
Apply Windows policies first, then apply macOS policies:
```
fleetctl apply -f ee/cis/win-10/cis-policy-queries.yml

# Leave running for some time

fleetctl apply -f ee/cis/macos-13/cis-policy-queries.yml
```

After applying CIS policies previous to these changes:
![Screenshot 2023-08-23 at 11 36
18](https://github.com/fleetdm/fleet/assets/2073526/72c1dc7d-e601-4248-be35-93c85b749f5d)

After applying these changes and applying the same policies:
![Screenshot 2023-08-28 at 15 42
57](https://github.com/fleetdm/fleet/assets/2073526/6b6d76b8-6acb-4893-a913-bf603a68f1a4)
2023-08-31 10:58:50 -03:00
Martin Angers
090b142c49
Implement script execution on the fleetd agent (disabled by default) (#13569) 2023-08-30 14:02:44 -04:00
Roberto Dip
b50e1939db
Allow to configure fleetd for script execution (#13564)
Related to #13310 and #13304 this adds two ways to enable script
execution in `fleetd` (the orbit component)

- By building a package with `--enable-scripts`
- By providing a setting via a configuration profile (macOS only)

Due to how the profile assignment works, this change automatically
updates the `com.fleetdm.fleetd.config` for hosts that already have the
profile installed.

> [!NOTE]
> Documentation is in
[#13577](https://github.com/fleetdm/fleet/pull/13577) to decouple
reviews.
2023-08-30 10:18:34 -03:00
Martin Angers
edf4a4d02f
Add script execution simulation to osquery-perf in preparation for load testing (part 3 of ticket) (#13456) 2023-08-23 18:31:47 -04:00
Tim Lee
3b61adf7a4
Add validation for policy specs (#13294) 2023-08-21 10:22:07 -06:00
Jacob Shandling
a7f1501054
Update fleetctl convert for schedulable queries (#13125) 2023-08-08 10:39:20 -07:00
Martin Angers
37ba43d404
Add fleetctl upgrade-packs command to migrate 2017 packs to queries (#13078) 2023-08-08 08:21:57 -04:00
Martin Angers
554e024f7b
Fix gitops access when using --dry-run with fleetctl apply (#13178) 2023-08-07 13:51:11 -04:00
Juan Fernandez
63c7c8a3ae
If no global queries, show proper message at the end (#13006)
If no global queries, show proper message at the end
2023-07-27 16:55:07 -04:00
Juan Fernandez
1dde10d5c3
Print inherited message if no queries found (#12998)
If the team has no queries, then fleetctl output should also show the number of inherited queries.
2023-07-27 15:08:40 -04:00
Juan Fernandez
e4cc0c3098
Fixed format issues with fleetctl get queries (#12983)
Output from `fleetctl get queries` should include the team the query is in and also the scheduling information.
2023-07-27 09:29:09 -04:00
Martin Angers
6f77911ffe
Fix performance regression found in load testing (#12981) 2023-07-26 17:13:27 -04:00
Lucas Manuel Rodriguez
2afbd24021
Combine Schedules and Queries: API changes (#12778)
Combining schedules and queries API changes.
2023-07-24 20:17:20 -04:00
Juan Fernandez
6b664a2a82
Merge branch 'main' into 7765-combined-schedules-and-queries 2023-07-21 13:53:13 -04:00
Lucas Manuel Rodriguez
b803bbe5bf
osquery-perf changes needed for load testing with simulated Windows hosts (#12754)
Changes in osquery-perf to allow for testing of Windows hosts in
loadtest environments.
2023-07-14 13:06:34 -03:00
Gabriel Hernandez
9aa7c0c714
add dark and light background logo colors and show them on mdm migrat… (#12681) 2023-07-13 19:35:25 +01:00
Roberto Dip
666ae8d787
ensure ds.TeamByName returns a 4xx response if no team is found (#12620)
this helps consumer of the datastore method handle the not found
scenario better and ensures we always return a 4xx code by default if we
can't find a matching team.

seems like calls to this method were special-cased everywhere except in
the apply user roles endpoint, where we returned a `500` status code if
we couldn't find a team.
2023-07-13 11:55:05 -03:00
Juan Fernandez
390e0565d0
Updated delete method on the DB layer 2023-07-07 07:31:36 -04:00