mirror of
https://github.com/empayre/fleet.git
synced 2024-11-06 17:05:18 +00:00
6415f062c6
This was found while working on #10957. When running a live query, a lot of unused host data is stored in Redis and sent on every live query result message via websockets. The frontend and fleetctl just need `id`, `hostname` and `display_name`. (This becomes worse every time we add new fields to the `Host` struct.) Sample of one websocket message result when running `SELECT * from osquery_info;`: size in `main`: 2234 bytes ``` a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":57,\"host\": {\"created_at\":\"2023-05-22T12:14:11Z\",\"updated_at\":\"2023-05-23T12:31:51Z\", \"software_updated_at\":\"0001-01-01T00:00:00Z\",\"id\":106,\"detail_updated_at\":\"2023-05-23T11:50:04Z\", \"label_updated_at\":\"2023-05-23T11:50:04Z\",\"policy_updated_at\":\"1970-01-02T00:00:00Z\", \"last_enrolled_at\":\"2023-05-22T12:14:12Z\", \"seen_time\":\"2023-05-23T09:52:23.876311-03:00\",\"refetch_requested\":false, \"hostname\":\"lucass-macbook-pro.local\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"platform\":\"darwin\",\"osquery_version\":\"5.8.2\",\"os_version\":\"macOS 13.3.1\",\"build\":\"22E261\",\"platform_like\":\"darwin\",\"code_name\":\"\", \"uptime\":91125000000000,\"memory\":34359738368,\"cpu_type\":\"x86_64h\",\"cpu_subtype\":\"Intel x86-64h Haswell\",\"cpu_brand\":\"Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz\",\"cpu_physical_cores\":4,\"cpu_logical_cores\":8,\"hardware_vendor\":\"Apple Inc.\",\"hardware_model\":\"MacBookPro16,2\",\"hardware_version\":\"1.0\", \"hardware_serial\":\"0DPQR4HMD1FZ\", \"computer_name\":\"Lucas’s MacBook Pro\",\"public_ip\":\"\", \"primary_ip\":\"192.168.0.230\",\"primary_mac\":\"68:2f:67:8e:b6:1f\", \"distributed_interval\":1,\"config_tls_refresh\":60,\"logger_tls_period\":10,\"team_id\":null, \"pack_stats\":null,\"team_name\":null, \"gigs_disk_space_available\":386.23,\"percent_disk_space_available\":40, \"issues\":{\"total_issues_count\":0,\"failing_policies_count\":0}, \"mdm\":{\"enrollment_status\":null,\"server_url\":null,\"name\":\"\",\"encryption_key_available\":false}, \"status\":\"online\",\"display_text\":\"lucass-macbook-pro.local\",\"display_name\":\"Lucas’s MacBook Pro\"}, \"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\", \"config_hash\":\"b7ee9363a7c686e76e99ffb122e9c5241a791e69\",\"config_valid\":\"1\", \"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\", \"host_hostname\":\"lucass-macbook-pro.local\",\"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\",\"start_time\":\"1684757652\", \"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\", \"version\":\"5.8.2\",\"watcher\":\"8364\"}],\"error\":null}}"] ``` vs. size of the message result on this branch: 675 bytes ``` a["{\"type\":\"result\",\"data\":{\"distributed_query_execution_id\":59, \"host\":{\"id\":106,\"hostname\":\"lucass-macbook-pro.local\", \"display_name\":\"Lucas’s MacBook Pro\"}, \"rows\":[{\"build_distro\":\"10.14\",\"build_platform\":\"darwin\", \"config_hash\":\"f80dee827635db39077a458243379b3ad63311fd\", \"config_valid\":\"1\",\"extensions\":\"active\",\"host_display_name\":\"Lucas’s MacBook Pro\", \"host_hostname\":\"lucass-macbook-pro.local\", \"instance_id\":\"cde5de81-344b-4c76-b1c5-dae964fdd4f2\",\"pid\":\"8370\",\"platform_mask\":\"21\", \"start_time\":\"1684757652\",\"uuid\":\"BD4DFA10-E334-41D9-8136-D2163A8FE588\",\"version\":\"5.8.2\", \"watcher\":\"8364\"}]}}"] ``` Manual tests included running with an old fleetctl running with a new fleet server, and vice-versa, a new fleetctl running against an old fleet server. - [X] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [X] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md) - ~[ ] Documented any permissions changes~ - ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements)~ - ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features.~ - [X] Added/updated tests - [X] Manual QA for all new/changed functionality - ~For Orbit and Fleet Desktop changes:~ - ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows and Linux.~ - ~[ ] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).~ |
||
---|---|---|
.. | ||
async | ||
externalsvc | ||
middleware | ||
mock | ||
osquery_utils | ||
redis_policy_set | ||
schedule | ||
testdata | ||
activities_test.go | ||
activities.go | ||
appconfig_test.go | ||
appconfig.go | ||
apple_mdm_test.go | ||
apple_mdm.go | ||
base_client_errors.go | ||
base_client_test.go | ||
base_client.go | ||
campaigns_test.go | ||
campaigns.go | ||
carves_test.go | ||
carves.go | ||
client_appconfig.go | ||
client_apple_mdm.go | ||
client_carves.go | ||
client_debug.go | ||
client_hosts.go | ||
client_labels.go | ||
client_live_query_test.go | ||
client_live_query.go | ||
client_mdm.go | ||
client_packs.go | ||
client_policies.go | ||
client_profiles.go | ||
client_queries.go | ||
client_sessions.go | ||
client_setup.go | ||
client_software.go | ||
client_targets.go | ||
client_teams.go | ||
client_test.go | ||
client_trigger.go | ||
client_users.go | ||
client.go | ||
cron_schedules_test.go | ||
cron_schedules.go | ||
debug_handler_test.go | ||
debug_handler.go | ||
device_client_test.go | ||
device_client.go | ||
devices_test.go | ||
devices.go | ||
endpoint_campaigns.go | ||
endpoint_middleware_test.go | ||
endpoint_middleware.go | ||
endpoint_setup.go | ||
endpoint_utils_test.go | ||
endpoint_utils.go | ||
frontend.go | ||
global_policies_test.go | ||
global_policies.go | ||
global_schedule_test.go | ||
global_schedule.go | ||
handler_test.go | ||
handler.go | ||
hosts_test.go | ||
hosts.go | ||
http_auth_test.go | ||
http_auth.go | ||
http_publicip.go | ||
installer_test.go | ||
installer.go | ||
integration_core_test.go | ||
integration_desktop_test.go | ||
integration_ds_only_test.go | ||
integration_enterprise_test.go | ||
integration_live_queries_test.go | ||
integration_logger_test.go | ||
integration_mdm_test.go | ||
integration_sandbox_test.go | ||
integration_sso_test.go | ||
invites_test.go | ||
invites.go | ||
jitter_test.go | ||
jitter.go | ||
labels_test.go | ||
labels.go | ||
live_queries.go | ||
mail_test.go | ||
mdm_test.go | ||
mdm.go | ||
mem_failing_policies_set_test.go | ||
metrics_appconfig.go | ||
metrics_change_email.go | ||
metrics_invites.go | ||
metrics_labels.go | ||
metrics_sessions.go | ||
metrics_users.go | ||
metrics.go | ||
orbit_client.go | ||
orbit.go | ||
osquery_test.go | ||
osquery.go | ||
packs_test.go | ||
packs.go | ||
queries_test.go | ||
queries.go | ||
scheduled_queries_test.go | ||
scheduled_queries.go | ||
service_appconfig_test.go | ||
service_appconfig.go | ||
service_campaign_test.go | ||
service_campaigns.go | ||
service_errors.go | ||
service_users.go | ||
service.go | ||
sessions_test.go | ||
sessions.go | ||
software_test.go | ||
software.go | ||
status.go | ||
targets_test.go | ||
targets.go | ||
team_policies_test.go | ||
team_policies.go | ||
team_schedule_test.go | ||
team_schedule.go | ||
teams_test.go | ||
teams.go | ||
testing_client.go | ||
testing_utils.go | ||
translator.go | ||
transport_error_test.go | ||
transport_error.go | ||
transport_setup.go | ||
transport_test.go | ||
transport.go | ||
trigger.go | ||
user_roles.go | ||
users_test.go | ||
users.go | ||
validation_setup.go |