Commit Graph

338 Commits

Author SHA1 Message Date
Zach Wasserman
2203bd81a2
Add support for IdP-initiated SSO login (#556)
This feature is off by default due to minor potential security concerns
with IdP-initiated SSO (see
https://auth0.com/docs/protocols/saml-protocol/saml-configuration-options/identity-provider-initiated-single-sign-on#risks-of-using-an-identity-provider-initiated-sso-flow).

Closes #478
2021-03-30 12:56:20 -07:00
Zach Wasserman
5b5dd1379b
Prefer IPv4 in Host Details (#548)
- Use a non-loopback, non-link-local IPv4 address if available
- Otherwise use non-loopback, non-link-local IPv6 address if available
- Otherwise use any address

Closes #532
2021-03-26 18:05:49 -07:00
Zach Wasserman
0bd6903b2d
Add version endpoint to API (#549)
Part of #371
2021-03-26 18:03:31 -07:00
Zach Wasserman
12d292164f
Add rate-limiting to login and password reset (#543)
Prevent abuse of these endpoints with rate limiting backed by Redis. The
limits assigned should be appropriate for almost any Fleet deployment.

Closes #530
2021-03-26 11:23:29 -07:00
noahtalerman
ef414153a9
Add "Last fetched" to Hosts page and Host details page (#540)
- Add a "Last fetched" column to the table on the Hosts page. This column uses the `detail_updated_at` property.
- Add a "Last fetched" timestamp to the Host details page.
- Adjust styles on _Host details_ page
2021-03-25 12:47:00 -07:00
Zach Wasserman
fb9706912d
Prevent user enumeration (#533)
- Return same error in all cases for login endpoint.
- Log error details in server logs.
- Make most login errors take ~1s to prevent timing attacks.
- Don't return forgot password errors.
- Log password errors in server logs.
- Make most forgot password requests take ~1s to prevent timing attacks.

Fixes #531
2021-03-24 19:36:30 -07:00
Zach Wasserman
d827a9d6b7
Log all errors regardless of log level (#496)
Modify the loggerDebug function to check whether the error is nil before
choosing the log level.

Fixes #473
2021-03-18 16:06:06 -07:00
Scott Lampert
5c69094571
Remove https check for client connection (#489)
Removes the https requirement for localhost only.
2021-03-18 11:21:36 -07:00
Lars Lehtonen
9b902428ea
server: prune unused constants and functions (#477)
This removes unused `const` variables and functions from `server` and its subpackages.
2021-03-15 17:08:25 -07:00
Lars Lehtonen
677ecf9eaf
server/service: prune dead code (#432) 2021-03-10 07:34:37 -08:00
Zach Wasserman
d1dfba1d68
Fix flapping IP addresses and host additional (#420)
The AuthenticateHost loading of hosts accidentally dropped IP addresses,
which would cause the IP to be dropped on save under certain scenarios.

Also fixes a potential issue with flapping host additional info.

Fixes #358
2021-03-09 09:01:26 -08:00
Zach Wasserman
f90da6a090
Make enrollment cooldown configurable (#418)
The enrollment cooldown period was sometimes causing problems when
osquery (probably unintentionally, see
https://github.com/osquery/osquery/issues/6993) tried to enroll more
than once from the same osqueryd process.

We now set this to default to off and make it configurable. With #417
this feature may be unnecessary for most deployments.
2021-03-08 21:26:09 -08:00
Zach Wasserman
cfba095cda
Make host identifier configurable within Fleet (#417)
Osquery now exposes more information during host enrollment than Fleet
previously handled. We can use this to provide more options to users in
problematic enrollment scenarios.

Users can configure --osquery_host_identifier in Fleet to set which
identifier is used to determine uniqueness of hosts. The
default (provided) replicates existing behavior in Fleet. For many
users, setting this to instance will provide better enrollment
stability.

Closes #373
2021-03-08 18:35:17 -08:00
Lars Lehtonen
9317c81edc
server/service: Remove duplicate log import (#408) 2021-03-08 10:59:39 -08:00
Zach Wasserman
2025afee71
Better handling of config_refresh values from clients (#388)
Since the original logic was implemented, there have been some changes
in the way that config refreshes are configured. This commit reflects
those changes and should be backwards compatible.

Closes #357
2021-03-05 08:25:46 -08:00
Zach Wasserman
f9312df174
Remove flaky log rotation test (#385)
This test has been flaky for years and it's time for it to go.
2021-03-04 16:29:50 -08:00
Zach Wasserman
471f25e470
Return denylist parameter with pack configs (#339)
This was previously handled correctly within the Fleet server datastores
and API endpoints, but not returned to the actual osquery client.

Fixes #338
2021-02-21 09:58:36 -08:00
Zach Wasserman
c6c5e65a7d
Add support for filtering hosts in label (#335)
Support the same query syntax as the hosts endpoint, here also bounded
by the membership of the label.
2021-02-18 12:52:43 -08:00
Zach Wasserman
cca4802642
Add support for query parameter in list hosts endpoint (#327)
Uses a LIKE clause to search for hosts matching the query against
columns `host_name`, `uuid`, `hardware_serial`, and `primary_ip`.

Introduces the `searchLike` helper to add the appropriate filters to the
SQL query.
2021-02-16 16:53:42 -08:00
Zach Wasserman
6b5019d65e
Add aliases for order keys in API (#329)
Aliases `hostname` (`host_name`) and `memory` (`physical_memory`) when
used as keys for ordering in the API this allows for better consistency
on the frontend.

To be cleaned up further in #317
2021-02-16 15:25:34 -08:00
Zach Wasserman
b83fbf0ec9
Special case host_name on the backend to enable frontend consistency (#318)
See #317 for followup to this.
2021-02-15 11:58:05 -08:00
Zach Wasserman
7a68e3de65
Deprecate /api/v1/kolide routes (#297)
- Support both /api/v1/fleet and /api/v1/kolide routes in server.
- Add logging for use of deprecated routes.
- Rename routes in frontend JS.
- Rename routes and add notes in documentation.
2021-02-10 12:13:11 -08:00
Zach Wasserman
f68f4238e8
Merge pull request from GHSA-xwh8-9p3f-3x45
- Fix the specific case that caused panic.
- Add panic handler around entire live query results handler. This will
  prevent similar issues from causing crashes in the future.

Note that other endpoints already have panic handling but this one is
special due to the use of websockets.
2021-02-03 08:47:43 -08:00
Zach Wasserman
22d9205d35
Add debug flag to fleetctl (#266)
This flag enables logging of HTTP requests and responses to stderr.

Closes #187
2021-02-02 18:55:16 -08:00
Zach Wasserman
a9c0c472fc
Update Kolide to Fleet as appropriate in documentation (#263) 2021-02-02 12:16:59 -08:00
Zach Wasserman
c3a18ea7d1
Rename prefixed system queries (#253)
Renames live query names that the system uses from `kolide_` to
`fleet_`.
2021-02-01 11:39:53 -08:00
Zach Wasserman
bb124dc2b9
Improvements to fleetctl preview experience (#234)
- Make the preview directory in the default .fleet directory.
- Check for Docker daemon installed but not running.
- Add message for Chrome users on self-signed certs.
- Display login information on later invocations of command.
- Remove "Kolide" from error messages.

Closes #190

Part of #197
2021-01-28 07:57:32 -08:00
Zach Wasserman
87330a9753
Add support for denylist parameter in scheduled queries (#209)
Closes #202
2021-01-25 16:26:14 -08:00
Zach Wasserman
6215acdd1b
Add host details in API responses (#223)
Add label and pack information for the returned hosts in the single-host
API endpoints.

Example:

```
curl -k 'https://localhost:8080/api/v1/kolide/hosts/7' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzZXNzaW9uX2tleSI6Ii9oNEZ4MUpEVmlvQWhtMC8wNUJKbzZpdldsUDZpMDhjQVBuZXRLeFIvWjNOUGgvMW9VdCsxQnFlNU1CVDVsMlU3ckVGMm5Sb1VxS3ZSUllzSmJJR2lBPT0ifQ.GQQsJgBU3JA1H1o4Y8fPjyfF78F_VY4c9AbrP5k0sCg'
{
  "host": {
    "created_at": "2021-01-16T00:22:33Z",
    "updated_at": "2021-01-16T00:22:51Z",
    "id": 7,
    "detail_updated_at": "1970-01-02T00:00:00Z",
    "label_updated_at": "1970-01-02T00:00:00Z",
    "last_enrolled_at": "2021-01-16T00:22:33Z",
    "seen_time": "2021-01-16T00:22:51Z",
    "hostname": "55d91fc9c303",
    "uuid": "853a4588-0000-0000-a061-7d494d04e9c4",
    "platform": "ubuntu",
    "osquery_version": "4.6.0",
    "os_version": "Ubuntu 20.04.0",
    "build": "",
    "platform_like": "debian",
    "code_name": "",
    "uptime": 0,
    "memory": 16794206208,
    "cpu_type": "x86_64",
    "cpu_subtype": "158",
    "cpu_brand": "Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz\u0000\u0000\u0000\u0000\u0000\u0000\u0000",
    "cpu_physical_cores": 8,
    "cpu_logical_cores": 8,
    "hardware_vendor": "",
    "hardware_model": "",
    "hardware_version": "",
    "hardware_serial": "",
    "computer_name": "55d91fc9c303",
    "primary_ip": "",
    "primary_mac": "",
    "distributed_interval": 10,
    "config_tls_refresh": 0,
    "logger_tls_period": 10,
    "enroll_secret_name": "default",
    "labels": [
      {
        "created_at": "2020-12-22T01:22:47Z",
        "updated_at": "2020-12-22T01:22:47Z",
        "id": 6,
        "name": "All Hosts",
        "description": "All hosts which have enrolled in Fleet",
        "query": "select 1;",
        "label_type": "builtin",
        "label_membership_type": "dynamic"
      }
    ],
    "packs": [
      {
        "created_at": "2021-01-20T16:36:42Z",
        "updated_at": "2021-01-20T16:36:42Z",
        "id": 2,
        "name": "test"
      }
    ],
    "status": "offline",
    "display_text": "55d91fc9c303"
  }
}
```
2021-01-25 13:05:02 -08:00
Zach Wasserman
deaf8880f3
Expose live query error messages via API (#205)
Somewhere around osquery 4.4.0 these messages were added to query
responses. We can now expose them to the API clients rather than using
the placeholder text.

Required for #192
2021-01-19 14:52:29 -08:00
Zach Wasserman
f254a9a343
Make name and secret required for enroll secrets (#207)
Adds a check to prevent users from unintentionally setting empty
secrets.

Fixes #188
2021-01-19 14:49:53 -08:00
Matteo Piano
c89cd370d5
Add AWS S3 as file carving backend (#126)
This adds the option to set up an S3 bucket as the storage backend for file carving (partially solving #111).

It works by using the multipart upload capabilities of S3 to maintain compatibility with the "upload in blocks" protocol that osquery uses. It does this basically replacing the carve_blocks table while still maintaining the metadata in the original place (it would probably be possible to rely completely on S3 by using object tagging at the cost of listing performance). To make this pluggable, I created a new field in the service struct dedicated to the CarveStore which, if no configuration for S3 is set up will be just a reference to the standard datastore, otherwise it will point to the S3 one (effectively this separation will allow in the future to add more backends).
2020-12-16 09:16:55 -08:00
Zach Wasserman
55a2aa2c95
Fix logging for query creation on error (#132) 2020-12-14 18:45:03 -08:00
Zach Wasserman
5b432cccee
Prevent SQLite ATTACH from being used in queries (#129)
Mitigate
[CVE-2020-26273](https://github.com/osquery/osquery/security/advisories/GHSA-4g56-2482-x7q8)
by attempting to prevent users from executing or saving queries that use
the SQLite `ATTACH` command.

Users must still update to osquery 4.6.0 to ensure the functionality is
fully disabled in osquery.
2020-12-14 18:13:34 -08:00
Zach Wasserman
bb921bc973
Allow scheduling same query more than once in UI (#96)
Fixes #88
2020-12-04 09:50:39 -08:00
Zach Wasserman
e7912c2d31
Minor code/comment cleanups (#47)
- Fixes an initialization error panic to a fatal log
2020-11-18 11:10:55 -08:00
Zach Wasserman
0bb9d69ece
Add debug endpoints and associated fleetctl commands (#45)
Adds endpoints and fleetctl commands to retrieve various debug profiles
from the Fleet server.

The best summary is from the help text:

```
fleetctl debug
NAME:
   fleetctl debug - Tools for debugging Fleet

USAGE:
   fleetctl debug command [command options] [arguments...]

COMMANDS:
   profile    Record a CPU profile from the Fleet server.
   cmdline    Get the command line used to invoke the Fleet server.
   heap       Report the allocated memory in the Fleet server.
   goroutine  Get stack traces of all goroutines (threads) in the Fleet server.
   trace      Record an execution trace on the Fleet server.
   archive    Create an archive with the entire suite of debug profiles.

OPTIONS:
   --config value   Path to the Fleet config file (default: "/Users/zwass/.fleet/config") [$CONFIG]
   --context value  Name of Fleet config context to use (default: "default") [$CONTEXT]
   --help, -h       show help
```
2020-11-17 17:12:37 -08:00
Zach Wasserman
ab94d94da0
Fix fleetctl Windows issues (#40)
- Properly set the path for the config file on Windows.
- Check for appropriate settings for TLS config.

Fixes #39
2020-11-17 16:02:14 -08:00
billcobbler
618ba56484
Add host additional info filters (#28)
This change adds the ability to filter additional host info via the list hosts endpoint; a continuation from [here](https://github.com/kolide/fleet/pull/2330), but now filtering is accomplished via SQL. 

Additional object without filter:
```
curl 'https://localhost:8080/api/v1/kolide/hosts'
...
"additional": {
        "macs": [
          {
            "mac": "00:00:00:00:00:00"
          },
          {
            "mac": "02:42:c0:a8:10:05"
          }
        ],
        "time": [
          {
            "day": "13",
            "hour": "3",
            "year": "2020",
            "month": "10",
            "minutes": "43",
            "seconds": "11",
            "weekday": "Tuesday",
            "datetime": "2020-10-13T03:43:11Z",
            "iso_8601": "2020-10-13T03:43:11Z",
            "timezone": "GMT",
            "timestamp": "Tue Oct 13 03:43:11 2020 UTC",
            "unix_time": "1602560591",
            "local_time": "1602560591",
            "local_timezone": "UTC"
          }
},
...
```

Additional object with filter:
```
curl 'https://localhost:8080/api/v1/kolide/hosts?additional_info_filters=macs,notreal'
...
"additional": {
        "macs": [
          {
            "mac": "00:00:00:00:00:00"
          },
          {
            "mac": "02:42:c0:a8:10:05"
          }
        ],
        "notreal": null
},
...
```
2020-11-13 16:33:25 -08:00
Zach Wasserman
8f435a95ba
Allow @ character in usernames (#37)
PR #9 unintentionally exposed the validation that prevented the @
character in usernames. We have decided there is no reason to block this
character.

Fixes #36
2020-11-13 15:52:20 -08:00
Zach Wasserman
6cbd10965c
Add dev infrastructure and docs for Prometheus monitoring (#33)
- Set up a simple example of Prometheus monitoring in the development
  docker-compose.yml.
- Add documentation for configuring Prometheus.
2020-11-12 19:06:56 -08:00
Mike Arpaia
af96e52a00
Update the Go import paths to new repo name (#27) 2020-11-11 09:59:12 -08:00
Zachary Wasserman
e452cc6a8a
Add file carving support (#15)
- Add endpoints for osquery to register and continue a carve.
- Implement client functionality for retrieving carve details and contents in fleetctl.
- Add documentation on using file carving with Fleet.

Addresses kolide/fleet#1714
2020-11-04 20:45:16 -08:00
Zachary Wasserman
7c923d9d19
Implement fleetctl user create (#9)
- Allow user creation via `fleetctl user create`
- Cleanup and rename existing methods for clarity

Fixes https://github.com/kolide/fleet/issues/2306
2020-11-04 17:06:55 -08:00
Brendan Shaklovitz
c6b285c56e
Remove soft-deletion pattern (#2327)
* Perform migration to delete any entries with `deleted` set, and
subsequently drop columns `deleted` and `deleted_at`.
* Remove `deleted` and `deleted_at` references.

Closes #2146
2020-10-22 10:51:26 -07:00
Zachary Wasserman
e8e4bc9879
Fix cleanup of orphaned queries (#2316)
The expiration logic was incorrect leading to queries not being cleaned
up properly. Tests added for the whole subroutine.

Fixes #2302
2020-10-06 09:30:24 -07:00
Zachary Wasserman
50dbdb3db4
Add explicit int64 for parsing physical_memory column (#2315)
Addresses an error when Fleet is compiled for a 32-bit architecture.

May not be comprehensive of all related errors.

Fixes #2314.
2020-10-05 17:03:32 -07:00
Zachary Wasserman
f2d3683d80
Log host identifier in EnrollAgent service logging (#2313)
This can help identify circumstances in which multiple hosts are
providing the same identifier and clobbering the node key for each
other.
2020-10-02 13:23:54 -07:00
Iain Steers
da99617882
Add DeleteHost func to service pkg (#2312) 2020-10-01 16:14:18 -07:00
James Alseth
3a63dac4a3
Add compression option for filesystem logs when they're rotated (#2292) 2020-09-09 13:33:32 -07:00