fleet/docs/Contributing/API-for-contributors.md
eashaw 0af5e161d0
Remove numbers from documentation filenames in Fleet repo (#4313)
* Renaming files and a lot of find and replace

* pageRank meta tags, sorting by page rank

* reranking

* removing numbers

* revert changing links that are locked to a commit

* update metatag name, uncomment github contributers

* Update basic-documentation.page.js

* revert link change

* more explicit errors, change pageOrderInSection numbers, updated sort

* Update build-static-content.js

* update comment

* update handbook link

* handbook entry

* update sort

* update changelog doc links to use fleetdm.com

* move standard query library back to old location, update links/references to location

* revert unintentional link changes

* Update handbook/community.md

Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>

Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
2022-02-23 12:17:55 -06:00

31 KiB
Raw Blame History

API for contributors

This document includes the Fleet API routes that are helpful when developing or contributing to Fleet.

Unlike the Fleet REST API documentation, the API routes in this document are only intended for use by the Fleet UI and fleetctl clients.

Get queries spec

Returns a list of all queries in the Fleet instance. Each item returned includes the name, description, and SQL of the query.

GET /api/v1/fleet/spec/queries

Parameters

None.

Example

GET /api/v1/fleet/spec/queries

Default response

Status: 200

{
  "specs": [
    {
      "name": "query1",
      "description": "query",
      "query": "SELECT * FROM osquery_info"
    },
    {
      "name": "osquery_schedule",
      "description": "Report performance stats for each file in the query schedule.",
      "query": "select name, interval, executions, output_size, wall_time, (user_time/executions) as avg_user_time, (system_time/executions) as avg_system_time, average_memory, last_executed from osquery_schedule;"
    }
  ]
}

Get query spec

Returns the name, description, and SQL of the query specified by name.

GET /api/v1/fleet/spec/queries/{name}

Parameters

Name Type In Description
name string path Required. The name of the query.

Example

GET /api/v1/fleet/spec/queries/query1

Default response

Status: 200

{
  "specs": {
    "name": "query1",
    "description": "query",
    "query": "SELECT * FROM osquery_info"
  }
}

Apply queries spec

Creates and/or modifies the queries included in the specs list. To modify an existing query, the name of the query included in specs must already be used by an existing query. If a query with the specified name doesn't exist in Fleet, a new query will be created.

POST /api/v1/fleet/spec/queries

Parameters

Name Type In Description
specs list body Required. The list of the queries to be created or modified.

Example

POST /api/v1/fleet/spec/queries

Request body
{
  "specs": [
    {
      "name": "new_query",
      "description": "This will be a new query because a query with the name 'new_query' doesn't exist in Fleet.",
      "query": "SELECT * FROM osquery_info"
    },
    {
      "name": "osquery_schedule",
      "description": "This queries description and SQL will be modified because a query with the name 'osquery_schedule' exists in Fleet.",
      "query": "SELECT * FROM osquery_info"
    }
  ]
}
Default response

Status: 200

Get packs spec

Returns the specs for all packs in the Fleet instance.

GET /api/v1/fleet/spec/packs

Example

GET /api/v1/fleet/spec/packs

Default response

Status: 200

{
  "specs": [
    {
      "id": 1,
      "name": "pack_1",
      "description": "Description",
      "disabled": false,
      "targets": {
        "labels": ["All Hosts"],
        "teams": null
      },
      "queries": [
        {
          "query": "new_query",
          "name": "new_query",
          "description": "",
          "interval": 456,
          "snapshot": false,
          "removed": true,
          "platform": "windows",
          "version": "4.5.0"
        },
        {
          "query": "new_title_for_my_query",
          "name": "new_title_for_my_query",
          "description": "",
          "interval": 677,
          "snapshot": true,
          "removed": false,
          "platform": "",
          "version": ""
        },
        {
          "query": "osquery_info",
          "name": "osquery_info",
          "description": "",
          "interval": 6667,
          "snapshot": true,
          "removed": false,
          "platform": "",
          "version": ""
        },
        {
          "query": "query1",
          "name": "query1",
          "description": "",
          "interval": 7767,
          "snapshot": false,
          "removed": true,
          "platform": "",
          "version": ""
        },
        {
          "query": "osquery_events",
          "name": "osquery_events",
          "description": "",
          "interval": 454,
          "snapshot": false,
          "removed": true,
          "platform": "",
          "version": ""
        },
        {
          "query": "osquery_events",
          "name": "osquery_events-1",
          "description": "",
          "interval": 120,
          "snapshot": false,
          "removed": true,
          "platform": "",
          "version": ""
        }
      ]
    },
    {
      "id": 2,
      "name": "pack_2",
      "disabled": false,
      "targets": {
        "labels": null,
        "teams": null
      },
      "queries": [
        {
          "query": "new_query",
          "name": "new_query",
          "description": "",
          "interval": 333,
          "snapshot": false,
          "removed": true,
          "platform": "windows",
          "version": "4.5.0",
          "shard": 10,
          "denylist": null
        }
      ]
    }
  ]
}

Apply packs spec

Returns the specs for all packs in the Fleet instance.

POST /api/v1/fleet/spec/packs

Parameters

Name Type In Description
specs list body Required. A list that includes the specs for each pack to be added to the Fleet instance.

Example

POST /api/v1/fleet/spec/packs

Request body
{
  "specs": [
    {
      "id": 1,
      "name": "pack_1",
      "description": "Description",
      "disabled": false,
      "targets": {
        "labels": ["All Hosts"],
        "teams": null
      },
      "queries": [
        {
          "query": "new_query",
          "name": "new_query",
          "description": "",
          "interval": 456,
          "snapshot": false,
          "removed": true
        },
        {
          "query": "new_title_for_my_query",
          "name": "new_title_for_my_query",
          "description": "",
          "interval": 677,
          "snapshot": true,
          "removed": false
        },
        {
          "query": "osquery_info",
          "name": "osquery_info",
          "description": "",
          "interval": 6667,
          "snapshot": true,
          "removed": false
        },
        {
          "query": "query1",
          "name": "query1",
          "description": "",
          "interval": 7767,
          "snapshot": false,
          "removed": true
        },
        {
          "query": "osquery_events",
          "name": "osquery_events",
          "description": "",
          "interval": 454,
          "snapshot": false,
          "removed": true
        },
        {
          "query": "osquery_events",
          "name": "osquery_events-1",
          "description": "",
          "interval": 120,
          "snapshot": false,
          "removed": true
        }
      ]
    },
    {
      "id": 2,
      "name": "pack_2",
      "disabled": false,
      "targets": {
        "labels": null,
        "teams": null
      },
      "queries": [
        {
          "query": "new_query",
          "name": "new_query",
          "description": "",
          "interval": 333,
          "snapshot": false,
          "removed": true,
          "platform": "windows"
        }
      ]
    }
  ]
}
Default response

Status: 200

Get pack spec by name

Returns the spec for the specified pack by pack name.

GET /api/v1/fleet/spec/packs/{name}

Parameters

Name Type In Description
name string path Required. The pack's name.

Example

GET /api/v1/fleet/spec/packs/pack_1

Default response

Status: 200

{
  "specs": {
    "id": 15,
    "name": "pack_1",
    "description": "Description",
    "disabled": false,
    "targets": {
      "labels": ["All Hosts"],
      "teams": null
    },
    "queries": [
      {
        "query": "new_title_for_my_query",
        "name": "new_title_for_my_query",
        "description": "",
        "interval": 677,
        "snapshot": true,
        "removed": false,
        "platform": "",
        "version": ""
      },
      {
        "query": "osquery_info",
        "name": "osquery_info",
        "description": "",
        "interval": 6667,
        "snapshot": true,
        "removed": false,
        "platform": "",
        "version": ""
      },
      {
        "query": "query1",
        "name": "query1",
        "description": "",
        "interval": 7767,
        "snapshot": false,
        "removed": true,
        "platform": "",
        "version": ""
      },
      {
        "query": "osquery_events",
        "name": "osquery_events",
        "description": "",
        "interval": 454,
        "snapshot": false,
        "removed": true,
        "platform": "",
        "version": ""
      },
      {
        "query": "osquery_events",
        "name": "osquery_events-1",
        "description": "",
        "interval": 120,
        "snapshot": false,
        "removed": true,
        "platform": "",
        "version": ""
      }
    ]
  }
}

Apply team spec

Available in Fleet Premium

If the name specified is associated with an existing team, this API route, completely replaces this team's existing agent_options and secrets with those that are specified.

If the name is not already associated with an existing team, this API route creates a new team with the specified name, agent_options, and secrets.

POST /api/v1/fleet/spec/teams

Parameters

Name Type In Description
name string body Required. The team's name.
agent_options string body Required. The agent options spec that is applied to the hosts assigned to the specified to team. These agent agent options completely override the global agent options specified in the GET /api/v1/fleet/config API route
secrets list body Required. A list of plain text strings used as the enroll secrets.

Example

POST /api/v1/fleet/spec/teams

Request body
{
  "specs": [
    {
      "name": "Client Platform Engineering",
      "agent_options": {
        "spec": {
          "config": {
            "options": {
              "logger_plugin": "tls",
              "pack_delimiter": "/",
              "logger_tls_period": 10,
              "distributed_plugin": "tls",
              "disable_distributed": false,
              "logger_tls_endpoint": "/api/v1/osquery/log",
              "distributed_interval": 10,
              "distributed_tls_max_attempts": 3
            },
            "decorators": {
              "load": [
                "SELECT uuid AS host_uuid FROM system_info;",
                "SELECT hostname AS hostname FROM system_info;"
              ]
            }
          },
          "overrides": {}
        }
      },
      "secrets": [
        {
          "secret": "fTp52/twaxBU6gIi0J6PHp8o5Sm1k1kn"
        },
        {
          "secret": "bhD5kiX2J+KBgZSk118qO61ZIdX/v8On"
        }
      ]
    }
  ]
}

Default response

Status: 200

Apply labels spec

Applies the supplied labels specs to Fleet. Each label requires the name, and label_membership_type properties.

If the label_membership_type is set to dynamic, the query property must also be specified with the value set to a query in SQL syntax.

If the label_membership_type is set to manual, the hosts property must also be specified with the value set to a list of hostnames.

POST /api/v1/fleet/spec/labels

Parameters

Name Type In Description
specs list path A list of the label to apply. Each label requires the name, query, and label_membership_type properties

Example

POST /api/v1/fleet/spec/labels

Request body
{
  "specs": [
    {
      "name": "Ubuntu",
      "description": "Filters ubuntu hosts",
      "query": "select 1 from os_version where platform = 'ubuntu';",
      "label_membership_type": "dynamic"
    },
    {
      "name": "local_machine",
      "description": "Includes only my local machine",
      "label_membership_type": "manual",
      "hosts": ["snacbook-pro.local"]
    }
  ]
}
Default response

Status: 200

Get labels spec

GET /api/v1/fleet/spec/labels

Parameters

None.

Example

GET /api/v1/fleet/spec/labels

Default response

Status: 200

{
  "specs": [
    {
      "id": 6,
      "name": "All Hosts",
      "description": "All hosts which have enrolled in Fleet",
      "query": "select 1;",
      "label_type": "builtin",
      "label_membership_type": "dynamic"
    },
    {
      "id": 7,
      "name": "macOS",
      "description": "All macOS hosts",
      "query": "select 1 from os_version where platform = 'darwin';",
      "platform": "darwin",
      "label_type": "builtin",
      "label_membership_type": "dynamic"
    },
    {
      "id": 8,
      "name": "Ubuntu Linux",
      "description": "All Ubuntu hosts",
      "query": "select 1 from os_version where platform = 'ubuntu';",
      "platform": "ubuntu",
      "label_type": "builtin",
      "label_membership_type": "dynamic"
    },
    {
      "id": 9,
      "name": "CentOS Linux",
      "description": "All CentOS hosts",
      "query": "select 1 from os_version where platform = 'centos' or name like '%centos%'",
      "label_type": "builtin",
      "label_membership_type": "dynamic"
    },
    {
      "id": 10,
      "name": "MS Windows",
      "description": "All Windows hosts",
      "query": "select 1 from os_version where platform = 'windows';",
      "platform": "windows",
      "label_type": "builtin",
      "label_membership_type": "dynamic"
    },
    {
      "id": 11,
      "name": "Ubuntu",
      "description": "Filters ubuntu hosts",
      "query": "select 1 from os_version where platform = 'ubuntu';",
      "label_membership_type": "dynamic"
    }
  ]
}

Get label spec

Returns the spec for the label specified by name.

GET /api/v1/fleet/spec/labels/{name}

Parameters

None.

Example

GET /api/v1/fleet/spec/labels/local_machine

Default response

Status: 200

{
  "specs": {
    "id": 12,
    "name": "local_machine",
    "description": "Includes only my local machine",
    "query": "",
    "label_membership_type": "manual"
  }
}

Get enroll secrets

Returns the valid global enroll secrets.

GET /api/v1/fleet/spec/enroll_secret

Parameters

None.

Example

GET /api/v1/fleet/spec/enroll_secret

Default response

Status: 200

{
  "spec": {
    "secrets": [
      {
        "secret": "fTp52/twaxBU6gIi0J6PHp8o5Sm1k1kn",
        "created_at": "2021-01-07T19:40:04Z"
      },
      {
        "secret": "bhD5kiX2J+KBgZSk118qO61ZIdX/v8On",
        "created_at": "2021-01-04T21:18:07Z"
      }
    ]
  }
}

Modify enroll secrets

Replaces the active global enroll secrets with the secrets specified.

POST /api/v1/fleet/spec/enroll_secret

Parameters

Name Type In Description
secret string body Required. The plain text string used as the enroll secret.

Example

Request body
{
  "spec": {
    "secrets": [
      {
        "secret": "fTp52/twaxBU6gIi0J6PHp8o5Sm1k1kn"
      }
    ]
  }
}

POST /api/v1/fleet/spec/enroll_secret

Default response

Status: 200

Live query health check

Checks the status of the Fleet's ability to run a live query. If an error is present in the response, Fleet won't be able to successfully run a live query. This endpoint is used by the Fleet UI to make sure that the Fleet instance is correctly configured to run live queries.

GET /api/v1/fleet/status/live_query

Parameters

None.

Example

GET /api/v1/fleet/status/live_query

Default response

Status: 200

Live query result store health check

Checks the status of the Fleet's result store. If an error is present in the response, Fleet won't be able to successfully run a live query. This endpoint is used by the Fleet UI to make sure that the Fleet instance is correctly configured to run live queries.

GET /api/v1/fleet/status/result_store

Parameters

None.

Example

GET /api/v1/fleet/status/result_store

Default response

Status: 200

Run live query

Runs the specified query as a live query on the specified hosts or group of hosts. Returns a new live query campaign. Individual hosts must be specified with the host's ID. Groups of hosts are specified by label ID.

After the query has been initiated, get results via WebSocket.

POST /api/v1/fleet/queries/run

Parameters

Name Type In Description
query string body The SQL if using a custom query.
query_id integer body The saved query (if any) that will be run. Required if running query as an observer. The observer_can_run property on the query effects which targets are included.
selected object body Required. The desired targets for the query specified by ID. This object can contain hosts, labels, and/or teams properties. See examples below.

One of query and query_id must be specified.

Example with one host targeted by ID

POST /api/v1/fleet/queries/run

Request body
{
  "query": "select instance_id from system_info",
  "selected": {
    "hosts": [171]
  }
}
Default response

Status: 200

{
  "campaign": {
    "created_at": "0001-01-01T00:00:00Z",
    "updated_at": "0001-01-01T00:00:00Z",
    "Metrics": {
      "TotalHosts": 1,
      "OnlineHosts": 0,
      "OfflineHosts": 1,
      "MissingInActionHosts": 0,
      "NewHosts": 1
    },
    "id": 1,
    "query_id": 3,
    "status": 0,
    "user_id": 1
  }
}

Example with multiple hosts targeted by label ID

POST /api/v1/fleet/queries/run

Request body
{
  "query": "select instance_id from system_info;",
  "selected": {
    "labels": [7]
  }
}
Default response

Status: 200

{
  "campaign": {
    "created_at": "0001-01-01T00:00:00Z",
    "updated_at": "0001-01-01T00:00:00Z",
    "Metrics": {
      "TotalHosts": 102,
      "OnlineHosts": 0,
      "OfflineHosts": 24,
      "MissingInActionHosts": 0,
      "NewHosts": 0
    },
    "id": 2,
    "query_id": 3,
    "status": 0,
    "user_id": 1
  }
}

Run live query by name

Runs the specified saved query as a live query on the specified targets. Returns a new live query campaign. Individual hosts must be specified with the host's hostname. Groups of hosts are specified by label name.

After the query has been initiated, get results via WebSocket.

POST /api/v1/fleet/queries/run_by_names

Parameters

Name Type In Description
query string body The SQL of the query.
query_id integer body The saved query (if any) that will be run. The observer_can_run property on the query effects which targets are included.
selected object body Required. The desired targets for the query specified by name. This object can contain hosts, labels, and/or teams properties. See examples below.

One of query and query_id must be specified.

Example with one host targeted by hostname

POST /api/v1/fleet/queries/run_by_names

Request body
{
  "query_id": 1,
  "selected": {
    "hosts": ["macbook-pro.local"]
  }
}
Default response

Status: 200

{
  "campaign": {
    "created_at": "0001-01-01T00:00:00Z",
    "updated_at": "0001-01-01T00:00:00Z",
    "Metrics": {
      "TotalHosts": 1,
      "OnlineHosts": 0,
      "OfflineHosts": 1,
      "MissingInActionHosts": 0,
      "NewHosts": 1
    },
    "id": 1,
    "query_id": 3,
    "status": 0,
    "user_id": 1
  }
}

Example with multiple hosts targeted by label name

POST /api/v1/fleet/queries/run_by_names

Request body
{
  "query": "select instance_id from system_info",
  "selected": {
    "labels": ["All Hosts"]
  }
}
Default response

Status: 200

{
  "campaign": {
    "created_at": "0001-01-01T00:00:00Z",
    "updated_at": "0001-01-01T00:00:00Z",
    "Metrics": {
      "TotalHosts": 102,
      "OnlineHosts": 0,
      "OfflineHosts": 24,
      "MissingInActionHosts": 0,
      "NewHosts": 1
    },
    "id": 2,
    "query_id": 3,
    "status": 0,
    "user_id": 1
  }
}

Retrieve live query results (standard WebSocket API)

You can retrieve the results of a live query using the standard WebSocket API.

Before you retrieve the live query results, you must create a live query campaign by running the live query. Use the Run live query or Run live query by name endpoints to create a live query campaign.

Note that live queries are automatically cancelled if this method is not called to start retrieving the results within 60 seconds of initiating the query.

/api/v1/fleet/results/websockets

Parameters

Name Type In Description
token string Required. The token used to authenticate with the Fleet API.
campaignID integer Required. The ID of the live query campaign.

Example

Example script to handle request and response

const socket = new WebSocket('wss://<your-base-url>/api/v1/fleet/results/websocket');

socket.onopen = () => {
  socket.send(JSON.stringify({ type: 'auth', data: { token: <auth-token> } }));
  socket.send(JSON.stringify({ type: 'select_campaign', data: { campaign_id: <campaign-id> } }));
};

socket.onmessage = ({ data }) => {
  console.log(data);
  const message = JSON.parse(data);
  if (message.type === 'status' && message.data.status === 'finished') {
    socket.close();
  }
}

Detailed request and response walkthrough with example data

webSocket.onopen()

Response data
o

webSocket.send()

Request data
[
  {
    "type": "auth",
    "data": { "token": <insert_token_here> }
  }
]
[
  {
    "type": "select_campaign",
    "data": { "campaign_id": 12 }
  }
]

webSocket.onmessage()

Response data
// Sends the total number of hosts targeted and segments them by status

[
  {
    "type": "totals",
    "data": {
      "count": 24,
      "online": 6,
      "offline": 18,
      "missing_in_action": 0
    }
  }
]
// Sends the expected results, actual results so far, and the status of the live query

[
  {
    "type": "status",
    "data": {
      "expected_results": 6,
      "actual_results": 0,
      "status": "pending"
    }
  }
]
// Sends the result for a given host

[
  {
    "type": "result",
    "data": {
      "distributed_query_execution_id": 39,
      "host": {
        // host data
      },
      "rows": [
        // query results data for the given host
      ],
      "error": null
    }
  }
]
// Sends the status of "finished" when messages with the results for all expected hosts have been sent

[
  {
    "type": "status",
    "data": {
      "expected_results": 6,
      "actual_results": 6,
      "status": "finished"
    }
  }
]

Retrieve live query results (SockJS)

You can also retrieve live query results with a SockJS client. The script to handle the request and response messages will look similar to the standard WebSocket API script with slight variations. For example, the constructor used for SockJS is SockJS while the constructor used for the standard WebSocket API is WebSocket.

Note that SockJS has been found to be substantially less reliable than the standard WebSockets approach.

/api/v1/fleet/results/

Parameters

Name Type In Description
token string Required. The token used to authenticate with the Fleet API.
campaignID integer Required. The ID of the live query campaign.

Example

Example script to handle request and response

const socket = new SockJS(`<your-base-url>/api/v1/fleet/results`, undefined, {});

socket.onopen = () => {
  socket.send(JSON.stringify({ type: 'auth', data: { token: <token> } }));
  socket.send(JSON.stringify({ type: 'select_campaign', data: { campaign_id: <campaignID> } }));
};

socket.onmessage = ({ data }) => {
  console.log(data);
  const message = JSON.parse(data);

  if (message.type === 'status' && message.data.status === 'finished') {
    socket.close();
  }
}
Detailed request and response walkthrough

socket.onopen()

Response data
o

socket.send()

Request data
[
  {
    "type": "auth",
    "data": { "token": <insert_token_here> }
  }
]
[
  {
    "type": "select_campaign",
    "data": { "campaign_id": 12 }
  }
]

socket.onmessage()

Response data
// Sends the total number of hosts targeted and segments them by status

[
  {
    "type": "totals",
    "data": {
      "count": 24,
      "online": 6,
      "offline": 18,
      "missing_in_action": 0
    }
  }
]
// Sends the expected results, actual results so far, and the status of the live query

[
  {
    "type": "status",
    "data": {
      "expected_results": 6,
      "actual_results": 0,
      "status": "pending"
    }
  }
]
// Sends the result for a given host

[
  {
    "type": "result",
    "data": {
      "distributed_query_execution_id": 39,
      "host": {
        // host data
      },
      "rows": [
        // query results data for the given host
      ],
      "error": null
    }
  }
]
// Sends the status of "finished" when messages with the results for all expected hosts have been sent

[
  {
    "type": "status",
    "data": {
      "expected_results": 6,
      "actual_results": 6,
      "status": "finished"
    }
  }
]

Apply policies spec

Creates and/or modifies the policies included in the specs list. To modify an existing policy, the name of the query included in specs must already be used by an existing policy. If a policy with the specified name doesn't exist in Fleet, a new policy will be created.

NOTE: when updating a policy, team and platform will be ignored.

POST /api/v1/fleet/spec/policies

Parameters

Name Type In Description
specs list body Required. The list of the policies to be created or modified.

Example

POST /api/v1/fleet/spec/policies

Request body
{
  "specs": [
    {
      "name": "new policy",
      "description": "This will be a new policy because a policy with the name 'new policy' doesn't exist in Fleet.",
      "query": "SELECT * FROM osquery_info",
      "resolution": "some resolution steps here"
    },
    {
      "name": "Is Filevault enabled on macOS devices?",
      "query": "SELECT 1 FROM disk_encryption WHERE user_uuid IS NOT “” AND filevault_status = on LIMIT 1;",
      "description": "Checks to make sure that the Filevault feature is enabled on macOS devices.",
      "resolution": "Choose Apple menu > System Preferences, then click Security & Privacy. Click the FileVault tab. Click the Lock icon, then enter an administrator name and password. Click Turn On FileVault.",
      "platform": "darwin"
    }
  ]
}
Default response

Status: 200