Commit Graph

36 Commits

Author SHA1 Message Date
Zach Wasserman
26eae438f6
Document supported MySQL versions (#5421)
- Pin versions in development and CI.
2022-04-27 16:21:16 -07:00
Martin Angers
fd0cd153ce
Fix SSO paths to always use /v1/ instead of /latest/ (#5246) 2022-04-20 12:46:45 -04:00
Martin Angers
90b15071a4
Introduce API version 2022-04, deprecate use of /global in paths (#4731) 2022-04-05 11:35:53 -04:00
Zach Wasserman
35523017c9
Make migrations compatible with GTID replication (#2615)
* Make migrations compatible with GTID replication

Fixes an issue some deployments encountered when migrations used a
statement that is unsupported in GTID replication mode (#2462).

Local dev MySQL now enforces this consistency, so it should be easier to
maintain compatibility going forward.

* Update docker-compose formatting

* if exists
2021-10-21 07:46:02 -03:00
Martin Angers
eda6d60be8
Add replicas to the redis docker-compose setup. (#2511) 2021-10-18 08:33:26 -04:00
Benjamin Edwards
16c5823692
add support for minio backend file carving (#2448)
* add support for minio backend file carving
* add changes file
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-10-12 15:32:06 -04:00
Martin Angers
9a0871a2f1
Address issues related to Redis Cluster support (#1885)
Closes #1847 .
2021-09-01 16:32:57 -04:00
Benjamin Edwards
c18214be37
add logging settings to config api response (#1467)
- add docker-compose file for locally testing aws dependencies
- update firehose & kinesis configs to optionally supply endpoint url override
- serialize `logging` field in appconfig api response
2021-07-30 11:45:49 -04:00
Tomas Touceda
a6cff7ea89
Migrate all mysql tests to the new form (#1408)
* Migrate all mysql tests to the new form

* Only dump sql if MYSQL_TEST is on

* Removing parallel until we get rid of this code

* Move TestMain to an actual _test file

* A little experiment with tmpfs to speed up the db

* Let's make sure the dump.sql file is also in ram
2021-07-19 18:20:31 -03:00
Zach Wasserman
8be9068c1c
Fix migration for MariaDB compatibility (#1280)
Give unique names to each new foreign key in the migrations.

Fixes #1279
2021-07-01 15:59:31 -07:00
Tomas Touceda
8e2db1fecf
For platform on mysql so it runs in Apple M1 macbooks (#1274) 2021-07-01 13:09:51 -03:00
Zach Wasserman
8c97b36764
Set up Cypress testing for Teams/Tiers (#1005)
- Update names/roles of users in `make e2e-setup`.
- Update test SSO user info.
- Add Cypress commands for seeding users/Teams.
- Stub Cypress tests for team/tier matrix.
2021-06-09 11:56:59 -07:00
Zach Wasserman
6860dec81a
Update flags for development cAdvisor (#679)
Updates to a minimal set of permissions for working cAdvisor.

Tested with Docker Desktop 3.3.1 on macOS 11.2.3
2021-04-23 17:58:23 -07:00
Zach Wasserman
e961cfe0c3
Batch updates to host seen time (#633)
Instead of synchronously updating the seen_time column for a host on an update, batch these updates to be written together every 1 second.

This results in a ~33% reduction in MySQL CPU usage in a local test with 4,000 simulated hosts and MySQL running in Docker.
2021-04-12 16:22:22 -07:00
Zach Wasserman
9f71fcf440
Speed up MySQL tests (#585)
Improves MySQL test time (on my 2020 MBP) to ~18s from ~125s.

- Use separate databases for each test to allow parallelization.
- Run migrations only once at beginning of tests and then reload
  generated schema.
- Add `--innodb-file-per-table=OFF` for ~20% additional speedup.
2021-04-03 11:42:27 -07:00
Zach Wasserman
ee580bf811
Add ability to test SSO locally (#552)
- Use IdP in Docker container.
- Document process.
2021-03-29 13:50:39 -07:00
Zach Wasserman
c5f931bb54
Bump development docker-compose Redis to version 5 (#414) 2021-03-08 14:36:10 -08:00
Zach Wasserman
8fcd14b394
Add --dev flag and change docker defaults (#251)
- Add --dev flag that will set default flag values. This simplifies the
  invocation of Fleet in a development environment.
- Change defaults in docker-compose to use `fleet` in place of `kolide`.
-  Skip prompt in `prepare db` when `--dev` specified.
- Update developer documentation.

Updates to MySQL configuration in docker-compose.yml may require
existing development containers and volumes to be deleted (this will
delete data in MySQL):

```shell
docker-compose rm -sf
docker volume rm fleet_mysql-persistent-volume
```

Closes #170
2021-02-01 18:14:16 -08:00
Zach Wasserman
6cbd10965c
Add dev infrastructure and docs for Prometheus monitoring (#33)
- Set up a simple example of Prometheus monitoring in the development
  docker-compose.yml.
- Add documentation for configuring Prometheus.
2020-11-12 19:06:56 -08:00
Brendan Shaklovitz
87137252de
docker-compose QoL improvements (#2319)
* Use YAML anchors to avoid repeating config blocks
* Use docker volumes to persist data for mysql
* Allow setting `FLEET_SERVER` (fixes #2127) when using the docker-compose file to spin up multiple osquery clients
2020-10-09 10:10:33 -07:00
Zachary Wasserman
2bfcecf0d3
Add CAdvisor container to dev docker-compose (#2263)
This container is useful for monitoring stats of other containers.
2020-07-21 15:36:11 -07:00
Austin Burnett
59efb495ca Add automatic host expiration capability (#2117)
When configured, this feature will delete hosts that have not checked in after the specified number of days.

Closes #1860.
2019-10-16 16:35:17 -07:00
Zachary Wasserman
07e7d336dd Enable slow query log on dev DBs (#1437)
With this change, MySQL will log "slow" queries to the `mysql.slow_log` table.
2017-03-20 17:02:44 -07:00
Zachary Wasserman
c0d87cb69a Use persistent MySQL for local server, tmpfs MySQL for tests (#1245) 2017-02-17 09:15:10 -08:00
John Murphy
56195b0768 Changed mysql version to match that of circle.yaml (#520) 2016-11-21 21:59:48 +08:00
Zachary Wasserman
1d06ad80c8 Use tmpfs with dev MySQL instances (#483)
This results in ~100x speedup in MySQL tests.
2016-11-15 10:06:33 -08:00
Zachary Wasserman
262a48f8eb Datastores for buffering distributed query results (#346)
A new datastore interface is needed for buffering incoming distributed query results to be sent to the client. This PR attempts to define and implement that interface.

It is intended that the ReadChannel() method be used by the goroutine that will push query results down a websocket to the client. Passing the results through this channel will allow that goroutine to perform a select on both the channel and the websocket, in order to properly handle IO.
2016-10-31 15:51:19 -07:00
Mike Arpaia
d5d3d4ce17 Updating some docker compose functionality (#220)
close #210
2016-09-21 08:22:53 -04:00
Mike Arpaia
5b78c0a7cc Organizing cobra commands (#100) 2016-09-03 13:25:16 -04:00
Victor Vrantchan
2946eeabfc add Dockerfile (#94)
* add Dockerfile to build
2016-08-25 09:25:02 -04:00
Victor Vrantchan
578b81558c add build container for testing in ci (#89)
build fails if container exit code fails. 
One step closer to #71
2016-08-19 20:05:24 -04:00
Mike Arpaia
736bce5033 Email-based password reset (#54)
* No more hard deletes

* scaffolding for password reset endpoint

* Ensure password reset state is accounted for in VC checks

* password reset endpoints and data structures

* ability to change password with reset token

* smtp server connection pool management

* stubbing out the sending of the email

* adding mailhog via docker

* HTML emails with confgurable host name

* fixing typo in the comments

* Fixing merge which undid DatabaseError replacement

* documentation in the readme

* webpack shortcut for components

* removing a sneaky merge line that snuck in

* temporary email content api

* tests for password reset flow

* fixing go vet

* comments and making all db use `&value` rather than `reference`

* more correct usage of the errors library and moving email sending to it's own method

* using the wrong error

* fixing email mock object error

* less incorrect error usage

* rebasing and merging

* http constants for status code

* using ParseAndValidateJSON instead of BindJSON

* validate instead of binding in struct tags

* NewFromError instead of New
2016-08-12 12:20:29 -07:00
Mike Arpaia
b6ac5b1bdb Updating README and documentation to reflect local building and testing (#58) 2016-08-09 19:17:07 -07:00
Jason Meller
d9f776c756 Use Godep to vendor in all dependencies (#40)
This commit vendors in all of our dependencies using
[GoDep](https://github.com/tools/godep). We are forgoing using a vendor/ folder to avoid checking in deps into the repo.

Note: Never manually modify `Godeps/Godeps.json` this file is dynamically
by the godep CLI

Common Actions:

To add a new package foo/bar, do this:

1. Run `go get foo/bar`
1. Edit your code to import foo/bar.
1. Run `godep save` (or `godep save ./...`).

To update a package from your `$GOPATH`, do this:

1. Run `go get -u foo/bar`
1. Run `godep update foo/bar`. (You can use the `...` wildcard, for example
`godep update foo/...`).
2016-08-04 16:57:22 -04:00
Jason Meller
5ad7c07e0c Dockerize app (#32)
This commit adds both a Dockerfile and updates the docker-compose.yml with local mounting so that you can standup a consistent dev environment. Please view the project README for more information.
2016-08-04 11:41:18 -04:00
Mike Arpaia
91e78d276f Initial commit 2016-07-28 18:34:26 -04:00