redash/bin
Omer Lachish 5a5fdecdde
Replace Celery with RQ (except for execute_query tasks) (#4093)
* add rq and an rq_worker service

* add rq_scheduler and an rq_scheduler service

* move beat schedule to periodic_jobs queue

* move version checks to RQ

* move query result cleanup to RQ

* use timedelta and DRY up a bit

* move custom tasks to RQ

* do actual schema refreshes in rq

* rename 'period_jobs' to 'periodic', as it obviously holds jobs

* move send_email to rq

* DRY up enqueues

* ditch  and use a partially applied  decorator

* move subscribe to rq

* move check_alerts_for_query to rq

* move record_event to rq

* make tests play nicely with rq

* 👋 beat

* rename rq_scheduler to plain scheduler, now that there's no Celery scheduler entrypoint

* add some color to rq-worker's output

* add logging context to rq jobs (while keeping execute_query context via get_task_logger for now)

* move schedule to its own module

* cancel previously scheduled periodic jobs. not sure this is a good idea.

* rename redash.scheduler to redash.schedule

* allow custom dynamic jobs to be added decleratively

* add basic monitoring to rq queues

* add worker monitoring

* pleasing the CodeClimate overlords

* adjust cypress docker-compose.yml to include rq changes

* DRY up Cypress docker-compose

* add rq dependencies to cypress docker-compose service

* an odd attempt at watching docker-compose logs when running with Cypress

* Revert "an odd attempt at watching docker-compose logs when running with Cypress"

This reverts commit 016bd1a93e3efa84a9f27d0f2acb972ce1957bcd.

* show docker-compose logs at Cypress shutdown

* Revert "DRY up Cypress docker-compose"

This reverts commit 43abac7084c207ab9e39192ac79d520448c2c527.

* minimal version for binding is 3.2

* remove unneccesary code reloads on cypress

* add a  command which errors if any of the workers running inside the current machine haven't been active in the last minute

* SCHEMAS_REFRESH_QUEUE is no longer a required setting

* split tasks/queries.py to execution.py and maintenance.py

* fix tests after query execution split

* pleasing the CodeClimate overlords

* rename worker to celery_worker and rq_worker to worker

* use /rq_status instead of /jobs

* show started jobs' time ago according to UTC

* replace all spaces in column names

* fix query tests after execution split

* exit with an int

* general lint

* add an entrypoint for rq_healthcheck

* fix indentation

* delete all existing periodic jobs before scheduling them

* remove some unrequired requires

* move schedule example to redash.schedule

* add RQ integration to Sentry's setup

* pleasing the CodeClimate overlords

* remove replication settings from docker-compose - a proper way to scale using docker-compose would be the --scale CLI option, which will be described in the knowledge based

* revert to calling a function in dynamic settings to allow periodic jobs to be scheduled after app has been loaded

* don't need to depend on context when templating failure reports

* set the timeout_ttl to double the interval to avoid job results from expiring and having periodic jobs not reschedule

* whoops, bad merge

* describe custom jobs and don't actually schedule them

* fix merge
2019-10-15 23:59:22 +03:00
..
bundle-extensions Fix loading of periodic tasks and clean up extension loading. (#4064) 2019-08-13 13:11:59 +03:00
docker-entrypoint Replace Celery with RQ (except for execute_query tasks) (#4093) 2019-10-15 23:59:22 +03:00
flake8_tests.sh Add more flake8 tests and fail build if any test fails (#4055) 2019-08-18 11:27:44 +03:00
get_changes.py CircleCI: Flake8 tests passing on Legacy Python and Python 3 (#2881) 2018-10-05 13:48:01 +03:00
pre_compile Update Heroku pre_compile hook 2016-06-14 14:23:56 +03:00
release_manager.py Consistently use simplejson to loading and dumping JSON. (#2817) 2018-10-09 15:38:06 +02:00
run Fix: support for .env files without EXPORT 2014-04-20 09:43:17 +03:00
upgrade Fix upgrade script to support CI builds 2018-02-08 11:35:12 +02:00