Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
Go to file
Omer Lachish 5a5fdecdde
Replace Celery with RQ (except for execute_query tasks) (#4093)
* add rq and an rq_worker service

* add rq_scheduler and an rq_scheduler service

* move beat schedule to periodic_jobs queue

* move version checks to RQ

* move query result cleanup to RQ

* use timedelta and DRY up a bit

* move custom tasks to RQ

* do actual schema refreshes in rq

* rename 'period_jobs' to 'periodic', as it obviously holds jobs

* move send_email to rq

* DRY up enqueues

* ditch  and use a partially applied  decorator

* move subscribe to rq

* move check_alerts_for_query to rq

* move record_event to rq

* make tests play nicely with rq

* 👋 beat

* rename rq_scheduler to plain scheduler, now that there's no Celery scheduler entrypoint

* add some color to rq-worker's output

* add logging context to rq jobs (while keeping execute_query context via get_task_logger for now)

* move schedule to its own module

* cancel previously scheduled periodic jobs. not sure this is a good idea.

* rename redash.scheduler to redash.schedule

* allow custom dynamic jobs to be added decleratively

* add basic monitoring to rq queues

* add worker monitoring

* pleasing the CodeClimate overlords

* adjust cypress docker-compose.yml to include rq changes

* DRY up Cypress docker-compose

* add rq dependencies to cypress docker-compose service

* an odd attempt at watching docker-compose logs when running with Cypress

* Revert "an odd attempt at watching docker-compose logs when running with Cypress"

This reverts commit 016bd1a93e3efa84a9f27d0f2acb972ce1957bcd.

* show docker-compose logs at Cypress shutdown

* Revert "DRY up Cypress docker-compose"

This reverts commit 43abac7084c207ab9e39192ac79d520448c2c527.

* minimal version for binding is 3.2

* remove unneccesary code reloads on cypress

* add a  command which errors if any of the workers running inside the current machine haven't been active in the last minute

* SCHEMAS_REFRESH_QUEUE is no longer a required setting

* split tasks/queries.py to execution.py and maintenance.py

* fix tests after query execution split

* pleasing the CodeClimate overlords

* rename worker to celery_worker and rq_worker to worker

* use /rq_status instead of /jobs

* show started jobs' time ago according to UTC

* replace all spaces in column names

* fix query tests after execution split

* exit with an int

* general lint

* add an entrypoint for rq_healthcheck

* fix indentation

* delete all existing periodic jobs before scheduling them

* remove some unrequired requires

* move schedule example to redash.schedule

* add RQ integration to Sentry's setup

* pleasing the CodeClimate overlords

* remove replication settings from docker-compose - a proper way to scale using docker-compose would be the --scale CLI option, which will be described in the knowledge based

* revert to calling a function in dynamic settings to allow periodic jobs to be scheduled after app has been loaded

* don't need to depend on context when templating failure reports

* set the timeout_ttl to double the interval to avoid job results from expiring and having periodic jobs not reschedule

* whoops, bad merge

* describe custom jobs and don't actually schedule them

* fix merge
2019-10-15 23:59:22 +03:00
.circleci Replace Celery with RQ (except for execute_query tasks) (#4093) 2019-10-15 23:59:22 +03:00
.github Create weekly-digest.yml 2019-03-10 11:39:13 +02:00
bin Replace Celery with RQ (except for execute_query tasks) (#4093) 2019-10-15 23:59:22 +03:00
client Replace Celery with RQ (except for execute_query tasks) (#4093) 2019-10-15 23:59:22 +03:00
migrations Migrate with SQL statements. (#4105) 2019-08-30 14:08:22 +03:00
redash Replace Celery with RQ (except for execute_query tasks) (#4093) 2019-10-15 23:59:22 +03:00
setup Use REDASH_BASE_PATH everywhere instead of hardcoded path (#3740) 2019-04-29 14:28:16 +03:00
tests Replace Celery with RQ (except for execute_query tasks) (#4093) 2019-10-15 23:59:22 +03:00
.codeclimate.yml Add Lint step to CircleCI (#3642) 2019-03-26 16:40:26 +02:00
.coveragerc Move to pytest since nose is unmaintained. 2017-10-18 13:51:09 +02:00
.dockerignore Ignore to copy some files onto docker container (#3388) 2019-02-03 20:30:08 +02:00
.editorconfig Convert query control dropdown button to React (#3698) 2019-05-01 07:20:54 +03:00
.gitignore Move cypress to client folder (#3566) 2019-03-24 11:24:59 -03:00
CHANGELOG.md CHANGELOG for v8.0.0-beta.2 (#4145) 2019-09-18 11:23:32 +03:00
CONTRIBUTING.md fix broken link in CONTRIBUTING.md (#3483) 2019-02-22 14:28:13 -03:00
cypress.json Migrate Box Plot visualization to React (#3948) 2019-07-04 22:25:09 +03:00
docker-compose.yml Replace Celery with RQ (except for execute_query tasks) (#4093) 2019-10-15 23:59:22 +03:00
Dockerfile Dockerfile front end stage copies client side files only (#3924) 2019-06-23 11:33:52 +03:00
LICENSE It's 2019 now ! 🎉 2019-01-01 08:40:51 +02:00
Makefile Add "Active at" column to user list. (#3026) 2019-01-07 10:30:42 +02:00
manage.py Move CLI logic into redash.cli and uses manager for tests. 2016-12-07 14:58:33 +02:00
netlify.toml Add netlify config (#2999) 2018-10-28 15:31:27 +02:00
package-lock.json Sortable component (#4199) 2019-09-30 19:12:27 +03:00
package.json Sortable component (#4199) 2019-09-30 19:12:27 +03:00
pytest.ini Remove missing coverage from pytest terminal output (#3180) 2018-12-12 08:30:46 +02:00
README.md Update badge in README.md to link to CircleCI (#4104) 2019-09-01 10:50:14 +03:00
requirements_all_ds.txt Adjust botocore dependency, so we don't need to update it as often (#4154) 2019-10-06 11:47:16 +03:00
requirements_bundles.txt Require a more up-to-date version of importlib-metadata. (#4069) 2019-08-15 11:50:27 +03:00
requirements_dev.txt Adjust botocore dependency, so we don't need to update it as often (#4154) 2019-10-06 11:47:16 +03:00
requirements_oracle_ds.txt Update oracle client version 2017-09-13 17:40:44 +02:00
requirements.txt Replace Celery with RQ (except for execute_query tasks) (#4093) 2019-10-15 23:59:22 +03:00
SECURITY.md Create SECURITY.md (#3823) 2019-05-26 09:32:41 +03:00
setup.cfg Add "Active at" column to user list. (#3026) 2019-01-07 10:30:42 +02:00
webpack.config.js Turned off max asset size warning (#4023) 2019-07-31 11:34:52 +03:00

Documentation Datree Build Status

Redash is our take on freeing the data within our company in a way that will better fit our culture and usage patterns.

Prior to Redash, we tried to use traditional BI suites and discovered a set of bloated, technically challenged and slow tools/flows. What we were looking for was a more hacker'ish way to look at data, so we built one.

Redash was built to allow fast and easy access to billions of records, that we process and collect using Amazon Redshift ("petabyte scale data warehouse" that "speaks" PostgreSQL). Today Redash has support for querying multiple databases, including: Redshift, Google BigQuery, PostgreSQL, MySQL, Graphite, Presto, Google Spreadsheets, Cloudera Impala, Hive and custom scripts.

Redash consists of two parts:

  1. Query Editor: think of JS Fiddle for SQL queries. It's your way to share data in the organization in an open way, by sharing both the dataset and the query that generated it. This way everyone can peer review not only the resulting dataset but also the process that generated it. Also it's possible to fork it and generate new datasets and reach new insights.
  2. Visualizations and Dashboards: once you have a dataset, you can create different visualizations out of it, and then combine several visualizations into a single dashboard. Currently Redash supports charts, pivot table, cohorts and more.

Getting Started

Supported Data Sources

Redash supports more than 35 data sources.

Getting Help

Reporting Bugs and Contributing Code

  • Want to report a bug or request a feature? Please open an issue.
  • Want to help us build Redash? Fork the project, edit in a dev environment, and make a pull request. We need all the help we can get!

Security

Please email security@redash.io to report any security vulnerabilities. We will acknowledge receipt of your vulnerability and strive to send you regular updates about our progress. If you're curious about the status of your disclosure please feel free to email us again. If you want to encrypt your disclosure email, you can use this PGP key.

License

BSD-2-Clause.