Commit Graph

74 Commits

Author SHA1 Message Date
Stefan Mees
940bd564d7
Datasource Exasol: support encryption setting (#4712)
* add pyexasol datasource, ensure that integer dont overflow in javascript

* support setting encryption for Exasol connections

Co-authored-by: Arik Fraimovich <arik@arikfr.com>
2020-04-14 12:38:01 +03:00
Arik Fraimovich
f1a2f8cb88
Enable ODBC and MS SQL ODBC support (#4676)
Closes #4356.
2020-02-23 11:57:39 +02:00
Jannis Leidel
5c7cb1af3d
Update cassandra-driver to 3.21.0. (#4636)
This provides binary wheel files and reduces Docker build times drastically.
2020-02-10 21:48:43 +02:00
Steve Buckingham
56b51be64a Add redshift role use option (#4532)
* Add redshift role use option

* Update requirements for SSL socket wrap issue fixes

* Split Redshift class into User and IAM logins

* Update incorrect register

* Change type names

* Correct class name to inherit

* Render IAM redshift image and field order correct

* Update redash/query_runner/pg.py

Co-Authored-By: Arik Fraimovich <arik@arikfr.com>

* Update redash/query_runner/pg.py

Co-Authored-By: Arik Fraimovich <arik@arikfr.com>

* Remove need for specified urllib - specify pyopenssl is enough

* Pyopenssl back down to 19.0.0

Co-authored-by: Arik Fraimovich <arik@arikfr.com>
2020-01-21 11:18:33 +02:00
Arik Fraimovich
a891160b4d
Upgrade snowflake-connector-python (#4567)
* Upgrade snowflake-connector-python

* Downgrade requests
2020-01-20 19:50:57 +02:00
Randy Zwitch
6b2f23f357 Update pymapd to 0.19.0 (#4424) 2019-12-24 15:34:42 +02:00
Jakdaw
3494e21cf4 Add user/pass authentication support for Druid (#4315)
* Add support for configuring a Username/Password for the connection to Druid

* Bump pydruid version for username/password support

* Deal with missing/empty configuration parameters
2019-12-05 09:27:59 +02:00
Arik Fraimovich
36ab8eae89
Update Snowflake connector version to address compatibility issue with Azure dependencies (#4407) 2019-11-27 18:44:08 +02:00
Stefan Mees
e82373ac1d add pyexasol datasource, ensure that integer dont overflow in javascript (#4378) 2019-11-27 18:43:58 +02:00
Nicolas Le Manchet
246eca1121 Migrate the application to Python 3 (#4251)
* Make core app compatible with Python 3

No backward compatibility with Python 2.7 is kept.
This commit mostly contains changes made with 2to3 and manual
tweaking when necessary.

* Use Python 3.7 as base docker image

Since it is not possible to change redash/base:debian to Python 3
without breaking future relases, its Dockerfile is temporarly
copied here.

* Upgrade some requirements to newest versions

Some of the older versions were not compatible with Python 3.

* Migrate tests to Python 3

* Build frontend on Python 3

* Make the HMAC sign function compatible with Python 3

In Python 3, HMAC only works with bytes so the strings and the
float used in the sign function need to be encoded.
Hopefully this is still backward compatible with already generated
signatures.

* Use assertCountEqual instead of assertItemsEqual

The latter is not available in Python 3.
See https://bugs.python.org/issue17866

* Remove redundant encoding header for Python 3 modules

* Remove redundant string encoding in CLI

* Rename list() functions in CLI

These functions shadow the builtin list function which is
problematic since 2to3 adds a fair amount of calls to the builtin
list when it finds dict.keys() and dict.values().

Only the Python function is renamed, from the perspective of the
CLI nothing changes.

* Replace usage of Exception.message in CLI

`message` is not available anymore, instead use the string
representation of the exception.

* Adapt test handlers to Python 3

* Fix test that relied on dict ordering

* Make sure test results are always uploaded (#4215)

* Support encoding memoryview to JSON

psycopg2 returns `buffer` objects in Python 2.7 and `memoryview`
in Python 3. See #3156

* Fix test relying on object address ordering

* Decode bytes returned from Redis

* Stop using e.message for most exceptions

Exception.message is not available in Python 3 anymore, except
for some exceptions defined by third-party libraries.

* Fix writing XLSX files in Python 3

The buffer for the file should be made of bytes and the actual
content written to it strings.

Note: I do not know why the diff is so large as it's only a two
lines change. Probably a white space or file encoding issue.

* Fix test by comparing strings to strings

* Fix another exception message unavailable in Python 3

* Fix export to CSV in Python 3

The UnicodeWriter is not used anymore. In Python 3, the interface
provided by the CSV module only deals with strings, in and out.
The encoding of the output is left to the user, in our case
it is given to Flask via `make_response`.

* (Python 3) Use Redis' decode_responses=True option (#4232)

* Fix test_outdated_queries_works_scheduled_queries_tracker (use utcnow)

* Make sure Redis connection uses decoded_responses option

* Remove unused imports.

* Use Redis' decode_responses option

* Remove cases of explicit Redis decoding

* Rename helper function and make sure it doesn't apply twice.

* Don't add decode_responses to Celery Redis connection URL

* Fix displaying error while connecting to SQLite

The exception message is always a string in Python 3, so no
need to try to decode things.

* Fix another missing exception message

* Handle JSON encoding for datasources returning bytes

SimpleJSON assumes the bytes it receives contain text data, so it
tries to UTF-8 encode them. It is sometimes not true, for instance
the SQLite datasource returns bytes for BLOB types, which typically
do not contain text but truly binary data.

This commit disables SimpleJSON auto encoding of bytes to str and
instead uses the same method as for memoryviews: generating a
hex representation of the data.

* Fix Python 3 compatibility with RQ

* Revert some changes 2to3 tends to do (#4261)

- Revert some changes 2to3 tends to do when it errs on the side of caution regarding dict view objects.

- Also fixed some naming issues with one character variables in list comprehensions.

- Fix Flask warning.

* Upgrade dependencies

* Remove useless `iter` added by 2to3

* Fix get_next_path tests (#4280)

* Removed setting SERVER_NAME in tests setup to avoid a warning.

* Change get_next_path to not return empty string in case of a domain only value.

* Fix redirect tests:

Since version 0.15 of Werkzeug it uses full path for fixing the location header instead of the root path.

* Remove explicit dependency for Werkzeug

* Switched pytz and certifi to unbinded versions.

* Switch to new library for getting country from IP

`python-geoip-geolite2` is not compatible with Python 3, instead
use `maxminddb-geolite2` which is very similar as it includes
the geolite2 database in the package .

* Python 3 RQ modifications (#4281)

* show current worker job (alongside with minor cosmetic column tweaks)

* avoid loading entire job data for queued jobs

* track general RQ queues (default, periodic and schemas)

* get all active RQ queues

* call get_celery_queues in another place

* merge dicts the Python 3 way

* extend the result_ttl of refresh_queries to 600 seconds to allow it to continue running periodically even after longer executions

* Remove legacy Python flake8 tests
2019-10-24 12:42:13 +03:00
Justin Clift
0a5dca5d72 Adjust botocore dependency, so we don't need to update it as often (#4154) 2019-10-06 11:47:16 +03:00
Arik Fraimovich
b2d1636f8e
Downgrade mysqlclient to 1.3.14 (#4165)
Closes #4164.
2019-09-22 14:56:09 +03:00
Arik Fraimovich
b8741f6cff
Sync botocor eversions across requirements files. (#4128) 2019-09-09 10:44:05 +03:00
Gleb Lesnikov
ef9a4d5eed [Data Sources] Add: Azure Data Explorer (Kusto) query runner (#4091)
* [Data Sources] Add: Azure Data Explorer (Kusto) query runner

* CodeClimate fixes

* Remove TODO

* Fixed configuration properties names for Azure Kusto

* Azure Kusto: get_schema in one query

* azure-kusto-data update to 0.0.32

* Add Kusto to the default query runners list
2019-08-26 10:17:49 +03:00
Arik Fraimovich
934a145ced
Switch to mysqlclient from Python-MySQL (#4061) 2019-08-14 10:11:53 +03:00
The Alchemist
69ba165565 [Data Sources] Initial commit for adding Dgraph support (#3987)
* Initial commit for adding Dgraph support

* Made suggestions from https://codeclimate.com/github/getredash/redash/pull/3964

* feedback from @arikfr

* added logo for Dgraph from Twitter

* Better conversion of Dgraph JSON to Redash's internal JSON

* made recommendations from @arikfr

* removed unused function
2019-08-13 13:14:37 +03:00
Arik Fraimovich
11cc274c1c
Update Snowflake connector version to latest (#4029) 2019-08-04 08:55:02 +03:00
koooge
c0e8ef3000 Upgrade gspread 3.1.0 for supporting team drive (#3838)
* Upgrade gspread 3.1.0 for supporting team drive

Signed-off-by: koooge <koooooge@gmail.com>

* Revert "Upgrade gspread 3.1.0 for supporting team drive"

This reverts commit e53e8cb75bc02a478c7f7d973beb0e559b7d6cae.

* Upgrade gspread 3.1.0 for supporting team drive

Signed-off-by: koooge <koooooge@gmail.com>

* Update Sheets query runner name
2019-06-06 11:02:08 +03:00
ialeinikov
7a7fdf9c99 allowing to specify a custom work group for AWS Athena queries (#3592)
* allowing to specify a custom work group for AWS Athena queries

* Fixing title + adding correct position in the UI
2019-03-27 17:58:48 +02:00
Byunghwa Yun
71afc99ec3 Add phoenix query runner. (#3153)
* Add phoenix query runner.

* Improved error handling.
2019-03-27 17:48:49 +02:00
Arik Fraimovich
61f143dfd3
Snowflake: add support for regions and enable by default (#3550)
* Bring back Snowflake from its exile.

* Snowflake: add support for regions.
2019-03-07 23:40:24 +02:00
Arik Fraimovich
4f79c86c0e
Pin pymapd version to 0.7.1 (#3543)
Newer versions dropped support for Python 2. Closes #3542.
2019-03-07 09:02:40 +02:00
Toshimitsu Takahashi
c9a4f07a7a Support AWS IAM profile for Amazon Elasticsearch (#3005) 2019-02-28 09:57:33 +02:00
Arik Fraimovich
60472e2fe0
Add support for Amazon ES service with IAM authentication (#3446)
* Add support for Amazon ES service with IAM authentication

* Add required dependency.
2019-02-17 15:11:40 +02:00
Jannis Leidel
bd559b6eeb Fix some incompatible dependencies (#3348)
* Pin requests-oauthlib to work around incompatible deptree.

* Update boto3/botocore to fix incompatible deptree.
2019-01-31 09:50:47 +02:00
Arik Fraimovich
ecb8a5c244
Hive/Databricks related improvements (#3143)
* Hive: fix issues in building options.

* Hive: add missing import.

* Split Hive into two query runners: one for http and a regular one.

* Upgrade pyhive to support http.

* Specific implementation for databricks:

* Different schema loader, because column names are different (or Hive's schema loader is broken).
* Simpler configuration.

* Simplify Databricks setup even more by removing username.
2018-11-29 13:01:23 +02:00
Arik Fraimovich
1cdfcfaa3c
Vertica: update driver & add support for connection timeout (#3125)
* Update Vertica driver version

* Vertica: add support for connection timeout
2018-11-26 21:21:49 +02:00
Omer Lachish
57d921dc2b Druid query runner (#3047)
* add Druid query runner skeleton

* enable Druid only if package is available

* add Druid

* remove redundant  override

* correct configuration schema

* implements run_query

* implement get_schema

* remove username and password

* fix small lint issues

* proper indentation

* add correct type mapping
2018-11-08 10:48:45 +01:00
Arik Fraimovich
7d1dbb87db
Change: update MongoDB requirements to support srv. (#2962) 2018-10-18 17:32:43 +03:00
Nicolas Ferrandini
e14c8b61a0 Add DB2 as a data source using ibm-db python package (#2848)
* Add DB2 as a data source using ibm-db python package

* fix some codeclimate issue

* fix long line and missing white space

* Manage case of bad import

* Add DB2 query_runner as default query runner

* Fixed minor PEP8 rules
2018-10-15 17:13:39 +03:00
Arik Fraimovich
99c73aef2d
Update snowflake_connector_python version (#2946) 2018-10-14 14:39:52 +03:00
Arik Fraimovich
7f76400550
Update botocore version.
Fixes #2400.
2018-04-09 22:55:20 +03:00
Arik Fraimovich
5f25bc480c Update pymongo version to support newer MongoDB versions 2018-04-08 15:26:35 +03:00
Arik Fraimovich
66a8941a23 Update botocore version to support Glue 2018-02-10 22:04:36 +02:00
Ronald Crooy
80a74934f0 fixing issue #2150 2017-12-12 11:37:24 +01:00
Sumit Maheshwari
660c72e608 Add Qubole query runner 2017-11-24 17:30:08 +05:30
cdessanti
886a5eff43 added python driver of Mapd database
the other packages need will be added as deps when pymapd is installed
2017-10-24 18:25:33 +02:00
Shalom Yerushalmy
fb4153add7 Upgarde cassasndra-driver version to 3.11.0 2017-09-28 13:28:21 +03:00
Antonio Terreno
6295e88d43 Upgraded dql version to 0.5.24 - this allows to query tables in dynamo which have keys with dashes in the name 2017-09-13 09:12:44 +01:00
Arik Fraimovich
3af9b333a8 Merge pull request #1898 from StantonVentures/security_lib_updates_7_27_2017
update libraries
2017-08-16 16:19:57 +03:00
Arik Fraimovich
31ebfb80d7 Snowflake support is no longer enabled by default 2017-08-06 20:45:10 +03:00
Alison
1bc8d586c3 update libraries
Based on pyup auto-PR httplib2 and cryptography needed updating which
necessitated updating pyOpenSSL as well.
2017-07-27 21:15:39 -05:00
Arik Fraimovich
019a09945e Update boto version to support Athena 2017-06-29 09:49:06 +03:00
laughingman7743
fe42195b5a Implement Athena query runner using RestAPI 2017-06-05 22:09:30 +09:00
Arik Fraimovich
9c606b9660 Update Snowflake connector 2017-05-22 14:18:34 +03:00
Arik Fraimovich
749171b186 Merge pull request #1746 from alexanderlz/master
[Data Sources] Add: MemSQL query runner
2017-05-18 14:01:53 +03:00
Arik Fraimovich
52084c322f Google Spreadsheets: add timeout to request. 2017-05-17 18:15:04 +03:00
Arik Fraimovich
4a0612328e TreasureData: improve error handling and upgrade client. 2017-05-15 16:25:18 +03:00
Alexander Leibzon
248808e165 Merge remote-tracking branch 'upstream/master' 2017-05-09 17:53:08 +03:00
Frank Bertsch
75ebbe148b Add presto query cancellation 2017-05-04 11:10:04 -05:00