Adding compatibility with SQLAlchemy 2.0 by mdeshmu · Pull Request #457 · dropbox/PyHive

mdeshmu · 2023-06-23T17:53:33Z

Description

This PR includes minimal changes to add SQLAlchemy 2.0 support without breaking compatibility with SQLAlchemy 1.3 and SQLAlchemy 1.4.

Test Environment

SQLALchemy 1.3.24, 1.4.36, and 2.0.16 was used with this hive docker setup for running the test cases. Trino isn't present in that setup, so I added the below lines to docker-compose.yaml and copied trino config from pyhive repository to the Trino container as a volume. That's it, My testing setup was ready.

trino:
image: trinodb/trino:351
ports:
- "18080:18080"
volumes:
- ./trino:/etc/trino

Software versions used in test setup

Hive version - 2.3.2
Presto version - 0.181
Trino version - 351

Local test case run results:

SQLAlchemy 1.3 + Hive

SQLAlchemy 1.3 + Presto

SQLAlchemy 1.3 + Trino

SQLAlchemy 1.4 + Hive

SQLAlchemy 1.4 + Presto

SQLAlchemy 1.4 + Trino

SQLAlchemy 2.0 + Hive

SQLAlchemy 2.0 + Presto

SQLAlchemy 2.0 + Trino

mdeshmu · 2023-06-28T06:40:37Z

@bkyryliuk This PR is ready for review.

bkyryliuk

LG%small nits

* feat: add HTTP and HTTPS to hive (dropbox#385) * feat: add https protocol * support HTTP * fix: make hive https py2 compat (dropbox#389) * fix: make hive https py2 compat * fix lint * Update README.rst (dropbox#423) * chore: rename Trino entry point (dropbox#428) * Support for Presto decimals (dropbox#430) * Support for Presto decimals * lower * Use str type for driver and name in HiveDialect (dropbox#450) PyHive's HiveDialect usage of bytes for the name and driver fields is not the norm is causing issues upstream: apache/superset#22316 Even other dialects within PyHive use strings. SQLAlchemy does not strictly require a string, but all the stock dialects return a string, so I figure it is heavily implied. I think the risk of breaking something upstream with this change is low (but it is there ofc). I figure in most cases we just make someone's `str(dialect.driver)` expression redundant. Examples for some of the other stock sqlalchemy dialects (name and driver fields using str): https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/sqlite/pysqlite.py#L501 https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/sqlite/base.py#L1891 https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/mysql/base.py#L2383 https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/mysql/mysqldb.py#L113 https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/mysql/pymysql.py#L59 * Correcting Iterable import for python 3.10 (dropbox#451) * changing drivers to support hive, presto and trino with sqlalchemy>=2.0 (dropbox#448) * Revert "changing drivers to support hive, presto and trino with sqlalchemy>=2.0 (dropbox#448)" (dropbox#452) This reverts commit b0206d3. * Update __init__.py (dropbox#453) dropbox@1c1da8bdropbox@1f99552 * use pure-sasl with python 3.11 (dropbox#454) * minimal changes for sqlalchemy 2.0 support (dropbox#457) * update readme to reflect recent changes (dropbox#459) * Update README.rst (dropbox#475) * Update README.rst (dropbox#476) * feat: JWT support * Add CI to build package --------- Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com> Co-authored-by: Bogdan <b.kyryliuk@gmail.com> Co-authored-by: serenajiang <serena.jiang@airbnb.com> Co-authored-by: Usiel Riedl <usiel.riedl@gmail.com> Co-authored-by: Multazim Deshmukh <57723564+mdeshmu@users.noreply.github.com> Co-authored-by: nicholas-miles <nicholas.miles6@gmail.com>

mdeshmu force-pushed the sqlalchemy-2.0-support branch 2 times, most recently from d1bf344 to eb1ed24Compare June 23, 2023 18:00