feat: remove python 3.8 support by chalmerlowe · Pull Request #1215 · googleapis/python-bigquery-sqlalchemy

chalmerlowe · 2025-07-21T14:09:09Z

Removes Python 3.8 support because 3.8 is End of Life.
Includes minor edits to support this change in configs and in the code

Also adds several minor tweaks that were useful for debugging some of the mods to this PR (and are useful in general for all future debugging efforts), such as:

pip freeze to display what dependencies have been installed
* UPDATE: removed upon request: @calculate_duration decorator to confirm the total time to execute a nox session.

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Linchin · 2025-07-23T17:35:37Z

 "tests",
 ]
 UNIT_TEST_EXTRAS_BY_PYTHON: Dict[str, List[str]] = {
- "3.8": [


Do we want to change this to 3.9 instead?

Broadly, the unit & system test extras are not handled well in this (and likely all our noxfiles). As I will show below, we do fully test this library during unit tests, but I think it could be cleaned up (determine which extras need to be done with which Python versions, why, provide comments to clarify the why, and ensure that the logic across the setup.py, owlbot.py, noxfile.py are all in sync, etc.). I recommend that the clean up be a separate Issue as out of scope for the focus of this PR.
For unit tests, our noxfile currently overrides the effect of the variable
UNIT_TEST_EXTRAS_BY_PYTHON. UNIT_TEST_EXTRAS_BY_PYTHON is used in the function install_unittest_dependencies().
In the unit session there is another snippet that also selects extras to be installed. As can be seen below, for any Python version from 3.11-3.13 we test against alembic. But we see that for any Python version outside of 3.11-3.13 we test against all extras, which by definition in setup.py includes alembic.
if install_extras and session.python in ["3.11", "3.12", "3.13"]: install_target = ".[geography,alembic,tests,bqstorage]" elif install_extras: install_target = ".[all]" else: install_target = "." session.install("-e", install_target, "-c", constraints_path)
The truthfulness of this can be see in the Kokoro CI/CD results which show:
the install command: [all] versus [geography,alembic,tests,bqstorage] based on the Python version
the actual install results from pip freeze (which includes alembic in every case)
the successful run for the alembic tests (in every case)
NOTE: I only show two versions, but I hand-checked every version.
3.9 (shortened for space reasons):
nox > Running session unit-3.9(protobuf_implementation='cpp') ... nox > python -m pip install -e '.[all]' -c /tmpfs/src/github/python-bigquery-sqlalchemy/testing/constraints-3.9.txt ... nox > python -m pip freeze alembic==1.16.4 asyncmock==0.4.2 ... tests/unit/test__types.py ........... [ 9%] tests/unit/test_alembic.py .. [ 10%] tests/unit/test_catalog_functions.py ................................... [ 21%]
3.13 (shortened for space reasons):
nox > Running session unit-3.13(protobuf_implementation='upb') ... nox > python -m pip install -e '.[geography,alembic,tests,bqstorage]' -c /tmpfs/src/github/python-bigquery-sqlalchemy/testing/constraints-3.13.txt nox > python -m pip freeze alembic==1.16.4 asyncmock==0.4.2 ... tests/unit/test_alembic.py .. [ 10%] tests/unit/test_catalog_functions.py ................................... [ 21%]

Linchin · 2025-07-23T17:36:18Z

 .. note::

- System tests are only configured to run under Python 3.8, 3.12, and 3.13.
+ System tests are only configured to run under Python 3.12, and 3.13.


Need to add 3.9 here

Good catch, thanks! Fixed.

Linchin · 2025-07-23T17:43:11Z

 constraints_path,
 )
- if session.python == "3.8":
- extras = "[tests,alembic]"


I wonder if we still need coverage for alembic?

The compliance session selects extras slightly differently than the unit session.
I added alembic to this list of extras to account for that difference. DONE.
I assert that as part of the cleanup task discussed in another comment, all our noxfiles need to be checked to see when/why we choose to do some tests and not others. I suspect there was a historical reason that may OR may not apply.

Linchin · 2025-07-23T17:44:16Z

 # ----------------------------------------------------------------------------
 extras = ["tests"]
 extras_by_python = {
- "3.8": ["tests", "alembic", "bqstorage"],


It feels like we need to change this into 3.9 too?

I added 3.9 with alembic as a system test.
NOTE: there appear to be issues with how owlbot and noxfile, etc are interacting, much like described above for the unit tests. That should be looked into as part of a separate cleanup task.

leahecole · 2025-07-24T14:39:32Z

 CURRENT_DIRECTORY = pathlib.Path(__file__).parent.absolute()

+
+def _calculate_duration(func):


If this is not necessary to the 3.8 removal and is a "would be nice", please split into a second PR - I'm happy to be a reviewer for it!

I removed this code.

leahecole · 2025-07-24T14:41:02Z

 "grpcio >= 1.47.0, < 2.0.0",
 "grpcio >= 1.49.1, < 2.0.0; python_version>='3.11'",
- "pyarrow >= 3.0.0",
+ "pyarrow >= 5.0.0",


Confirming - tests would fail if there were consequences to this +2 major version jump, right?

Tests will fail.
For context: the current version of pyarrow is 20.0
Why version 5.0?:
Version 5.0 came out in 2021.
Version 3.0 and version 4.0 do NOT run on Python 3.9.
Version 5.0 is the first and oldest version that ran on Python 3.9.

This reverts commit 4e86c04.

🤖 I have created a release *beep* *boop* --- ## [1.16.0](v1.15.0...v1.16.0) (2025-11-05) ### Features * Add support for Python 3.14 ([#1278](#1278)) ([c09a009](c09a009)) * Remove python 3.8 support ([#1215](#1215)) ([632d6ef](632d6ef)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

🤖 I have created a release *beep* *boop* --- ## [1.16.0](googleapis/python-bigquery-sqlalchemy@v1.15.0...v1.16.0) (2025-11-05) ### Features * Add support for Python 3.14 ([#1278](googleapis/python-bigquery-sqlalchemy#1278)) ([c09a009](googleapis/python-bigquery-sqlalchemy@c09a009)) * Remove python 3.8 support ([#1215](googleapis/python-bigquery-sqlalchemy#1215)) ([632d6ef](googleapis/python-bigquery-sqlalchemy@632d6ef)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

feat: remove python 3.8 support
6f11d53

chalmerlowe requested a review from a team as a code owner July 21, 2025 14:09

chalmerlowe requested review from a team, agrawal-siddharth and briandorsey July 21, 2025 14:09

blunderbuss-gcfBot assigned logachev Jul 21, 2025

product-auto-labelBot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. samples Issues that are directly related to samples. labels Jul 21, 2025

chalmerlowe assigned chalmerlowe and unassigned logachev Jul 21, 2025

chalmerlowe added 2 commits July 23, 2025 12:05

adds debugging to noxfile and tweaks to correct coverage
4e86c04

updates lowest version for system test and owlbot
21fb2ae

chalmerlowe added the kokoro:run Add this label to force Kokoro to re-run the tests. label Jul 23, 2025

updates DEFAULT_PYTHON_VERSION to 3.10
a8fa953

yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Jul 23, 2025

🦉 Updates from OwlBot post-processor
a00b378
See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

chalmerlowe requested a review from Linchin July 23, 2025 16:44

chalmerlowe added the automerge Merge the pull request once unit tests and other checks pass. label Jul 23, 2025

Linchin reviewed Jul 23, 2025
View reviewed changes

updates extras in several sessions & lists additional Python ver.
a9881fa

leahecole reviewed Jul 24, 2025
View reviewed changes

chalmerlowe added 3 commits July 24, 2025 15:01

Revert "adds debugging to noxfile and tweaks to correct coverage"
8af7c47
This reverts commit 4e86c04.

adds back in code that was removed
762a46d

adds pragma to avoid cover fallure.
e50cad3

leahecole approved these changes Jul 24, 2025
View reviewed changes

chalmerlowe merged commit 632d6ef into mainJul 24, 2025
15 checks passed

chalmerlowe deleted the remove-python-3.8 branch July 24, 2025 18:22

release-pleaseBot mentioned this pull request Jul 24, 2025
chore(main): release 1.16.0 #1217
Merged

Navigation Menu

Uh oh!

feat: remove python 3.8 support - #1215

feat: remove python 3.8 support#1215
chalmerlowe merged 9 commits into
mainfrom
remove-python-3.8

chalmerlowe commented Jul 21, 2025•
edited
Loading

Uh oh!

LinchinJul 23, 2025

Uh oh!

chalmerloweJul 24, 2025•
edited
Loading

Uh oh!

LinchinJul 23, 2025

Uh oh!

chalmerloweJul 24, 2025

Uh oh!

LinchinJul 23, 2025

Uh oh!

chalmerloweJul 24, 2025

Uh oh!

LinchinJul 23, 2025

Uh oh!

chalmerloweJul 24, 2025

Uh oh!

leahecoleJul 24, 2025

Uh oh!

chalmerloweJul 24, 2025

Uh oh!

leahecoleJul 24, 2025

Uh oh!

chalmerloweJul 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		CURRENT_DIRECTORY = pathlib.Path(__file__).parent.absolute()


		def _calculate_duration(func):

Uh oh!

Conversation

chalmerlowe commented Jul 21, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chalmerloweJul 24, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

3.9 (shortened for space reasons):

3.13 (shortened for space reasons):

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

chalmerlowe commented Jul 21, 2025•
edited
Loading

chalmerloweJul 24, 2025•
edited
Loading