Skip to content

Remove "from __future__" from airflow/providers __init__.py - #43173

Merged
kaxil merged 2 commits into
apache:mainfrom
potiuk:fix-intellij-pycharm-imports-for-providers
Oct 18, 2024
Merged

Remove "from __future__" from airflow/providers __init__.py#43173
kaxil merged 2 commits into
apache:mainfrom
potiuk:fix-intellij-pycharm-imports-for-providers

Conversation

@potiuk

@potiukpotiuk commented Oct 18, 2024

Copy link
Copy Markdown
Member

Cleans-up airflow and providers __init__.py files in order to get providers import work again.

This is done by excluding the two __init__.py files from automated ruff isort rules adding from __future__ import annotations.

Also removed the __init__.py file from "providers" directory, it is not needed there, because "providers" is just a folder where we keep provider files, it's not a Python package.

That should finally get rid of the Intellij teething import problem that has been introduced in #42505.

There were earlier - unsuccesful - attempts to fix it in the #43116 and #43081 that followed #42951 - but the key is that Pycharm requires the namespace's extend_path to be first "real" line of code in the __init__.py to understand that the package is an "explicit" namespace package - and it conflicts with the requirement of "from future import annotations" to be the first line of Python code.

Also this PR fixes a few other teething problems with setup of tests that were introcuded in #42505 and #43802 "masked" by having __init__.py added in providers package:

  • common.sql interface pre-commit used wrong path to generated files
  • openlineage extractor test that should not expect "providers.tests."
    but "tests.
    " package
  • common_sql_api_stubs wrongly calculating generated path for
    stub-generated files
  • pytest_plugin expecting .asf.yml in "airflow" sources - even during
    compatibility tests with older version of airflow (where the
    .asf.yml is not present)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk

Copy link
Copy Markdown
MemberAuthor

It nicely works for my IntelliJ:

Screenshot 2024-10-18 at 19 53 11

@potiuk

Copy link
Copy Markdown
MemberAuthor

Ha.. Still some errors to fix but we get closer :) . Let me work on those :)

@potiuk
potiukforce-pushed the fix-intellij-pycharm-imports-for-providers branch from 0cea19f to a732655CompareOctober 18, 2024 20:10
@potiuk

potiuk commented Oct 18, 2024

Copy link
Copy Markdown
MemberAuthor

I had to fix (I hope 🤞) a few other problems that wer "masked" by having "init.py" in providers folder.

@potiuk
potiukforce-pushed the fix-intellij-pycharm-imports-for-providers branch from a732655 to 6ce161eCompareOctober 18, 2024 20:19
Comment threadproviders/src/airflow/providers/__init__.py Outdated
Comment threadproviders/__init__.py Outdated
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
Also removed the `__init__.py` file from "providers" directory,
it is not needed there, because "providers" is just a folder where
we keep provider files, it's not a Python package.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccesful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes a few other teething problems with setup of
tests that were introcuded in apache#42505 and apache#43802 "masked" by having
`__init__.py` added in providers package:
* common.sql interface pre-commit used wrong path to generated files
* openlineage extractor test that should not expect "providers.tests.*"
but "tests.*" package
* common_sql_api_stubs wrongly calculating generated path for
stub-generated files
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
@kaxil
kaxilforce-pushed the fix-intellij-pycharm-imports-for-providers branch from f9c4020 to 6cd6564CompareOctober 18, 2024 21:27
@kaxil
kaxilforce-pushed the fix-intellij-pycharm-imports-for-providers branch from 6cd6564 to 78e990cCompareOctober 18, 2024 21:53
@kaxil
kaxil merged commit 6e9c536 into apache:mainOct 18, 2024
harjeevanmaan pushed a commit to harjeevanmaan/airflow that referenced this pull request Oct 23, 2024
…ache#43173)
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccessful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes following problem:
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
---------
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
PaulKobow7536 pushed a commit to PaulKobow7536/airflow that referenced this pull request Oct 24, 2024
…ache#43173)
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccessful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes following problem:
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
---------
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@potiukpotiuk mentioned this pull request Oct 24, 2024
ellisms pushed a commit to ellisms/airflow that referenced this pull request Nov 13, 2024
…ache#43173)
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccessful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes following problem:
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
---------
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
@potiuk
potiuk deleted the fix-intellij-pycharm-imports-for-providers branch April 21, 2026 15:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@potiuk@kaxil@vincbeck