Skip to content

Enable explicit namespaces - #42951

Merged
shahar1 merged 11 commits into
apache:mainfrom
aws-mwaa:ferruzzi/init-files-fix
Oct 16, 2024
Merged

Enable explicit namespaces#42951
shahar1 merged 11 commits into
apache:mainfrom
aws-mwaa:ferruzzi/init-files-fix

Conversation

@ferruzzi

Copy link
Copy Markdown
Contributor

The recent PR to reorganize the package requires a change to explicit namespaces for at least some IDEs (PyCharm for sure, possibly others).


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

… namespaces for at least some IDEs (PyChasrm for sure, possibly others).
Comment threadairflow/__init__.py
Comment threadproviders/src/airflow/providers/__init__.py Outdated
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
@gopidesupavan

Copy link
Copy Markdown
Member

@ferruzzi Just wanted to let you know, the lowest direct dependency test failures are in microsoft provider, it is due to recent release of microsoft-kiota-http. imports are not working. am trying to fix these deps on different pr.

@ferruzzi

Copy link
Copy Markdown
ContributorAuthor

I got caught up trying to fix a merge conflict in another PR, I'll get this one sorted out now. Sorry for the holdup.

Comment threadproviders/src/airflow/providers/__init__.py
@shahar1

shahar1 commented Oct 12, 2024

Copy link
Copy Markdown
Contributor

There's an error when using breeze build-docs:

/opt/airflow/docs/apache-airflow-providers-<PROVIDER_NAME>/_api/airflow/providers/index.rst:document isn't included in any
toctree

It seems that after applying the changes and running this command, it creates an extra index.rst for airflow/providers/__init__.py, which is indeed not in use anywhere.

This file should be added to the autoapi_ignore list in airflow/docs/conf.py (i.e., "*/providers/src/airflow/providers/__init__.py").

@ferruzzi
ferruzzi requested a review from potiuk as a code ownerOctober 15, 2024 19:10
Comment threaddocs/conf.py Outdated
Comment threadairflow/__init__.py
@potiuk

Copy link
Copy Markdown
Member

You need to rebase @ferruzzi -> all the problems you see in here have been solved in main.

ferruzziand others added 2 commits October 15, 2024 14:05
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
@ferruzzi
ferruzzi requested review from ashb and potiukOctober 15, 2024 22:22
@ferruzzi

Copy link
Copy Markdown
ContributorAuthor

Alright. This is all green, up-to-date, and typo-free.

@shahar1shahar1 added the airflow3.0:candidate Potential candidates for Airflow 3.0 label Oct 16, 2024
@shahar1
shahar1 merged commit 251fc58 into apache:mainOct 16, 2024
@ashbashb removed the airflow3.0:candidate Potential candidates for Airflow 3.0 label Oct 16, 2024
@vincbeck
vincbeck deleted the ferruzzi/init-files-fix branch October 16, 2024 14:33
R7L208 pushed a commit to R7L208/airflow that referenced this pull request Oct 17, 2024
* The recent PR to reorganize the package requires a change to explicit namespaces for at least some IDEs (PyChasrm for sure, possibly others).
* static fixes
* Update providers/src/airflow/providers/__init__.py
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
* Remove duplicate line
* Rephrase relevant comments
* Add providers tests init file to autoapi_ignore
* typo
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
---------
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
potiuk added a commit to potiuk/airflow that referenced this pull request Oct 18, 2024
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
Also removed the `__init__.py` file from "providers" directory,
it is not needed there, because "providers" is just a folder where
we keep provider files, it's not a Python package.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccesful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
potiuk added a commit to potiuk/airflow that referenced this pull request Oct 18, 2024
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
Also removed the `__init__.py` file from "providers" directory,
it is not needed there, because "providers" is just a folder where
we keep provider files, it's not a Python package.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccesful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes a few other teething problems with setup of
tests that were introcuded in apache#42505 and apache#43802 "masked" by having
`__init__.py` added in providers package:
* openlineage extractor test that should not expect "providers.tests.*"
but "tests.*" package
* common_sql_api_stubs wrongly calculating generated path for
stub-generated files
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
potiuk added a commit to potiuk/airflow that referenced this pull request Oct 18, 2024
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
Also removed the `__init__.py` file from "providers" directory,
it is not needed there, because "providers" is just a folder where
we keep provider files, it's not a Python package.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccesful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes a few other teething problems with setup of
tests that were introcuded in apache#42505 and apache#43802 "masked" by having
`__init__.py` added in providers package:
* common.sql interface pre-commit used wrong path to generated files
* openlineage extractor test that should not expect "providers.tests.*"
but "tests.*" package
* common_sql_api_stubs wrongly calculating generated path for
stub-generated files
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
kaxil pushed a commit to potiuk/airflow that referenced this pull request Oct 18, 2024
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
Also removed the `__init__.py` file from "providers" directory,
it is not needed there, because "providers" is just a folder where
we keep provider files, it's not a Python package.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccesful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes a few other teething problems with setup of
tests that were introcuded in apache#42505 and apache#43802 "masked" by having
`__init__.py` added in providers package:
* common.sql interface pre-commit used wrong path to generated files
* openlineage extractor test that should not expect "providers.tests.*"
but "tests.*" package
* common_sql_api_stubs wrongly calculating generated path for
stub-generated files
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
kaxil added a commit that referenced this pull request Oct 18, 2024
…3173)
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
That should finally get rid of the Intellij teething import
problem that has been introduced in #42505.
There were earlier - unsuccessful - attempts to fix it in
the #43116 and #43081 that followed #42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes following problem:
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
---------
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
harjeevanmaan pushed a commit to harjeevanmaan/airflow that referenced this pull request Oct 23, 2024
* The recent PR to reorganize the package requires a change to explicit namespaces for at least some IDEs (PyChasrm for sure, possibly others).
* static fixes
* Update providers/src/airflow/providers/__init__.py
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
* Remove duplicate line
* Rephrase relevant comments
* Add providers tests init file to autoapi_ignore
* typo
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
---------
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
harjeevanmaan pushed a commit to harjeevanmaan/airflow that referenced this pull request Oct 23, 2024
…ache#43173)
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccessful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes following problem:
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
---------
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
PaulKobow7536 pushed a commit to PaulKobow7536/airflow that referenced this pull request Oct 24, 2024
* The recent PR to reorganize the package requires a change to explicit namespaces for at least some IDEs (PyChasrm for sure, possibly others).
* static fixes
* Update providers/src/airflow/providers/__init__.py
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
* Remove duplicate line
* Rephrase relevant comments
* Add providers tests init file to autoapi_ignore
* typo
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
---------
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
PaulKobow7536 pushed a commit to PaulKobow7536/airflow that referenced this pull request Oct 24, 2024
…ache#43173)
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccessful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes following problem:
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
---------
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
ellisms pushed a commit to ellisms/airflow that referenced this pull request Nov 13, 2024
* The recent PR to reorganize the package requires a change to explicit namespaces for at least some IDEs (PyChasrm for sure, possibly others).
* static fixes
* Update providers/src/airflow/providers/__init__.py
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
* Remove duplicate line
* Rephrase relevant comments
* Add providers tests init file to autoapi_ignore
* typo
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
---------
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
ellisms pushed a commit to ellisms/airflow that referenced this pull request Nov 13, 2024
…ache#43173)
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.
This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.
That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.
There were earlier - unsuccessful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.
Also this PR fixes following problem:
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
compatibility tests with older version of airflow (where the
.asf.yml is not present)
---------
Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@ferruzzi@gopidesupavan@shahar1@potiuk@ashb@kaxil@vincbeck