refactor(utils/decorators): rewrite remove task decorator to use cst by josix · Pull Request #43383 · apache/airflow

josix · 2024-10-25T15:51:24Z

due to #42766 the dropping of python3.8 support, I rewrite the remove_task_decorator to achieve the same purpose through CST.

TODO:

implementation
update unit tests

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

Lee-W

As we decide to use CST, just mark it as draft

josix · 2025-01-27T13:31:28Z

Hi @potiuk @Lee-W sorry for waiting for a period of time, I've updated to use libcst version to remove the task decorators. Please take a look when you are available, thanks!

Lee-W

Nice work! left some nits. I'll keep this open for a few days so others can take a lot. if no one object, I'll merge it in 1 or 2 days

potiuk

Approved with the few nits pointed out by @Lee-W. Looks cool. I also rebased it to take into account the failing @Retry operator -> has been introduced and fixed yesterday by the http_retry library author, so main should be fine already.

Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…se ast

…thon_virtualenv.py Co-authored-by: Wei Lee <weilee.rx@gmail.com>

Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…se ast

…pache#43383) * refactor(utils/decorators): rewrite remove task decorator to use ast * Update airflow/utils/decorators.py Co-authored-by: Wei Lee <weilee.rx@gmail.com> * Update airflow/utils/decorators.py Co-authored-by: Wei Lee <weilee.rx@gmail.com> * Update airflow/utils/decorators.py Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! refactor(utils/decorators): rewrite remove task decorator to use ast * fixup! refactor(utils/decorators): rewrite remove task decorator to use ast * Update providers/standard/tests/provider_tests/standard/utils/test_python_virtualenv.py Co-authored-by: Wei Lee <weilee.rx@gmail.com> * Update airflow/utils/decorators.py Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! refactor(utils/decorators): rewrite remove task decorator to use ast * fixup! refactor(utils/decorators): rewrite remove task decorator to use ast --------- Co-authored-by: Wei Lee <weilee.rx@gmail.com>

josix force-pushed the refactor/remove_task_decorator branch 2 times, most recently from 991cccc to 4e926d5Compare October 25, 2024 16:02

josix marked this pull request as ready for review October 25, 2024 16:14

josix marked this pull request as draft October 25, 2024 16:18

Lee-W self-requested a review October 25, 2024 16:19

josix force-pushed the refactor/remove_task_decorator branch 2 times, most recently from d5e6684 to 606b2eaCompare October 25, 2024 20:02

josix commented Oct 25, 2024
View reviewed changes

Comment threadtests/utils/test_python_virtualenv.py Outdated

josix force-pushed the refactor/remove_task_decorator branch from 1ec8a0a to 28d8f2cCompare October 26, 2024 03:45

josix marked this pull request as ready for review October 26, 2024 05:09

Lee-W marked this pull request as draft November 26, 2024 07:13

Lee-W reviewed Nov 26, 2024
View reviewed changes

josix force-pushed the refactor/remove_task_decorator branch from 28d8f2c to 2a82c10Compare December 16, 2024 16:58

josix force-pushed the refactor/remove_task_decorator branch from 2a82c10 to f2cbefaCompare December 28, 2024 14:46

josix force-pushed the refactor/remove_task_decorator branch 4 times, most recently from 292076c to 066d757Compare January 27, 2025 07:28

josix commented Jan 27, 2025
View reviewed changes

Comment threadproviders/standard/tests/provider_tests/standard/utils/test_python_virtualenv.py Outdated

josix marked this pull request as ready for review January 27, 2025 08:22

josix force-pushed the refactor/remove_task_decorator branch from 066d757 to 324301bCompare January 27, 2025 13:29

josix changed the title ~~refactor(utils/decorators): rewrite remove task decorator to use ast~~refactor(utils/decorators): rewrite remove task decorator to use cstJan 28, 2025

Lee-W reviewed Feb 3, 2025
View reviewed changes

josix requested a review from Lee-W February 4, 2025 02:45

josix force-pushed the refactor/remove_task_decorator branch from 6631ea8 to bc09e4bCompare February 4, 2025 04:14

Lee-W reviewed Feb 4, 2025
View reviewed changes

Comment threadairflow/utils/decorators.py Outdated
Comment threadproviders/standard/tests/provider_tests/standard/utils/test_python_virtualenv.py Outdated
Comment threadtests/utils/test_preexisting_python_virtualenv_decorator.py Outdated

josix requested a review from Lee-W February 4, 2025 12:34

Lee-W approved these changes Feb 5, 2025
View reviewed changes

potiuk force-pushed the refactor/remove_task_decorator branch from e7feec6 to 696e0eaCompare February 6, 2025 07:49

potiuk approved these changes Feb 6, 2025
View reviewed changes

josixand others added 9 commits February 7, 2025 03:09

refactor(utils/decorators): rewrite remove task decorator to use ast
26ec8a7

Update airflow/utils/decorators.py
77d961f
Co-authored-by: Wei Lee <weilee.rx@gmail.com>

Update airflow/utils/decorators.py
d34d4c3
Co-authored-by: Wei Lee <weilee.rx@gmail.com>

Update airflow/utils/decorators.py
df0c82f
Co-authored-by: Wei Lee <weilee.rx@gmail.com>

fixup! refactor(utils/decorators): rewrite remove task decorator to u…
d4babb1
…se ast

fixup! refactor(utils/decorators): rewrite remove task decorator to u…
b9acb5a
…se ast

Update providers/standard/tests/provider_tests/standard/utils/test_py…
07e3ba8
…thon_virtualenv.py Co-authored-by: Wei Lee <weilee.rx@gmail.com>

Update airflow/utils/decorators.py
0f2a9bd
Co-authored-by: Wei Lee <weilee.rx@gmail.com>

fixup! refactor(utils/decorators): rewrite remove task decorator to u…
31868ea
…se ast

josix force-pushed the refactor/remove_task_decorator branch from 6df6641 to 31868eaCompare February 6, 2025 19:11

fixup! refactor(utils/decorators): rewrite remove task decorator to u…
9cc3c7d
…se ast

Lee-W merged commit 9689cf5 into apache:mainFeb 7, 2025

eladkal mentioned this pull request Feb 21, 2025
Status of testing Providers that were prepared on February 21, 2025 #46973
Closed

Navigation Menu

Uh oh!

refactor(utils/decorators): rewrite remove task decorator to use cst - #43383

refactor(utils/decorators): rewrite remove task decorator to use cst#43383
Lee-W merged 10 commits into
apache:mainfrom
josix:refactor/remove_task_decorator

josix commented Oct 25, 2024•
edited
Loading

Uh oh!

Uh oh!

Lee-W left a comment

Uh oh!

Uh oh!

josix commented Jan 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Lee-W left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

potiuk left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

josix commented Oct 25, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Lee-W left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

josix commented Jan 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Lee-W left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

potiuk left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

josix commented Oct 25, 2024•
edited
Loading