Skip to content

Fix task configuration defaults for AbstractOperator - #52871

Merged
kaxil merged 1 commit into
apache:mainfrom
astronomer:remove-abstract-operator
Jul 4, 2025
Merged

Fix task configuration defaults for AbstractOperator#52871
kaxil merged 1 commit into
apache:mainfrom
astronomer:remove-abstract-operator

Conversation

@kaxil

@kaxilkaxil commented Jul 4, 2025

Copy link
Copy Markdown
Member

Some defaults weren't being taken from configuration -- this is now fixed.

Simple Test to ensure back-compat works:

In [1]: fromairflow.models.abstractoperatorimportAbstractOperator<ipython-input-1-1a7685527e0f>:1DeprecationWarning: The`airflow.models.abstractoperator.AbstractOperator`classisdeprecated. Pleaseuse`'airflow.sdk.definitions._internal.abstractoperator.AbstractOperator'`.
In [2]: fromairflow.models.abstractoperatorimportDEFAULT_QUEUE<ipython-input-2-f543b59082e8>:1DeprecationWarning: The`airflow.models.abstractoperator.DEFAULT_QUEUE`classisdeprecated. Pleaseuse`'airflow.sdk.definitions._internal.abstractoperator.DEFAULT_QUEUE'`.
In [3]: fromairflow.models.abstractoperatorimportDEFAULT_OWNER, DEFAULT_TASK_EXECUTION_TIMEOUT, TaskStateChangeCallback, NotMapped<ipython-input-3-59defa9c8ac0>:1DeprecationWarning: The`airflow.models.abstractoperator.DEFAULT_OWNER`classisdeprecated. Pleaseuse`'airflow.sdk.definitions._internal.abstractoperator.DEFAULT_OWNER'`.
<ipython-input-3-59defa9c8ac0>:1DeprecationWarning: The`airflow.models.abstractoperator.DEFAULT_TASK_EXECUTION_TIMEOUT`classisdeprecated. Pleaseuse`'airflow.sdk.definitions._internal.abstractoperator.DEFAULT_TASK_EXECUTION_TIMEOUT'`.
<ipython-input-3-59defa9c8ac0>:1DeprecationWarning: The`airflow.models.abstractoperator.TaskStateChangeCallback`classisdeprecated. Pleaseuse`'airflow.sdk.definitions._internal.abstractoperator.TaskStateChangeCallback'`.
<ipython-input-3-59defa9c8ac0>:1DeprecationWarning: The`airflow.models.abstractoperator.NotMapped`classisdeprecated. Pleaseuse`'airflow.sdk.definitions._internal.abstractoperator.NotMapped'`.
In [4]: DEFAULT_QUEUEOut[4]: 'default'

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Comment threadairflow-core/src/airflow/models/__init__.py
ashb
ashb approved these changes Jul 4, 2025

@ashbashb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you are also missing

  • default_task_retry_delay
  • default_task_retries
  • default_task_weight_rule
  • default_queue

@kaxil

kaxil commented Jul 4, 2025

Copy link
Copy Markdown
MemberAuthor

Looks like you are also missing

  • default_task_retry_delay
  • default_task_retries
  • default_task_weight_rule
  • default_queue

aah, yeah will fix them in this PR itself so it is all in one

@kaxil
kaxilforce-pushed the remove-abstract-operator branch 4 times, most recently from bb51a52 to 139c058CompareJuly 4, 2025 16:00
Some defaults weren't being taken from configuration -- this is now fixed.
@kaxil
kaxilforce-pushed the remove-abstract-operator branch from 139c058 to dcedeffCompareJuly 4, 2025 17:12
@kaxil

kaxil commented Jul 4, 2025

Copy link
Copy Markdown
MemberAuthor

Static checks fixed by #52887

@kaxil
kaxil merged commit 04d2d3b into apache:mainJul 4, 2025
1 check passed
@kaxil
kaxil deleted the remove-abstract-operator branch July 4, 2025 17:49
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-0-test. View the failure log Run details

StatusBranchResult
v3-0-testCommit Link

You can attempt to backport this manually by running:

cherry_picker 04d2d3b v3-0-test

This should apply the commit to the v3-0-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

amoghrajesh pushed a commit to astronomer/airflow that referenced this pull request Aug 11, 2025
Some defaults weren't being taken from configuration -- this is now fixed.
(cherry picked from commit 04d2d3b)
amoghrajesh pushed a commit to astronomer/airflow that referenced this pull request Aug 11, 2025
Some defaults weren't being taken from configuration -- this is now fixed.
(cherry picked from commit 04d2d3b)
kaxil added a commit that referenced this pull request Aug 11, 2025
Some defaults weren't being taken from configuration -- this is now fixed.
(cherry picked from commit 04d2d3b)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providersarea:task-sdkprovider:edgeEdge Executor / Worker (AIP-69) / edge3

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@kaxil@ashb@hussein-awala@amoghrajesh