Fix task retries executing wrong method after deferred state by kaxil · Pull Request #56731 · apache/airflow

kaxil · 2025-10-16T16:05:40Z

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch.

The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts.

Fixes#56704

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

tirkarthi

LGTM. Thanks @kaxil .

potiuk

Nice. Bug squasher @kaxil

kaxil · 2025-10-16T17:01:42Z

the k8s failure will be fixed by #56733

…ate (#56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixes#56704 (cherry picked from commit 16e9b35) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

github-actions · 2025-10-16T17:02:07Z

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

…ate (#56731) (#56737)

…56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

kaxil · 2025-10-16T16:05:40Z

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch.

The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts.

Fixes#56704

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

tirkarthi

LGTM. Thanks @kaxil .

potiuk

Nice. Bug squasher @kaxil

kaxil · 2025-10-16T17:01:42Z

the k8s failure will be fixed by #56733

…ate (#56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixes#56704 (cherry picked from commit 16e9b35) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

github-actions · 2025-10-16T17:02:07Z

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

…ate (#56731) (#56737)

…56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

kaxil · 2025-10-16T16:05:40Z

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch.

The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts.

Fixes#56704

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

tirkarthi

LGTM. Thanks @kaxil .

potiuk

Nice. Bug squasher @kaxil

kaxil · 2025-10-16T17:01:42Z

the k8s failure will be fixed by #56733

…ate (#56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixes#56704 (cherry picked from commit 16e9b35) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

github-actions · 2025-10-16T17:02:07Z

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

…ate (#56731) (#56737)

…56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

kaxil · 2025-10-16T16:05:40Z

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch.

The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts.

Fixes#56704

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

tirkarthi

LGTM. Thanks @kaxil .

potiuk

Nice. Bug squasher @kaxil

kaxil · 2025-10-16T17:01:42Z

the k8s failure will be fixed by #56733

…ate (#56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixes#56704 (cherry picked from commit 16e9b35) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

github-actions · 2025-10-16T17:02:07Z

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

…ate (#56731) (#56737)

…56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

kaxil · 2025-10-16T16:05:40Z

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch.

The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts.

Fixes#56704

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

tirkarthi

LGTM. Thanks @kaxil .

potiuk

Nice. Bug squasher @kaxil

kaxil · 2025-10-16T17:01:42Z

the k8s failure will be fixed by #56733

…ate (#56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixes#56704 (cherry picked from commit 16e9b35) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

github-actions · 2025-10-16T17:02:07Z

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

…ate (#56731) (#56737)

…56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

kaxil · 2025-10-16T16:05:40Z

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch.

The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts.

Fixes#56704

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

tirkarthi

LGTM. Thanks @kaxil .

potiuk

Nice. Bug squasher @kaxil

kaxil · 2025-10-16T17:01:42Z

the k8s failure will be fixed by #56733

…ate (#56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixes#56704 (cherry picked from commit 16e9b35) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

github-actions · 2025-10-16T17:02:07Z

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

…ate (#56731) (#56737)

…56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

kaxil · 2025-10-16T16:05:40Z

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch.

The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts.

Fixes#56704

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

tirkarthi

LGTM. Thanks @kaxil .

potiuk

Nice. Bug squasher @kaxil

kaxil · 2025-10-16T17:01:42Z

the k8s failure will be fixed by #56733

…ate (#56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixes#56704 (cherry picked from commit 16e9b35) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

github-actions · 2025-10-16T17:02:07Z

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

…ate (#56731) (#56737)

…56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

kaxil · 2025-10-16T16:05:40Z

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch.

The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts.

Fixes#56704

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

tirkarthi

LGTM. Thanks @kaxil .

potiuk

Nice. Bug squasher @kaxil

kaxil · 2025-10-16T17:01:42Z

the k8s failure will be fixed by #56733

…ate (#56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixes#56704 (cherry picked from commit 16e9b35) Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

github-actions · 2025-10-16T17:02:07Z

Backport successfully created: v3-1-test

Status	Branch	Result
✅	v3-1-test

…ate (#56731) (#56737)

…56731) When a deferred task resumed and failed in its completion callback, retries would incorrectly re-execute the callback instead of restarting from the beginning. This caused retry attempts to fail repeatedly at the same step rather than attempting the full task execution from scratch. The fix ensures next_method and next_kwargs fields are properly cleared when tasks transition to retry, failed, or success states, preventing stale deferral parameters from persisting across task attempts. Fixesapache#56704

kaxil added this to the Airflow 3.1.1 milestone Oct 16, 2025

kaxil requested a review from tirkarthi October 16, 2025 16:05

kaxil requested review from amoghrajesh and ashb as code owners October 16, 2025 16:05

kaxil added the backport-to-v3-1-test label Oct 16, 2025

boring-cyborgBot added the area:API Airflow's REST/HTTP API label Oct 16, 2025

amoghrajesh approved these changes Oct 16, 2025
View reviewed changes

tirkarthi approved these changes Oct 16, 2025
View reviewed changes

potiuk reviewed Oct 16, 2025
View reviewed changes

potiuk approved these changes Oct 16, 2025
View reviewed changes

kaxil merged commit 16e9b35 into apache:mainOct 16, 2025
109 of 112 checks passed

kaxil deleted the fix-retry-deferral branch October 16, 2025 17:01

kaxil pushed a commit that referenced this pull request Oct 16, 2025
[v3-1-test] Fix task retries executing wrong method after deferred st…
53c05a0
…ate (#56731) (#56737)

potiuk mentioned this pull request Nov 8, 2025
Add missing translations for PL for 3.1.3 #58076
Merged