Uh oh!
There was an error while loading. Please reload this page.
Apply rerun_with_latest_version to TriggerDagRunOperator reruns - #67273
Merged
kaxil merged 1 commit intoJun 19, 2026
Merged
Conversation
nathadfieldforce-pushed
the
apply-rerun-with-latest-version-to-trigger-dagrun-resets
branch
3 times, most recently
from
May 22, 2026 07:43
c17c7d6 to
0900680Comparenathadfield
marked this pull request as ready for review
May 22, 2026 09:36
nathadfield
commented
Jun 8, 2026
ContributorAuthor
@ashb@kaxil@amoghrajesh Might one of you be able to review this at some point? Thanks very much! |
kaxil
reviewed
Jun 19, 2026
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
When `TriggerDagRunOperator(reset_dag_run=True)` is invoked and the
target Dag run already exists, the task SDK falls back to
`client.clear(dag_id, run_id)`, which hits
`POST /execution/dag-runs/{dag_id}/{run_id}/clear`. That endpoint
previously called `dag.clear(run_id=run_id)` with no
`run_on_latest_version`, so the global and DAG-level
`rerun_with_latest_version` config (added in apache#63884) had no effect on
operator-driven reruns.
The endpoint now resolves the value via the existing
`resolve_run_on_latest_version` hierarchy (explicit > DAG-level >
global config > fallback) and forwards it to `dag.clear()` — matching
the behavior already in place for UI, public API, and CLI clears.
closes: apache#60880nathadfieldforce-pushed
the
apply-rerun-with-latest-version-to-trigger-dagrun-resets
branch
from
June 19, 2026 08:23
0900680 to
ca6990cComparenathadfield
requested review from
bugraoz93, choo121600, dheerajturaga, ephraimbuddy, jason810496, pierrejeambrun, potiuk, rawwar and shubhamraj-git
as code ownersJune 19, 2026 08:23
kaxil
approved these changes
Jun 19, 2026
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
PR #63884 introduced a 4-tier
rerun_with_latest_versionconfig hierarchy (explicit > DAG-level > global config > fallback) controlling whether clearing/rerunning a Dag run uses the latest bundle version. That hierarchy is wired into the UI clear dialogs, the public clear/backfill API endpoints, and theairflow backfill createCLI command but not intoTriggerDagRunOperator'sreset_dag_run=Truepath.When
TriggerDagRunOperator(reset_dag_run=True)is invoked and the target Dag run already exists, the task SDK trigger client gets a 409 and falls back toclient.clear(dag_id, run_id). That hitsPOST /execution/dag-runs/{dag_id}/{run_id}/clearon the execution API, which previously calleddag.clear(run_id=run_id)with norun_on_latest_versionargument. As a result, the global and DAG-levelrerun_with_latest_versionconfig had no effect on operator-driven reruns.What this PR does
Wires the existing
resolve_run_on_latest_versionresolver into the execution-APIclear_dag_runendpoint, so operator-driven reruns honor the same hierarchy as UI-, API-, and CLI-driven reruns.Noneis passed as the explicit override because the operator does not (intentionally) accept a per-call override. The DAG-level toggle and global config from #63884 are already sufficient, and adding an operator parameter that accepts arbitrary versions would create a security gap (a git bundle could be made to run a commit that was never on the tracking branch).Brand-new Dag runs from
TriggerDagRunOperatorare unaffected: they already use the latest bundle version because_create_orm_dagrunreadsDagModel.bundle_versionat trigger time. Only the rerun/clear path needed wiring.No payload/schema change; no client change; no Cadwyn version bump.
Test plan
test_dag_run_clear_invokes_resolververifies the resolver is invoked withNoneas the explicit override.TestDagRunCleartests still pass unchanged.prek run --from-ref apache/main --stage pre-commit— all hooks pass (ruff, mypy-airflow-core, etc.).closes: #60880
Was generative AI tooling used to co-author this PR?