Resolve VariableInterval deadlines safely at DagRun creation

[seanghaeli]

Why

This re-introduces the VariableInterval resolution portion of #66608.

A DeadlineAlert configured with a VariableInterval is resolved when the scheduler creates a DagRun, inside DAG._process_dagrun_deadline_alerts (which runs under the prohibit_commit guard). Two problems are fixed:

1. Resolution goes through the full secrets chain (env vars, configured secrets backends, then the metadata DB) via a dedicated _resolve_variable_interval helper, rather than Variable.get / begin_nested. Variable.get and a SAVEPOINT release both commit on the scheduler's session, tripping prohibit_commit (UNEXPECTED COMMIT) and silently dropping deadlines for every scheduled DagRun. The helper passes the scheduler session through to the metastore backend so the DB read does not commit, and reading via the secrets chain (not the variable table directly) means AIRFLOW_VAR_* env vars and secrets-backend-backed Variables resolve too.

2. Each deadline alert is isolated with a plain try/except (deliberately notbegin_nested, which would commit a SAVEPOINT and trip the same guard). Creating a deadline is auxiliary to creating the DagRun; a single bad alert — a missing/invalid backing Variable, or an undecodable serialized blob — must never abort the DagRun and stop the DAG from scheduling.

VariableInterval.resolve is split into resolve + coerce_to_timedelta so the scheduler-side reader reuses the exact same validation (including the OverflowError -> ValueError translation) without going through Variable.get.

Tests

• airflow-core/tests/unit/models/test_dagrun.py: VariableInterval resolves from a real Variable row and from an AIRFLOW_VAR_* env var; a missing Variable and an undecodable alert are isolated (DagRun still created, no Deadline row, error logged).
• task-sdk/tests/task_sdk/definitions/test_deadline.py: coerce_to_timedelta validation (non-integer, <= 0, overflow).

Verified locally in Breeze: 8 passed (dagrun deadline/variable) + 14 passed (SDK TestVariableInterval).

Generated-by: Claude Code (Opus via Claude Code) on behalf of Sean Ghaeli

[SameerMesiah97]

I did not manage to get to the tests but if I have to be honest, I think this needs a round of polish before a maintainer can review it. Conceptually, it looks correct but there are classic smells of unvetted AI-generated content such as messy code, long commnets/docstrings etc. I would convert it to draft and clean it up before requesting review.

I have left some comments.

[seanghaeli]

@SameerMesiah97 could I get your inputs on the updated version?

[SameerMesiah97]

Looks good to me. I just left a few more comments.

[ferruzzi]

Apparently I reviewed this last week and forgot to submit. Sorry for the delay

[ferruzzi]

Looks like my concerns have all been addressed. LGTM

[ferruzzi]

I'm sorry about this, there are now eight PRs all interacting with the Deadlines code and I'm trying to untangle the conflicts before they get worse. #71802 is approved and needs to be merged before this one because it also blocks a few other fixes, but it is going to kind of pull the rug out from under you a bit on this.

Once it merges, dag.py checks isinstance(interval, SerializedVariableInterval) and no longer imports VariableInterval at all. That means:

• You can drop _resolve_variable_interval. #71968 adds an optional keyword-only session to Variable.get, so SerializedVariableInterval.resolve(session=session) covers it without duplicating the backend walk. Your comment there already points at #71801, which is the issue #71968 fixes.

• coerce_to_timedelta then loses its only caller, so that can be dropped along with the related test_coerce_to_timedelta_invalid. The OverflowError to ValueError translation is a real improvement nobody else hit on, but it needs to live where the live path actually runs, so wrap the return timedelta(seconds=seconds) at the end of the new SerializedVariableInterval.resolve() in airflow-core/src/airflow/serialization/definitions/deadline.py, alongside the int() cast. Then add a "too large" case to test_resolve_invalid which #71802 adds in airflow-core/tests/unit/serialization/definitions/test_deadline.py.

• The seconds <= 0 guard is the one real conflict. An interval can be negative so that check is a distinct bug. Please drop the guard along with the ("0", "must be > 0") and ("-5", "must be > 0") test cases.

Side note: The per-alert try/except isolation is good and nobody else is doing that. It may turn out to be one of the most valuable additions in your PR.

One thing to check: #71287 is doing failure isolation with savepoints around DagRun creation. Your comment says you chose a plain try/except over session.begin_nested() deliberately, so you may already have looked, but the two haven't been cross-referenced anywhere that I can tell.

Overall, it should result in a smaller PR for you, trimming out some redundant (and conflicting) code, not a rewrite.

I am also submitting this as a "Request changes" so it blocks merging out of order and doesn't screw up the other PRs. Unblock conditions: #71802 must merge first, this PR must be merged on top of that one, and the above changes need to be made. Once this one merges, It will unblock #71968 which will have to rebase on top of this and get some tweaks.

[ferruzzi]

You are going to end up having to rebase on top of a couple of other PRs that need to land before this one. I'm trying to make that future rebase a bit easier here.

#71767 has an elif branch right after the stats.incr you have on L800. If you can add that here, it should ease your merge pain later:

stats.incr("deadline_alerts.deadline_created", tags=metrics_tags)
elifrequired_dagrun_column:= {
SerializedReferenceModels.DagRunLogicalDateDeadline: "logical_date",
SerializedReferenceModels.DagRunQueuedAtDeadline: "queued_at",
}.get(type(deserialized_deadline_alert.reference)):
log.warning(
"skipping deadline alert because the deadline reference evaluated to None",
dag_id=self.dag_id,
run_id=orm_dagrun.run_id,
deadline_alert_id=deadline_alert.id,
reference_type=deserialized_deadline_alert.reference.reference_name,
required_dagrun_column=required_dagrun_column,
)

That PR also adds two tests in airflow-core/tests/unit/models/test_dagrun.py that you'll want to pull over which cover that branch. That can wait till the merge but may be easier to just paste it over now since you are copying this over.