Add row lock to ADRQ before Dag run creation by Lee-W · Pull Request #60773 · apache/airflow

Lee-W · 2026-01-19T10:43:41Z

Why

2 or more different schedulers might read the same set of ADRQs and attempt to create more then 1 Dag run while only the first scheduler should create one

What

Add row_ock to ADRQ before Dag run creation

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Lee-W · 2026-03-03T12:32:17Z

would be nice if we can make it part of 3.1.8

potiuk

LGTM - but I will defer to @uranusjr !

Nice catch!

…ndition between multiple scheduler

jason810496

Thanks!

github-actions · 2026-03-17T03:36:14Z

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker b3055d7 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…he#60773) (cherry picked from commit b3055d7)

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

hussein-awala · 2026-03-23T21:44:09Z

I encountered this exact race condition in production on 3.1.7 — two schedulers both read the ADRQ before either committed, resulting in duplicate asset-triggered DagRuns with the same run_after timestamp. Independently traced it to the missing row lock on ADRQ in _create_dag_runs_asset_triggered. Thanks for the fix!

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

Lee-W · 2026-01-19T10:43:41Z

closes: #63507

Why

2 or more different schedulers might read the same set of ADRQs and attempt to create more then 1 Dag run while only the first scheduler should create one

What

Add row_ock to ADRQ before Dag run creation

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Lee-W · 2026-03-03T12:32:17Z

would be nice if we can make it part of 3.1.8

potiuk

LGTM - but I will defer to @uranusjr !

Nice catch!

…ndition between multiple scheduler

jason810496

Thanks!

github-actions · 2026-03-17T03:36:14Z

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker b3055d7 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…he#60773) (cherry picked from commit b3055d7)

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

hussein-awala · 2026-03-23T21:44:09Z

I encountered this exact race condition in production on 3.1.7 — two schedulers both read the ADRQ before either committed, resulting in duplicate asset-triggered DagRuns with the same run_after timestamp. Independently traced it to the missing row lock on ADRQ in _create_dag_runs_asset_triggered. Thanks for the fix!

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

Lee-W · 2026-01-19T10:43:41Z

closes: #63507

Why

2 or more different schedulers might read the same set of ADRQs and attempt to create more then 1 Dag run while only the first scheduler should create one

What

Add row_ock to ADRQ before Dag run creation

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Lee-W · 2026-03-03T12:32:17Z

would be nice if we can make it part of 3.1.8

potiuk

LGTM - but I will defer to @uranusjr !

Nice catch!

…ndition between multiple scheduler

jason810496

Thanks!

github-actions · 2026-03-17T03:36:14Z

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker b3055d7 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…he#60773) (cherry picked from commit b3055d7)

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

hussein-awala · 2026-03-23T21:44:09Z

I encountered this exact race condition in production on 3.1.7 — two schedulers both read the ADRQ before either committed, resulting in duplicate asset-triggered DagRuns with the same run_after timestamp. Independently traced it to the missing row lock on ADRQ in _create_dag_runs_asset_triggered. Thanks for the fix!

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

Lee-W · 2026-01-19T10:43:41Z

closes: #63507

Why

2 or more different schedulers might read the same set of ADRQs and attempt to create more then 1 Dag run while only the first scheduler should create one

What

Add row_ock to ADRQ before Dag run creation

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Lee-W · 2026-03-03T12:32:17Z

would be nice if we can make it part of 3.1.8

potiuk

LGTM - but I will defer to @uranusjr !

Nice catch!

…ndition between multiple scheduler

jason810496

Thanks!

github-actions · 2026-03-17T03:36:14Z

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker b3055d7 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…he#60773) (cherry picked from commit b3055d7)

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

hussein-awala · 2026-03-23T21:44:09Z

I encountered this exact race condition in production on 3.1.7 — two schedulers both read the ADRQ before either committed, resulting in duplicate asset-triggered DagRuns with the same run_after timestamp. Independently traced it to the missing row lock on ADRQ in _create_dag_runs_asset_triggered. Thanks for the fix!

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

Lee-W · 2026-01-19T10:43:41Z

closes: #63507

Why

2 or more different schedulers might read the same set of ADRQs and attempt to create more then 1 Dag run while only the first scheduler should create one

What

Add row_ock to ADRQ before Dag run creation

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Lee-W · 2026-03-03T12:32:17Z

would be nice if we can make it part of 3.1.8

potiuk

LGTM - but I will defer to @uranusjr !

Nice catch!

…ndition between multiple scheduler

jason810496

Thanks!

github-actions · 2026-03-17T03:36:14Z

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker b3055d7 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…he#60773) (cherry picked from commit b3055d7)

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

hussein-awala · 2026-03-23T21:44:09Z

I encountered this exact race condition in production on 3.1.7 — two schedulers both read the ADRQ before either committed, resulting in duplicate asset-triggered DagRuns with the same run_after timestamp. Independently traced it to the missing row lock on ADRQ in _create_dag_runs_asset_triggered. Thanks for the fix!

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

Lee-W · 2026-01-19T10:43:41Z

closes: #63507

Why

2 or more different schedulers might read the same set of ADRQs and attempt to create more then 1 Dag run while only the first scheduler should create one

What

Add row_ock to ADRQ before Dag run creation

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Lee-W · 2026-03-03T12:32:17Z

would be nice if we can make it part of 3.1.8

potiuk

LGTM - but I will defer to @uranusjr !

Nice catch!

…ndition between multiple scheduler

jason810496

Thanks!

github-actions · 2026-03-17T03:36:14Z

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker b3055d7 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…he#60773) (cherry picked from commit b3055d7)

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

hussein-awala · 2026-03-23T21:44:09Z

I encountered this exact race condition in production on 3.1.7 — two schedulers both read the ADRQ before either committed, resulting in duplicate asset-triggered DagRuns with the same run_after timestamp. Independently traced it to the missing row lock on ADRQ in _create_dag_runs_asset_triggered. Thanks for the fix!

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

Lee-W · 2026-01-19T10:43:41Z

closes: #63507

Why

2 or more different schedulers might read the same set of ADRQs and attempt to create more then 1 Dag run while only the first scheduler should create one

What

Add row_ock to ADRQ before Dag run creation

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Lee-W · 2026-03-03T12:32:17Z

would be nice if we can make it part of 3.1.8

potiuk

LGTM - but I will defer to @uranusjr !

Nice catch!

…ndition between multiple scheduler

jason810496

Thanks!

github-actions · 2026-03-17T03:36:14Z

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker b3055d7 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…he#60773) (cherry picked from commit b3055d7)

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

hussein-awala · 2026-03-23T21:44:09Z

I encountered this exact race condition in production on 3.1.7 — two schedulers both read the ADRQ before either committed, resulting in duplicate asset-triggered DagRuns with the same run_after timestamp. Independently traced it to the missing row lock on ADRQ in _create_dag_runs_asset_triggered. Thanks for the fix!

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

Lee-W · 2026-01-19T10:43:41Z

closes: #63507

Why

2 or more different schedulers might read the same set of ADRQs and attempt to create more then 1 Dag run while only the first scheduler should create one

What

Add row_ock to ADRQ before Dag run creation

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Lee-W · 2026-03-03T12:32:17Z

would be nice if we can make it part of 3.1.8

potiuk

LGTM - but I will defer to @uranusjr !

Nice catch!

…ndition between multiple scheduler

jason810496

Thanks!

github-actions · 2026-03-17T03:36:14Z

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker b3055d7 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

…he#60773) (cherry picked from commit b3055d7)

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

hussein-awala · 2026-03-23T21:44:09Z

I encountered this exact race condition in production on 3.1.7 — two schedulers both read the ADRQ before either committed, resulting in duplicate asset-triggered DagRuns with the same run_after timestamp. Independently traced it to the missing row lock on ADRQ in _create_dag_runs_asset_triggered. Thanks for the fix!

…3776) * [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com> * fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (#60773) (cherry picked from commit b3055d7)

Lee-W changed the title ~~Lock adpr~~Add row lock to ADRQJan 19, 2026

Lee-W force-pushed the lock-adpr branch 3 times, most recently from daebc01 to 5b93f53Compare February 5, 2026 14:08

Lee-W force-pushed the lock-adpr branch from 5b93f53 to 64d49f1Compare February 13, 2026 10:21

Lee-W force-pushed the lock-adpr branch 2 times, most recently from 2606938 to 4abdc74Compare March 2, 2026 03:23

Lee-W changed the title ~~Add row lock to ADRQ~~Add row lock to ADRQ before Dag run creationMar 2, 2026

Lee-W marked this pull request as ready for review March 2, 2026 03:25

Lee-W requested review from XD-DENG and ashb as code owners March 2, 2026 03:25

Lee-W force-pushed the lock-adpr branch 3 times, most recently from 2ffff9e to 5df556cCompare March 3, 2026 11:35

Lee-W requested review from uranusjr and vatsrahul1001 March 3, 2026 12:31

uranusjr reviewed Mar 5, 2026
View reviewed changes

Comment threadairflow-core/src/airflow/jobs/scheduler_job_runner.py Outdated

Lee-W added the backport-to-v3-1-test label Mar 5, 2026

uranusjr reviewed Mar 5, 2026
View reviewed changes

Comment threadairflow-core/src/airflow/jobs/scheduler_job_runner.py Outdated

Lee-W force-pushed the lock-adpr branch 5 times, most recently from e2873dc to b96509aCompare March 9, 2026 08:58

Lee-W marked this pull request as draft March 9, 2026 09:02

Lee-W force-pushed the lock-adpr branch from b96509a to b8f86faCompare March 10, 2026 13:25

Lee-W marked this pull request as ready for review March 10, 2026 13:39

Lee-W force-pushed the lock-adpr branch from b8f86fa to 49999e2Compare March 11, 2026 04:47

Lee-W added this to the Airflow 3.1.9 milestone Mar 11, 2026

potiuk approved these changes Mar 11, 2026
View reviewed changes

jason810496 reviewed Mar 16, 2026
View reviewed changes

Comment threadairflow-core/src/airflow/jobs/scheduler_job_runner.py Outdated
Comment threadairflow-core/src/airflow/jobs/scheduler_job_runner.py Outdated

uranusjr approved these changes Mar 16, 2026
View reviewed changes

Lee-W force-pushed the lock-adpr branch from 49999e2 to b5ecf45Compare March 16, 2026 09:50

Lee-W requested a review from jason810496 March 16, 2026 09:51

Lee-W added 6 commits March 16, 2026 18:49

fix(ADRQ): lock ADRQs right before Dag run creationg to avoid race co…
659ea0c
…ndition between multiple scheduler

style: ignore MockExecutor as it intends to be a mocked BaseExecutor
56ec448

refactor: address review comments
6c474d1

refactor: sort by created_by so that we don't need to use max again
b866aeb

refactor: remove redundant triggered_date_by_dag usage
e140c41

fixup! refactor: remove redundant triggered_date_by_dag usage
51da4e1

Lee-W force-pushed the lock-adpr branch from b5ecf45 to 51da4e1Compare March 16, 2026 14:49

jason810496 approved these changes Mar 17, 2026
View reviewed changes

Lee-W merged commit b3055d7 into apache:mainMar 17, 2026
75 checks passed

Lee-W deleted the lock-adpr branch March 17, 2026 03:35

Lee-W added a commit to astronomer/airflow that referenced this pull request Mar 17, 2026
[v3-1-test] Add row lock to ADRQ before Dag run creation (apache#60773)
443c0a4
(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

Lee-W added a commit to astronomer/airflow that referenced this pull request Mar 17, 2026
[v3-1-test] Add row lock to ADRQ before Dag run creation (apache#60773)
d8842f1
(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

Lee-W added a commit to astronomer/airflow that referenced this pull request Mar 17, 2026
[v3-1-test] Add row lock to ADRQ before Dag run creation (apache#60773)
2fa2933
(cherry picked from commit b3055d7) Co-authored-by: Wei Lee <weilee.rx@gmail.com>

Lee-W added a commit to astronomer/airflow that referenced this pull request Mar 17, 2026
fixup! [v3-1-test] Add row lock to ADRQ before Dag run creation (apac…
600b878
…he#60773) (cherry picked from commit b3055d7)

fat-catTW pushed a commit to fat-catTW/airflow that referenced this pull request Mar 22, 2026
Add row lock to ADRQ before Dag run creation (apache#60773)
4825f5a

abhijeets25012-tech pushed a commit to abhijeets25012-tech/airflow that referenced this pull request Apr 9, 2026
Add row lock to ADRQ before Dag run creation (apache#60773)
54f9464

Mercury2699 mentioned this pull request Jun 4, 2026
Duplicate DAG runs for PartitionedAssetTimetable with multiple schedulers (HA) — race not covered by #60773 #68045
Closed

bujjibabukatta mentioned this pull request Jun 5, 2026
fix(scheduler): add row lock to AssetPartitionDagRun fetch to prevent… #68061
Closed