Uh oh!
There was an error while loading. Please reload this page.
Fix: Use a short-live session instead of passing sessionDeps into DagRunWaiter - #64520
Merged
Merged
Conversation
1 task
uranusjr
approved these changes
Mar 31, 2026
sjyangkevinforce-pushed
the
fix-dag-wait-endpoint-session
branch
from
March 31, 2026 12:14
17fe834 to
477e2a6Comparesjyangkevinforce-pushed
the
fix-dag-wait-endpoint-session
branch
from
March 31, 2026 15:40
477e2a6 to
4461b43Comparesjyangkevin
marked this pull request as ready for review
March 31, 2026 17:45
sjyangkevin
requested review from
bugraoz93, choo121600, ephraimbuddy, jason810496, pierrejeambrun, rawwar and shubhamraj-git
as code ownersMarch 31, 2026 17:45
potiuk
approved these changes
Apr 1, 2026
Uh oh!
There was an error while loading. Please reload this page.
Subham-KRLX pushed a commit
to Subham-KRLX/airflow
that referenced
this pull request
Apr 3, 2026
Suraj-kumar00 pushed a commit
to Suraj-kumar00/airflow
that referenced
this pull request
Apr 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
related: #64415
Why
The
SessionDepsession is held open from the moment the endpoint function starts until theStreamingResponsegenerator is fully consumed and FastAPI cleans up the dependency. During the polling loop (whiledag_run.statenot inState.finished_dr_states: await asyncio.sleep(self.interval)), the sync DB connection sits idle but checked out from the pool. If a DAG run takes minutes or hours to complete, that connection is locked for the entire duration. With many concurrent/waitrequests, this can exhaust the connection pool.This PR improves the above by opening a short-live session in
_serialize_xcoms, similar to what have been done in_get_dag_run.#64415 (comment)
Test Cases
Tests should have been handled in test_collect_task and test_should_respond_200_immediately_for_finished_run
Was generative AI tooling used to co-author this PR?
Generated-by: [Claude Code Opus 4.6] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.