Skip to content

Fix async connection retrieval in triggerer context - #55812

Merged
kaxil merged 1 commit into
apache:mainfrom
astronomer:fix-async-connection-secrets-backend
Sep 23, 2025
Merged

Fix async connection retrieval in triggerer context#55812
kaxil merged 1 commit into
apache:mainfrom
astronomer:fix-async-connection-secrets-backend

Conversation

@kaxil

@kaxilkaxil commented Sep 17, 2025

Copy link
Copy Markdown
Member

Add async support for secrets backends in _async_get_connection using sync_to_async to prevent event loop blocking. This resolves the 'async_to_sync forbidden in event loop' error when triggers call get_connection() and secrets backends need to be checked.

Also fix Databricks provider to use async connection methods in async contexts instead of cached properties that cause sync calls.

Fixes triggerer failures when providers access connections during async trigger execution.

Related PRs;


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Add async support for secrets backends in _async_get_connection using
sync_to_async to prevent event loop blocking. This resolves the
'async_to_sync forbidden in event loop' error when triggers call
get_connection() and secrets backends need to be checked.
Also fix Databricks provider to use async connection methods in async
contexts instead of cached properties that cause sync calls.
Fixes triggerer failures when providers access connections during
async trigger execution.
@kaxil
kaxilforce-pushed the fix-async-connection-secrets-backend branch from 0366cd9 to 3823c5bCompareSeptember 17, 2025 23:15
Comment threadtask-sdk/src/airflow/sdk/execution_time/context.py
@kaxilkaxil added this to the Airflow 3.1.0 milestone Sep 23, 2025
@kaxil
kaxil merged commit 2f76b41 into apache:mainSep 23, 2025
169 of 170 checks passed
@kaxil
kaxil deleted the fix-async-connection-secrets-backend branch September 23, 2025 02:16
kaxil added a commit that referenced this pull request Sep 23, 2025
Add async support for secrets backends in _async_get_connection using
sync_to_async to prevent event loop blocking. This resolves the
'async_to_sync forbidden in event loop' error when triggers call
get_connection() and secrets backends need to be checked.
Also fix Databricks provider to use async connection methods in async
contexts instead of cached properties that cause sync calls.
Fixes triggerer failures when providers access connections during
async trigger execution.
(cherry picked from commit 2f76b41)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@kaxil@ashb@jedcunningham@prdai