Uh oh!
There was an error while loading. Please reload this page.
Avoid using extra_dejson method on connection in KiotaRequestAdapterHook to avoid AsyncToSync RuntimeError - #55179
Conversation
…therwise expect: RuntimeError: You cannot use AsyncToSync in the same thread as an async event loop.
…async aget_connection method or not, if not fallback to sync_to_async solution
…or to execute_operator" This reverts commit 7d57de1.
Should you be adding a unit test to cover the following scenario?
|
Uh oh!
There was an error while loading. Please reload this page.
dabla
commented
Sep 9, 2025
What need's to be done to have this one merged? |
Uh oh!
There was an error while loading. Please reload this page.
bugraoz93
left a comment
There was a problem hiding this comment.
In general, looks good! CI needs some love and it would be safe to merge.
For the TODO part, can you include the issue number with (#...) like. That would make it easier to track
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
dabla
commented
Sep 15, 2025
All remarks have been fixed, issue number has been added in TODO, all tests have been cleaned and fixed even for PowerBI. |
Uh oh!
There was an error while loading. Please reload this page.
potiuk
commented
Sep 15, 2025
Nice! |
…ook to avoid AsyncToSync RuntimeError (apache#55179) * fix: do not use connection.extra_dejson until it's fixed in Airflow otherwise expect: RuntimeError: You cannot use AsyncToSync in the same thread as an async event loop. * refactor: Refactored get_async_conn to check whether BaseHook has an async aget_connection method or not, if not fallback to sync_to_async solution * refactor: Import Connection from version_compat * refactor: Moved render_template_fields from deferrable_operator to execute_operator * Revert "refactor: Moved render_template_fields from deferrable_operator to execute_operator" This reverts commit 7d57de1. * refactor: Refactored patch_hook_and_request_adapter * refactor: Patch directly on BaseHook class * refactor: Extracted patch_hook from patch_hook_and_request_adapter * refactor: Try fixing static checks * refactor: Fixed test connection * refactor: Added test on get_conn in MSGraphTrigger * refactor: Changed match on test_get_conn in TestKiotaRequestAdapterHook * refactor: fixed static checks * refactor: Inlined connection side effects * fix: Also add method when triggering next page in MSGraphOperator, otherwise it will always fall back to GET and not the currently used * refactor: templated fields don't make sense in MSGraphTrigger * refactor: Removed unused import of Sequence * refactor: Refactored tests for msgraph * refactor: Applied same for powerbi * refactor: Fixed static checks * refactor: Fixed test powerbi list * refactor: cross compat import timezone * refactor: Added issue number in TODO * refactor: Moved Context import out of type checking * refactor: Fixed hook patches TestPowerBITrigger * refactor: Fixed classpath assertion in TestPowerBITrigger
…ook to avoid AsyncToSync RuntimeError (apache#55179) * fix: do not use connection.extra_dejson until it's fixed in Airflow otherwise expect: RuntimeError: You cannot use AsyncToSync in the same thread as an async event loop. * refactor: Refactored get_async_conn to check whether BaseHook has an async aget_connection method or not, if not fallback to sync_to_async solution * refactor: Import Connection from version_compat * refactor: Moved render_template_fields from deferrable_operator to execute_operator * Revert "refactor: Moved render_template_fields from deferrable_operator to execute_operator" This reverts commit 7d57de1. * refactor: Refactored patch_hook_and_request_adapter * refactor: Patch directly on BaseHook class * refactor: Extracted patch_hook from patch_hook_and_request_adapter * refactor: Try fixing static checks * refactor: Fixed test connection * refactor: Added test on get_conn in MSGraphTrigger * refactor: Changed match on test_get_conn in TestKiotaRequestAdapterHook * refactor: fixed static checks * refactor: Inlined connection side effects * fix: Also add method when triggering next page in MSGraphOperator, otherwise it will always fall back to GET and not the currently used * refactor: templated fields don't make sense in MSGraphTrigger * refactor: Removed unused import of Sequence * refactor: Refactored tests for msgraph * refactor: Applied same for powerbi * refactor: Fixed static checks * refactor: Fixed test powerbi list * refactor: cross compat import timezone * refactor: Added issue number in TODO * refactor: Moved Context import out of type checking * refactor: Fixed hook patches TestPowerBITrigger * refactor: Fixed classpath assertion in TestPowerBITrigger
…ook to avoid AsyncToSync RuntimeError (apache#55179) * fix: do not use connection.extra_dejson until it's fixed in Airflow otherwise expect: RuntimeError: You cannot use AsyncToSync in the same thread as an async event loop. * refactor: Refactored get_async_conn to check whether BaseHook has an async aget_connection method or not, if not fallback to sync_to_async solution * refactor: Import Connection from version_compat * refactor: Moved render_template_fields from deferrable_operator to execute_operator * Revert "refactor: Moved render_template_fields from deferrable_operator to execute_operator" This reverts commit 7d57de1. * refactor: Refactored patch_hook_and_request_adapter * refactor: Patch directly on BaseHook class * refactor: Extracted patch_hook from patch_hook_and_request_adapter * refactor: Try fixing static checks * refactor: Fixed test connection * refactor: Added test on get_conn in MSGraphTrigger * refactor: Changed match on test_get_conn in TestKiotaRequestAdapterHook * refactor: fixed static checks * refactor: Inlined connection side effects * fix: Also add method when triggering next page in MSGraphOperator, otherwise it will always fall back to GET and not the currently used * refactor: templated fields don't make sense in MSGraphTrigger * refactor: Removed unused import of Sequence * refactor: Refactored tests for msgraph * refactor: Applied same for powerbi * refactor: Fixed static checks * refactor: Fixed test powerbi list * refactor: cross compat import timezone * refactor: Added issue number in TODO * refactor: Moved Context import out of type checking * refactor: Fixed hook patches TestPowerBITrigger * refactor: Fixed classpath assertion in TestPowerBITrigger
This PR fixes the "You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly." RuntimeError. The error originates from the Connection.extra_dejson property which now masks the secret.
So to temporarely avoid the issue, I just use:
json.loads(connection.extra)instead of:
connection.extra_dejsonBeside that, this PR will also check if the BaseHook has the new async aget_connection method, if it does it will use it as is, if it doesn't (which will be the case when azure provider is used with older airflow versions), then it will use the sync_to_async fallbaclk mechanism.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.