The new release of Pytest Asyncio (0.25.1 and 0.25.2) makes a number of asyncio tests fail

[potiuk]

After Pytest-asyncio 0.25.1 our canary tests started to fail with "RuntimeError: There is no current event loop in thread 'MainThread'." This happens randomly - a lot of times in Python 3.12 non-db tests, but sometimes they succeed there, and fail in other Python versions. It happens with xdist with xdist n=1 and without xdist alike.

Example failures:

• https://github.com/apache/airflow/actions/runs/12583221724/job/35070671929#step:6:904
• https://github.com/apache/airflow/actions/runs/12579028168/job/35066133378#step:6:904

Stacktrace:

_ ERROR at setup of TestRunPipelineJobTrigger.test_run_yields_success_event_on_successful_pipeline_state[4] _
[gw2] linux -- Python 3.11.11 /usr/local/bin/python
src/python/grpcio/grpc/_cython/_cygrpc/aio/common.pyx.pxi:184: in grpc._cython.cygrpc.get_working_loop
 ???
E RuntimeError: no running event loop

During handling of the above exception, another exception occurred:
providers/tests/google/cloud/triggers/test_vertex_ai.py:133: in pipeline_service_async_client
 return PipelineServiceAsyncClient(
/usr/local/lib/python3.11/site-packages/google/cloud/aiplatform_v1/services/pipeline_service/async_client.py:304: in __init__
 self._client = PipelineServiceClient(
/usr/local/lib/python3.11/site-packages/google/cloud/aiplatform_v1/services/pipeline_service/client.py:[889](https://github.com/apache/airflow/actions/runs/12583221724/job/35070671929#step:6:906): in __init__
 self._transport = transport_init(
/usr/local/lib/python3.11/site-packages/google/cloud/aiplatform_v1/services/pipeline_service/transports/grpc_asyncio.py:310: in __init__
 self._grpc_channel = channel_init(
/usr/local/lib/python3.11/site-packages/google/cloud/aiplatform_v1/services/pipeline_service/transports/grpc_asyncio.py:177: in create_channel
 return grpc_helpers_async.create_channel(
/usr/local/lib/python3.11/site-packages/google/api_core/grpc_helpers_async.py:290: in create_channel
 return aio.secure_channel(
/usr/local/lib/python3.11/site-packages/grpc/aio/_channel.py:621: in secure_channel
 return Channel(
/usr/local/lib/python3.11/site-packages/grpc/aio/_channel.py:367: in __init__
 self._loop = cygrpc.get_working_loop()
src/python/grpcio/grpc/_cython/_cygrpc/aio/common.pyx.pxi:186: in grpc._cython.cygrpc.get_working_loop
 ???
src/python/grpcio/grpc/_cython/_cygrpc/aio/common.pyx.pxi:190: in grpc._cython.cygrpc.get_working_loop
 ???
/usr/local/lib/python3.11/asyncio/events.py:681: in get_event_loop
 raise RuntimeError('There is no current event loop in thread %r.'
E RuntimeError: There is no current event loop in thread 'MainThread'.
_ ERROR at setup of TestRunPipelineJobTrigger.test_run_yields_success_event_on_successful_pipeline_state[8] _
[gw2] linux -- Python 3.11.11 /usr/local/bin/python
src/python/grpcio/grpc/_cython/_cygrpc/aio/common.pyx.pxi:184: in grpc._cython.cygrpc.get_working_loop
 ???
E RuntimeError: no running event loop

Failing tests:

ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestRunPipelineJobTrigger::test_run_yields_success_event_on_successful_pipeline_state[4] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestRunPipelineJobTrigger::test_run_yields_success_event_on_successful_pipeline_state[8] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestRunPipelineJobTrigger::test_run_yields_error_event_on_failed_pipeline_state[5] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestRunPipelineJobTrigger::test_run_yields_error_event_on_failed_pipeline_state[7] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomTrainingJobTrigger::test_run_yields_success_event_on_successful_pipeline_state[4] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomTrainingJobTrigger::test_run_yields_success_event_on_successful_pipeline_state[8] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomTrainingJobTrigger::test_run_yields_error_event_on_failed_pipeline_state[5] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomTrainingJobTrigger::test_run_yields_error_event_on_failed_pipeline_state[7] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomContainerTrainingJobTrigger::test_run_yields_success_event_on_successful_pipeline_state[4] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomContainerTrainingJobTrigger::test_run_yields_success_event_on_successful_pipeline_state[8] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomContainerTrainingJobTrigger::test_run_yields_error_event_on_failed_pipeline_state[5] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomContainerTrainingJobTrigger::test_run_yields_error_event_on_failed_pipeline_state[7] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomPythonPackageTrainingJobTrigger::test_run_yields_success_event_on_successful_pipeline_state[4] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomPythonPackageTrainingJobTrigger::test_run_yields_success_event_on_successful_pipeline_state[8] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomPythonPackageTrainingJobTrigger::test_run_yields_error_event_on_failed_pipeline_state[5] - RuntimeError: There is no current event loop in thread 'MainThread'.
ERROR providers/tests/google/cloud/triggers/test_vertex_ai.py::TestCustomPythonPackageTrainingJobTrigger::test_run_yields_error_event_on_failed_pipeline_state[7] - RuntimeError: There is no current event loop in thread 'MainThread'.

Also https://github.com/apache/airflow/actions/runs/12666006582/job/35297011421#step:6:4025

==================================== ERRORS ====================================
__ ERROR at teardown of TestAirbyteSyncTrigger.test_airbyte_run_sync_trigger ___
/usr/local/lib/python3.11/site-packages/pytest_asyncio/plugin.py:1151: in event_loop
 with _temporary_event_loop_policy(new_loop_policy), _provide_event_loop() as loop:
/usr/local/lib/python3.11/contextlib.py:144: in __exit__
 next(self.gen)
/usr/local/lib/python3.11/site-packages/pytest_asyncio/plugin.py:1168: in _provide_event_loop
 loop.run_until_complete(loop.shutdown_asyncgens())
/usr/local/lib/python3.11/asyncio/base_events.py:652: in run_until_complete
 raise RuntimeError('Event loop stopped before Future completed.')
E RuntimeError: Event loop stopped before Future completed.
----------------------------- Captured stdout call -----------------------------
[2025-01-08T07:44:52.728+0000] {base.py:66} INFO - Retrieving connection 'airbyte_default'
------------------------------ Captured log call -------------------------------
INFO airflow.hooks.base:base.py:66 Retrieving connection 'airbyte_default'
____________ ERROR at teardown of TestTrigger.test_trigger_run_bad _____________
/usr/local/lib/python3.11/site-packages/pytest_asyncio/plugin.py:1151: in event_loop
 with _temporary_event_loop_policy(new_loop_policy), _provide_event_loop() as loop:
/usr/local/lib/python3.11/contextlib.py:144: in __exit__
 next(self.gen)
/usr/local/lib/python3.11/site-packages/pytest_asyncio/plugin.py:1168: in _provide_event_loop
 loop.run_until_complete(loop.shutdown_asyncgens())
/usr/local/lib/python3.11/asyncio/base_events.py:652: in run_until_complete
 raise RuntimeError('Event loop stopped before Future completed.')
E RuntimeError: Event loop stopped before Future completed.
______ ERROR at teardown of TestWasbBlobSensorTrigger.test_running[True] _______
/usr/local/lib/python3.11/site-packages/pytest_asyncio/plugin.py:1151: in event_loop
 with _temporary_event_loop_policy(new_loop_policy), _provide_event_loop() as loop:
/usr/local/lib/python3.11/contextlib.py:144: in __exit__
 next(self.gen)
/usr/local/lib/python3.11/site-packages/pytest_asyncio/plugin.py:1168: in _provide_event_loop
 loop.run_until_complete(loop.shutdown_asyncgens())
/usr/local/lib/python3.11/asyncio/base_events.py:652: in run_until_complete
 raise RuntimeError('Event loop stopped before Future completed.')
E RuntimeError: Event loop stopped before Future completed.
______ ERROR at teardown of TestWasbBlobSensorTrigger.test_running[False] ______