Cannot cleanly kill a subprocess using high-level asyncio APIs

[rabraham]

BPO	43884
Nosy	@asvetlov, @1st1, @sweeneyde, @kumaraditya303, @skeggse
PRs	bpo-43884: Fix asyncio subprocess kill process cleanly when process is blocked #31611 gh-88050: Fix asyncio subprocess kill process cleanly when process is blocked #32073
Files	kill_subprocess.py: Kill subprocess using the high-level asyncio subprocess APIs

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-04-19.02:47:38.245>
labels = ['3.11', 'type-bug', 'expert-asyncio']
title = 'Cannot cleanly kill a subprocess using high-level asyncio APIs'
updated_at = <Date 2022-03-23.09:08:45.631>
user = 'https://bugs.python.org/rabraham'

bugs.python.org fields:

activity = <Date 2022-03-23.09:08:45.631>
actor = 'kumaraditya'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2021-04-19.02:47:38.245>
creator = 'rabraham'
dependencies = []
files = ['49965']
hgrepos = []
issue_num = 43884
keywords = ['patch']
message_count = 6.0
messages = ['391349', '391350', '391369', '391379', '399937', '415859']
nosy_count = 6.0
nosy_names = ['asvetlov', 'yselivanov', 'rabraham', 'Dennis Sweeney', 'kumaraditya', 'skeggse']
pr_nums = ['31611', '32073']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43884'
versions = ['Python 3.11']

[rabraham]

There doesn't appear to be a way to prematurely kill a subprocess using the high-level asyncio subprocess APIs (https://docs.python.org/3.9/library/asyncio-subprocess.html) without getting a traceback on exit.

On exit, the attached program writes the following to stderr:

$ python3.9 kill_subprocess.py

Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x1065f0dc0>
Traceback (most recent call last):
 ...
 raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

If I uncomment # process._transport.close() or comment asyncio.sleep(1), the walkback disappears. (I get the same behavior in python 3.8. I haven't tried other python versions.)

[rabraham]

Reproducing the program here:

 import asyncio
 
 async def test():
 process = await asyncio.create_subprocess_shell(
 "sleep 2 && echo done",
 stdout=asyncio.subprocess.PIPE,
 )
 await asyncio.sleep(1)
 process.kill()
 await process.wait()
 # process._transport.close()
 
 asyncio.run(test())

Can I use the high-level API to kill a subprocess cleanly without having to access the protected member process._transport? Seems like an oversight perhaps?

[sweeneyde]

Running kill_subprocess.py on Windows 10, I get these results:

Python 3.7.2 (tags/v3.7.2:9a3ffc0492)
- raises NotImplementedError in base_events.py, _make_subprocess_transport
Python 3.8.2 (tags/v3.8.2:7b3ab59)
- Success
Python 3.9.0 (tags/v3.9.0:9cf6752)
- Success
Python 3.10.0a6 (tags/v3.10.0a6:cc12888)
- Success

What is your OS?

[rabraham]

I see this on MacOS and Linux, but I suspect any Unix-like system would have the same behavior.

[skeggse]

I'm also experiencing this, with virtually identical code, on macOS 10.15.7. The given fix (process._transport.close()) also works for me, so I'm just using that workaround for the time being.