Add an API to indicate if the process might have multiple threads

[gpshead]

BPO	46493
Nosy	@gpshead

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-01-23.21:50:43.058>
labels = ['type-feature', 'library', '3.11']
title = 'Add an API to indicate if the process might have multiple threads'
updated_at = <Date 2022-01-23.21:50:43.058>
user = 'https://github.com/gpshead'

bugs.python.org fields:

activity = <Date 2022-01-23.21:50:43.058>
actor = 'gregory.p.smith'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2022-01-23.21:50:43.058>
creator = 'gregory.p.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 46493
keywords = []
message_count = 1.0
messages = ['411420']
nosy_count = 1.0
nosy_names = ['gregory.p.smith']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46493'
versions = ['Python 3.11']

[gpshead]

It'd be handy to have a function to determine if there are multiple threads in the current process or not - at least on POSIXish systems. This would be useful anytime a library is trying to use os.fork(), as fork() is not safe in a multi-threaded process.

Motivation: I want to _use_ this API to consider raising RuntimeWarnings in libraries that call fork() to highlight the problem to people attempting to use that code in multithreaded processes.

As POSIXy OSes don't usually have a simple API to get this information, one implementation such as this could make sense:

def is_process_multithreaded() -> bool:
 """Are there multiple threads in this process? OSError if no such OS API is available."""
 threads = 0
 ourself = str(os.gettid())
 i_feel_seen = False
 try:
 # Linux, NetBSD, any others?
 with os.scandir('/proc/self/task') as tasks:
 for task_dir_entry in tasks:
 # tid named subdirs should be the only thing that exists.
 # We do a simple conformity check just in case.
 if task_dir_entry.name.isdigit():
 if task_dir_entry.name == ourself:
 i_feel_seen = True
 threads += 1
 if i_feel_seen and threads > 1:
 return True # Multiple threads confirmed.
 except EnvironmentError:
 raise OSError('Unable to count threads on this platform.')
 if i_feel_seen and threads == 1:
 return False
 else:
 raise OSError(f'Unclear. Found {threads} in /proc/self/task and did not find ourself.')

macOS has mach darwin kernel APIs that can do this. Not well documented but they do work - https://stackoverflow.com/questions/21478229/how-to-count-number-of-alive-threads-in-ios

FreeBSD has APIs that can do this (see FreeBSD usr.bin/procstat/ source).

configure.ac checks and an extension module would be needed to integrate those.

My use case is not relevant to Windows (a platform unburdened by fork) but Windows APIs to answer the question exist if anyone has a reason to want this there as well - https://stackoverflow.com/questions/3749668/how-to-query-the-thread-count-of-a-process-using-the-regular-windows-c-c-apis

[gpshead]

The C code I added in #100229 to implement the os.fork() warning when threads can be detected #100228 is a good start for this.

[encukou]

Is there another use case?
If we can't do this reliably on all platforms, I'd rather leave it to third-party libraries. (A warning can be skipped if there's any doubt, but for other use cases this might need to “fail” differently.)

[gpshead]

I think the internal function is sufficient. I don't see a reason for us to make it public.