str.maketrans on a mutating dict is not thread-safe

[eendebakpt]

Crash report

What happened?

A reproducer (in the format of Lib/tests/test_free_threading/xxx.py):

import unittest
from test.support import threading_helper

threading_helper.requires_working_threading(module=True)


class ItertoolsThreading(unittest.TestCase):

 @threading_helper.reap_threads
 def test_maketrans(self):
 number_of_iterations = 10
 number_of_attempts=100
 for _ in range(number_of_iterations):
 print(f'iteration {_}')
 d = {2000: 'a'}
 def work_iterator(dct):
 for idx in range(number_of_attempts):
 str.maketrans(dct)
 idx = 2000 + idx # outside range of small ints
 dct[idx] = chr(idx%16)
 dct.pop(idx, None)
 threading_helper.run_concurrently(work_iterator, nthreads=10, args=[d])

if __name__ == "__main__":
 unittest.main()
 print('done')

At first glance it seems that the iteration using PyDict_Next is not safe (but maybe there are more unsafe parts):

cpython/Objects/unicodeobject.c

Line 13158 in 24cc998

while (PyDict_Next(x, &i, &key, &value)) {

(found when reviewing #145129)

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Output from running 'python -VV' on the command line:

No response

Linked PRs

• gh-145142: Make str.maketrans safe under free-threading #145157
• [3.14] gh-145142: Make str.maketrans safe under free-threading (gh-145157) #145320

[colesbury]

Yes, PyDict_Next isn't thread-safe if there are concurrent modifications of the dict. We should wrap the loop with a critical section on x here.

Do you want to make the change?

[eendebakpt]

Do you want to make the change?

I am adding the easy label for newcomers. If not picked up, I will make the change.

[VanshAgarwal24036]

I am working on this issue.