Skip to content

gh-127085: Fix memoryview->exports race condition. - #127412

Merged
kumaraditya303 merged 15 commits into
python:mainfrom
LindaSummer:bugfix/gh-127085
Dec 16, 2024
Merged

gh-127085: Fix memoryview->exports race condition.#127412
kumaraditya303 merged 15 commits into
python:mainfrom
LindaSummer:bugfix/gh-127085

Conversation

@LindaSummer

@LindaSummerLindaSummer commented Nov 29, 2024•
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Contributor

Issue

Close#127085 .

Proposed Changes

  • Make memoryview->exports protected by critical section.
  • Add unit test from the issue.

Cause of the problem

With TSAN, I find that the root cause should be racing between memory_getbuf and memory_releasebuf.

After adding the critical section, this racing has been fixed.

Here are the TSAN outputs.

Python 3.14.0a2+ experimental free-threading build (heads/main:db5c5763f3, Nov 28 2024, 21:41:41) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
warning: can't use pyrepl: No module named 'msvcrt'
>>> import multiprocessing.shared_memory
from threading import Thread
obj = multiprocessing.shared_memory.ShareableList("Uq..SeDAmB+EBrkLl.SG.Z+Z.ZdsV..wT+zLxKwdN\b")
for x in range(10):
Thread(target=obj.count, args=(1,)).start()
del obj
>>> >>> >>> >>> >>> ... ... ==================
WARNING: ThreadSanitizer: data race (pid=3465000)
Read of size 8 at 0x7f056b7e1010 by thread T2:
#0 memory_getbuf Objects/memoryobject.c:1592 (python+0x5c0b9c)
#1 PyObject_GetBuffer Objects/abstract.c:442 (python+0x4d8844)
#2 unpack_from Modules/clinic/_struct.c.h:370 (_struct.cpython-314td-x86_64-linux-gnu.so+0xc66a)
#3 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9ec7)
#4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
#5 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
#6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d0fa)
#7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
#8 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
#9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x649082)
#11 vectorcall_unbound Objects/typeobject.c:2776 (python+0x649082)
#12 vectorcall_method Objects/typeobject.c:2807 (python+0x649082)
#13 slot_sq_item Objects/typeobject.c:9617 (python+0x649446)
#14 PySequence_GetItem Objects/abstract.c:1847 (python+0x4dd14f)
#15 iter_iternext Objects/iterobject.c:66 (python+0x572a7d)
#16 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3927 (python+0x794437)
#17 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x550b43)
#18 gen_send_ex2 Objects/genobject.c:245 (python+0x550b43)
#19 gen_iternext Objects/genobject.c:618 (python+0x5514e3)
#20 iternext Objects/abstract.c:2874 (python+0x4d60e7)
#21 PyIter_Next Objects/abstract.c:2924 (python+0x4dffb4)
#22 builtin_sum_impl Python/bltinmodule.c:2716 (python+0x764943)
#23 builtin_sum Python/clinic/bltinmodule.c.h:1167 (python+0x766254)
#24 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9ec7)
#25 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
#26 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
#27 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d0fa)
#28 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
#29 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
#30 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#31 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x515230)
#32 method_vectorcall Objects/classobject.c:93 (python+0x51603f)
#33 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
#34 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
#35 PyObject_Call Objects/call.c:373 (python+0x513578)
#36 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1745 (python+0x782ad6)
#37 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
#38 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
#39 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#40 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x51623d)
#41 method_vectorcall Objects/classobject.c:71 (python+0x51623d)
#42 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
#43 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
#44 PyObject_Call Objects/call.c:373 (python+0x513578)
#45 thread_run Modules/_threadmodule.c:337 (python+0x985d20)
#46 pythread_wrapper Python/thread_pthread.h:242 (python+0x8aaff3)
Previous write of size 8 at 0x7f056b7e1010 by thread T1:
#0 memory_getbuf Objects/memoryobject.c:1592 (python+0x5c0ba8)
#1 PyObject_GetBuffer Objects/abstract.c:442 (python+0x4d8844)
#2 unpack_from Modules/clinic/_struct.c.h:370 (_struct.cpython-314td-x86_64-linux-gnu.so+0xc66a)
#3 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9ec7)
#4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
#5 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
#6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d0fa)
#7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
#8 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
#9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x649082)
#11 vectorcall_unbound Objects/typeobject.c:2776 (python+0x649082)
#12 vectorcall_method Objects/typeobject.c:2807 (python+0x649082)
#13 slot_sq_item Objects/typeobject.c:9617 (python+0x649446)
#14 PySequence_GetItem Objects/abstract.c:1847 (python+0x4dd14f)
#15 iter_iternext Objects/iterobject.c:66 (python+0x572a7d)
#16 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3927 (python+0x794437)
#17 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x550b43)
#18 gen_send_ex2 Objects/genobject.c:245 (python+0x550b43)
#19 gen_iternext Objects/genobject.c:618 (python+0x5514e3)
#20 iternext Objects/abstract.c:2874 (python+0x4d60e7)
#21 PyIter_Next Objects/abstract.c:2924 (python+0x4dffb4)
#22 builtin_sum_impl Python/bltinmodule.c:2716 (python+0x764943)
#23 builtin_sum Python/clinic/bltinmodule.c.h:1167 (python+0x766254)
#24 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9ec7)
#25 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
#26 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
#27 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d0fa)
#28 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
#29 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
#30 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#31 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x515230)
#32 method_vectorcall Objects/classobject.c:93 (python+0x51603f)
#33 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
#34 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
#35 PyObject_Call Objects/call.c:373 (python+0x513578)
#36 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1745 (python+0x782ad6)
#37 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
#38 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
#39 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#40 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x51623d)
#41 method_vectorcall Objects/classobject.c:71 (python+0x51623d)
#42 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
#43 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
#44 PyObject_Call Objects/call.c:373 (python+0x513578)
#45 thread_run Modules/_threadmodule.c:337 (python+0x985d20)
#46 pythread_wrapper Python/thread_pthread.h:242 (python+0x8aaff3)
SUMMARY: ThreadSanitizer: data race Objects/memoryobject.c:1592 in memory_getbuf
==================
>>> >>> >>> Exception ignored in: <function SharedMemory.__del__ at 0x7f056b26f350>
Traceback (most recent call last):
File "/home/mizuki/project/cpython/Lib/multiprocessing/shared_memory.py", line 189, in __del__
self.close()
File "/home/mizuki/project/cpython/Lib/multiprocessing/shared_memory.py", line 229, in close
self._buf.release()
BufferError: memoryview has 2 exported buffers
python: Objects/memoryobject.c:1143: memory_dealloc: Assertion `self->exports == 0' failed.
/home/mizuki/project/cpython/Lib/multiprocessing/resource_tracker.py:276: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown: {'/psm_3a39690b'}
warnings.warn(

Python 3.14.0a2+ experimental free-threading build (heads/main-dirty:db5c5763f3, Nov 28 2024, 21:44:57) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
warning: can't use pyrepl: No module named 'msvcrt'
>>> import multiprocessing.shared_memory
from threading import Thread
obj = multiprocessing.shared_memory.ShareableList("Uq..SeDAmB+EBrkLl.SG.Z+Z.ZdsV..wT+zLxKwdN\b")
for x in range(10):
Thread(target=obj.count, args=(1,)).start()
del obj
>>> >>> >>> >>> >>> ... ... ==================
WARNING: ThreadSanitizer: data race (pid=3470244)
Read of size 8 at 0x7f221cf8fb70 by thread T1:
#0 memory_getbuf Objects/memoryobject.c:1593 (python+0x5c0ba8)
#1 PyObject_GetBuffer Objects/abstract.c:442 (python+0x4d8844)
#2 unpack_from Modules/clinic/_struct.c.h:370 (_struct.cpython-314td-x86_64-linux-gnu.so+0xc66a)
#3 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9edc)
#4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
#5 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
#6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d10f)
#7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
#8 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
#9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x649097)
#11 vectorcall_unbound Objects/typeobject.c:2776 (python+0x649097)
#12 vectorcall_method Objects/typeobject.c:2807 (python+0x649097)
#13 slot_sq_item Objects/typeobject.c:9617 (python+0x64945b)
#14 PySequence_GetItem Objects/abstract.c:1847 (python+0x4dd14f)
#15 iter_iternext Objects/iterobject.c:66 (python+0x572a7d)
#16 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3927 (python+0x79444c)
#17 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x550b43)
#18 gen_send_ex2 Objects/genobject.c:245 (python+0x550b43)
#19 gen_iternext Objects/genobject.c:618 (python+0x5514e3)
#20 iternext Objects/abstract.c:2874 (python+0x4d60e7)
#21 PyIter_Next Objects/abstract.c:2924 (python+0x4dffb4)
#22 builtin_sum_impl Python/bltinmodule.c:2716 (python+0x764958)
#23 builtin_sum Python/clinic/bltinmodule.c.h:1167 (python+0x766269)
#24 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9edc)
#25 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
#26 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
#27 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d10f)
#28 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
#29 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
#30 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#31 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x515230)
#32 method_vectorcall Objects/classobject.c:93 (python+0x51603f)
#33 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
#34 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
#35 PyObject_Call Objects/call.c:373 (python+0x513578)
#36 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1745 (python+0x782aeb)
#37 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
#38 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
#39 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#40 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x51623d)
#41 method_vectorcall Objects/classobject.c:71 (python+0x51623d)
#42 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
#43 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
#44 PyObject_Call Objects/call.c:373 (python+0x513578)
#45 thread_run Modules/_threadmodule.c:337 (python+0x985d35)
#46 pythread_wrapper Python/thread_pthread.h:242 (python+0x8ab008)
Previous write of size 8 at 0x7f221cf8fb70 by thread T2:
#0 memory_releasebuf Objects/memoryobject.c:1603 (python+0x5bd1d3)
#1 PyBuffer_Release Objects/abstract.c:812 (python+0x4d91bd)
#2 unpack_from Modules/clinic/_struct.c.h:396 (_struct.cpython-314td-x86_64-linux-gnu.so+0xc8d4)
#3 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9edc)
#4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
#5 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
#6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d10f)
#7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
#8 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
#9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x649097)
#11 vectorcall_unbound Objects/typeobject.c:2776 (python+0x649097)
#12 vectorcall_method Objects/typeobject.c:2807 (python+0x649097)
#13 slot_sq_item Objects/typeobject.c:9617 (python+0x64945b)
#14 PySequence_GetItem Objects/abstract.c:1847 (python+0x4dd14f)
#15 iter_iternext Objects/iterobject.c:66 (python+0x572a7d)
#16 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3927 (python+0x79444c)
#17 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x550b43)
#18 gen_send_ex2 Objects/genobject.c:245 (python+0x550b43)
#19 gen_iternext Objects/genobject.c:618 (python+0x5514e3)
#20 iternext Objects/abstract.c:2874 (python+0x4d60e7)
#21 PyIter_Next Objects/abstract.c:2924 (python+0x4dffb4)
#22 builtin_sum_impl Python/bltinmodule.c:2716 (python+0x764958)
#23 builtin_sum Python/clinic/bltinmodule.c.h:1167 (python+0x766269)
#24 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9edc)
#25 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
#26 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
#27 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d10f)
#28 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
#29 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
#30 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#31 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x515230)
#32 method_vectorcall Objects/classobject.c:93 (python+0x51603f)
#33 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
#34 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
#35 PyObject_Call Objects/call.c:373 (python+0x513578)
#36 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1745 (python+0x782aeb)
#37 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
#38 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
#39 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
#40 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x51623d)
#41 method_vectorcall Objects/classobject.c:71 (python+0x51623d)
#42 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
#43 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
#44 PyObject_Call Objects/call.c:373 (python+0x513578)
#45 thread_run Modules/_threadmodule.c:337 (python+0x985d35)
#46 pythread_wrapper Python/thread_pthread.h:242 (python+0x8ab008)
SUMMARY: ThreadSanitizer: data race Objects/memoryobject.c:1593 in memory_getbuf
==================
>>> >>> Exception ignored in: <function SharedMemory.__del__ at 0x7f221cf81eb0>
Traceback (most recent call last):
File "/home/mizuki/project/cpython/Lib/multiprocessing/shared_memory.py", line 189, in __del__
self.close()
File "/home/mizuki/project/cpython/Lib/multiprocessing/shared_memory.py", line 229, in close
self._buf.release()
BufferError: memoryview has 7 exported buffers
python: Objects/memoryobject.c:1143: memory_dealloc: Assertion `self->exports == 0' failed.
/home/mizuki/project/cpython/Lib/multiprocessing/resource_tracker.py:276: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown: {'/psm_616253a2'}
warnings.warn(

@ghost

ghost commented Nov 29, 2024•
edited by ghost
Loading

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app

bedevere-appBot commented Nov 29, 2024

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-appbedevere-appBot mentioned this pull request Nov 29, 2024
Closed
@bedevere-app

bedevere-appBot commented Nov 29, 2024

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app

bedevere-appBot commented Nov 30, 2024

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@ZeroIntensityZeroIntensity added the needs backport to 3.13 bugs and security fixes label Nov 30, 2024
ZeroIntensity
ZeroIntensity reviewed Nov 30, 2024
View reviewed changes

@ZeroIntensityZeroIntensity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing! I'm always happy to see new contributors 😄. Please address my comments and let me know if you have any questions.

Comment threadLib/test/test_free_threading/test_memoryview.py Outdated
Comment threadLib/test/test_free_threading/test_memoryview.py Outdated
Comment threadLib/test/test_free_threading/test_memoryview.py Outdated
Comment threadLib/test/test_free_threading/test_memoryview.py Outdated
Comment threadLib/test/test_free_threading/test_memoryview.py Outdated
Comment threadObjects/memoryobject.c Outdated
Comment threadObjects/memoryobject.c
@bedevere-app

bedevere-appBot commented Nov 30, 2024

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app

bedevere-appBot commented Nov 30, 2024

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@ZeroIntensity

ZeroIntensity commented Nov 30, 2024

Copy link
Copy Markdown
Member

Please add a news entry so the bot stops spamming.

ZeroIntensity
ZeroIntensity reviewed Nov 30, 2024
View reviewed changes
Comment threadObjects/memoryobject.c
ZeroIntensity
ZeroIntensity reviewed Nov 30, 2024
View reviewed changes
Comment threadLib/test/test_memoryview.py
picnixz
picnixz reviewed Nov 30, 2024
View reviewed changes
Comment threadLib/test/test_memoryview.py Outdated
Comment threadLib/test/test_memoryview.py Outdated
Comment threadObjects/memoryobject.c
ZeroIntensity
ZeroIntensity reviewed Dec 1, 2024
View reviewed changes

@ZeroIntensityZeroIntensity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some formatting nitpicks. I'll do a more thorough audit of the thread safety later today.

Comment threadObjects/memoryobject.c Outdated
Comment threadObjects/memoryobject.c Outdated
@LindaSummer

LindaSummer commented Dec 1, 2024

Copy link
Copy Markdown
ContributorAuthor

Just some formatting nitpicks. I'll do a more thorough audit of the thread safety later today.

Hi @ZeroIntensity ,

Thanks very much for your patience and warm help! 😊

Wish a nice day!

Best Regards,
Edward

@picnixz

picnixz commented Dec 2, 2024

Copy link
Copy Markdown
Member

@LindaSummer Please do not update the branch with the latest main unless you have conflicts to resolve. This wastes resources and notifies everyone subscribed to the PR.

@LindaSummer

LindaSummer commented Dec 2, 2024

Copy link
Copy Markdown
ContributorAuthor

OK, on the basis that there will be a follow-up PR, this LGTM. Thanks, @LindaSummer!

Hi @ZeroIntensity , @picnixz ,

Thanks again for your guidance and patience! ❤

Best Regards,
Edward

@LindaSummer

LindaSummer commented Dec 5, 2024

Copy link
Copy Markdown
ContributorAuthor

Hi @ZeroIntensity ,

Sorry to bother you again. 😊

This PR has been pending for three days in awaiting core review.

Should I mention someone in this thread for next steps? 😊

Best Regards,
Edward

@ZeroIntensity

ZeroIntensity commented Dec 5, 2024

Copy link
Copy Markdown
Member

No worries. Typically the topic-free-threading label is enough to get one of the free threading experts to look at it, but not always. Tag me next week if nobody has taken a look.

@picnixzpicnixz mentioned this pull request Dec 7, 2024
Open
ZeroIntensity
ZeroIntensity reviewed Dec 7, 2024
View reviewed changes

@ZeroIntensityZeroIntensity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I didn't notice the blurb before. That needs to get changed.

Comment threadMisc/NEWS.d/next/Core_and_Builtins/2024-11-30-23-35-45.gh-issue-127085.KLKylb.rst Outdated
@kumaraditya303kumaraditya303 self-assigned this Dec 9, 2024
@kumaraditya303

kumaraditya303 commented Dec 9, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

I'll be reviewing this by next week

kumaraditya303
kumaraditya303 approved these changes Dec 16, 2024
View reviewed changes

@kumaraditya303kumaraditya303 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this doesn't fix all of the thread safety issues but is a step in right direction so I'm merging it

@kumaraditya303
kumaraditya303 merged commit 4937ba5 into python:mainDec 16, 2024
@miss-islington-app

miss-islington-appBot commented Dec 16, 2024

Copy link
Copy Markdown

Thanks @LindaSummer for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

@miss-islington-app

miss-islington-appBot commented Dec 16, 2024

Copy link
Copy Markdown

Sorry, @LindaSummer and @kumaraditya303, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4937ba54c0ff7cc4a83d7345d398b804365af2d6 3.13

@kumaraditya303

kumaraditya303 commented Dec 16, 2024

Copy link
Copy Markdown
Contributor

I'll take of backporting this if needed as part of #127716

@bedevere-bot

bedevere-bot commented Dec 16, 2024

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Android 3.x has failed when building commit 4937ba5.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1594/builds/825) and take a look at the build logs.
  4. Check if the failure is related to this commit (4937ba5) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1594/builds/825

Failed tests:

  • test_memoryview

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
File "/data/user/0/org.python.testbed/files/python/lib/python3.14/test/test_memoryview.py", line 742, in test_racing_getbuf_and_releasebuffrom multiprocessing.managers import SharedMemoryManager
ImportError: cannot import name 'SharedMemoryManager' from 'multiprocessing.managers' (/data/user/0/org.python.testbed/files/python/lib/python3.14/multiprocessing/managers.py)

@bedevere-bot

bedevere-bot commented Dec 16, 2024

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.x has failed when building commit 4937ba5.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1380/builds/2128) and take a look at the build logs.
  4. Check if the failure is related to this commit (4937ba5) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1380/builds/2128

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):

@bedevere-bot

bedevere-bot commented Dec 16, 2024

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Android 3.x has failed when building commit 4937ba5.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1591/builds/728) and take a look at the build logs.
  4. Check if the failure is related to this commit (4937ba5) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1591/builds/728

Failed tests:

  • test_memoryview

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
File "/data/user/0/org.python.testbed/files/python/lib/python3.14/test/test_memoryview.py", line 742, in test_racing_getbuf_and_releasebuffrom multiprocessing.managers import SharedMemoryManager
ImportError: cannot import name 'SharedMemoryManager' from 'multiprocessing.managers' (/data/user/0/org.python.testbed/files/python/lib/python3.14/multiprocessing/managers.py)

@mhsmith

mhsmith commented Dec 16, 2024

Copy link
Copy Markdown
Member

iOS and Android don't support subprocesses, and therefore don't support the multiprocessing. It looks like this test doesn't actually use multiple processes, so maybe it can be fixed to avoid the use of multiprocessing. Otherwise, it'll have to be given a decorator to skip it when multiprocessing is unavailable.

@LindaSummer

LindaSummer commented Dec 17, 2024

Copy link
Copy Markdown
ContributorAuthor

iOS and Android don't support subprocesses, and therefore don't support the multiprocessing. It looks like this test doesn't actually use multiple processes, so maybe it can be fixed to avoid the use of multiprocessing. Otherwise, it'll have to be given a decorator to skip it when multiprocessing is unavailable.

Hi @mhsmith ,

Thanks for your suggestion! 😊

I will try to fix the test case issue.

Best Regards,
Edward

@freakboy3742freakboy3742 mentioned this pull request Dec 17, 2024
Merged
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Jan 8, 2025
@LindaSummer@srinivasreddy
pythongh-127085: fix some data races in memoryview in free-threading (p…
0be115d
…ython#127412)
@mhsmithmhsmith mentioned this pull request Jan 23, 2025
Closed
@hugovk

hugovk commented Feb 26, 2025

Copy link
Copy Markdown
Member

I'll take of backporting this if needed as part of #127716

@kumaraditya303 Triage: Please could you make the backport if needed? Otherwise let's remove the label. Thanks!

@hugovkhugovk removed the needs backport to 3.13 bugs and security fixes label May 22, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@picnixzpicnixzpicnixz left review comments
@ZeroIntensityZeroIntensityZeroIntensity approved these changes
@kumaraditya303kumaraditya303kumaraditya303 approved these changes

Assignees

@kumaraditya303kumaraditya303

Labels

topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Calling ShareableList.count in threads aborts: Assertion 'self->exports == 0' failed

7 participants

@LindaSummer@ZeroIntensity@picnixz@kumaraditya303@bedevere-bot@mhsmith@hugovk