gh-125828 : Fix 'get_value' missing on [Bounded]Semaphore on multiprocessing MacOSX

[YvesDup]

This proposal is a workaround to fix absence of 'sem_getvalue' C function in the Semaphore MacOSX implementation.

Alls unit tests succeed except on test.test_concurrent_futures.test_init relative to the Resource Tracker manager.

• Issue: Multiprocessing Semaphore and BoundedSemaphore - incomplete features ('get_value' missing) on MacOSX. #125828

[picnixz]

Is there an alternative way to do it without a temporary class like this? (I don't have the answer but I feel it overcomplicates the code base)

[picnixz]

Wouldn't it be better to handle the SemLock class differently at the C level?

[YvesDup]

Wouldn't it be better to handle the SemLock class differently at the C level?

As I mentioned in the problem, I'm waiting for feedback (from the Mac OS team ?) before going any further.

A C implementation is an option, even if it seems more complicated than Python.
That said, I'm available to go deep into the SemLock C code

[picnixz]

Actually, since the SemLock class is implemented in C, it's preferrable to first patch it there IMO. But let's CC some people I know they are on macOS: @ned-deily@ronaldoussoren@freakboy3742.

[freakboy3742]

I can't profess any particular expertise in this area. However, given the underlying implementation is written in C, I agree that it makes some sense for the implementation of this workaround to also be in C (rather than have an underlying implementation that returns NotImplemented, wrapped by a layer that overrides that.

[YvesDup]

rather than have an underlying implementation that returns NotImplemented, wrapped by a layer that overrides that.

The main idea is to listen to the acquire and release methods to update a shared counter, and to return the counter value when the get_value method is invoked.

I am going to open a new PR with a workaround written in C.

[ZeroIntensity]

This needs a NEWS entry :)

[ronaldoussoren]

Is get_value part of the public API for these classes? The name suggests it is, but the method is not document in the reference docs and does not have a docsstring.

I haven't looked with enough detail at the code to be certain, and am not a multiprocessing expert, but I the code doesn't seem correct. In particular, how does the implementation handle blocking callers of acquire when the count is 0 but not when it is larger than 0?

[YvesDup]

is get_value part of the public API for these classes? The name suggests it is, but the method is not document in the reference docs and does not have a docsstring.

Yes, it is and I confirm this is not documented. This public method is rarely used to get a value of semaphore, except in unit tests. I should open an issue about this case.

In particular, how does the implementation handle blocking callers of acquire when the count is 0 but not when it is larger than 0?

As I explain above, this class only listen to acquire and release to update shared counter and then call respectively acquire and releasemethods of _semlock class. When get_value is invoked, the class returns the counter value, and never calls the semlock._get_value method (which raises NotImplementedError).

In this way, the behaviour of semaphores is always done at a low level in the implementation of the semlockclass in the semaphore module (at C level).

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #32588791 | 📁 Comparing e69bfd8 against main (299cf3c)

🔍 Preview build

194 files changed · ± 191 modified · - 3 deleted

±Modified

• glossary.html
• index.html
• c-api/allocation.html
• c-api/buffer.html
• c-api/bytearray.html
• c-api/bytes.html
• c-api/capsule.html
• c-api/code.html
• c-api/concrete.html
• c-api/descriptor.html
• and 181 more...

-Deleted

• c-api/sentinel.html
• c-api/slots.html
• deprecations/soft-deprecations.html