Skip to content

gh-118392: Update genrand_uint32 to use atomic operation#118393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
corona10 wants to merge 1 commit into from
Closed

gh-118392: Update genrand_uint32 to use atomic operation #118393

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions Include/internal/pycore_pyatomic_ft_wrappers.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -35,6 +35,8 @@ extern "C" {
_Py_atomic_load_uintptr_acquire(&value)
#define FT_ATOMIC_LOAD_PTR_RELAXED(value) \
_Py_atomic_load_ptr_relaxed(&value)
#define FT_ATOMIC_LOAD_INT(value) \
_Py_atomic_load_int(&value)
#define FT_ATOMIC_LOAD_UINT8(value) \
_Py_atomic_load_uint8(&value)
#define FT_ATOMIC_STORE_UINT8(value, new_value) \
Expand All@@ -49,6 +51,10 @@ extern "C" {
_Py_atomic_store_ssize_relaxed(&value, new_value)
#define FT_ATOMIC_STORE_UINT8_RELAXED(value, new_value) \
_Py_atomic_store_uint8_relaxed(&value, new_value)
#define FT_ATOMIC_STORE_INT(value, new_value) \
_Py_atomic_store_int(&value, new_value)
#define FT_ATMOIC_INT_INCREMENT(value) \
_Py_atomic_add_int(&value, 1)

#else
#define FT_ATOMIC_LOAD_PTR(value) value
Expand All@@ -60,13 +66,16 @@ extern "C" {
#define FT_ATOMIC_LOAD_PTR_ACQUIRE(value) value
#define FT_ATOMIC_LOAD_UINTPTR_ACQUIRE(value) value
#define FT_ATOMIC_LOAD_PTR_RELAXED(value) value
#define FT_ATOMIC_LOAD_INT(value) value
#define FT_ATOMIC_LOAD_UINT8(value) value
#define FT_ATOMIC_STORE_UINT8(value, new_value) value = new_value
#define FT_ATOMIC_STORE_PTR_RELAXED(value, new_value) value = new_value
#define FT_ATOMIC_STORE_PTR_RELEASE(value, new_value) value = new_value
#define FT_ATOMIC_STORE_UINTPTR_RELEASE(value, new_value) value = new_value
#define FT_ATOMIC_STORE_SSIZE_RELAXED(value, new_value) value = new_value
#define FT_ATOMIC_STORE_UINT8_RELAXED(value, new_value) value = new_value
#define FT_ATOMIC_STORE_INT(value, new_value) value = new_value
#define FT_ATMOIC_INT_INCREMENT(value) value++

#endif

Expand Down
14 changes: 7 additions & 7 deletions Modules/_randommodule.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -75,6 +75,7 @@
#include "pycore_modsupport.h" // _PyArg_NoKeywords()
#include "pycore_moduleobject.h" // _PyModule_GetState()
#include "pycore_pylifecycle.h" // _PyOS_URandomNonblock()
#include "pycore_pyatomic_ft_wrappers.h" // FT_LOAD_INT()

#ifdef HAVE_UNISTD_H
# include <unistd.h> // getpid()
Expand DownExpand Up@@ -139,7 +140,8 @@ genrand_uint32(RandomObject *self)
uint32_t *mt;

mt = self->state;
if (self->index >= N) { /* generate N words at one time */
int index = FT_ATOMIC_LOAD_INT(self->index);
if (index >= N) { /* generate N words at one time */
Comment on lines +143 to +144

@colesburycolesburyApr 29, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this will work well. This can end up returning the same generated numbers to different threads, which we don't want.

@corona10corona10Apr 29, 2024•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm in that case, should we change the algorithm(thread-safe one) or fine-grained locking for this use case?

@colesburycolesburyApr 29, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's a simple fix to the random module.

I commented on the issue. It'd be helpful to document the behavior and provide alternatives. For example, Java's java.util.Random includes the following comment:

Instances of java.util.Random are threadsafe. However, the concurrent use of the same java.util.Random instance across threads may encounter contention and consequent poor performance. Consider instead using ThreadLocalRandom in multithreaded designs.

@corona10corona10Apr 29, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I commented on the issue. It'd be helpful to document the behavior and provide alternatives. For example, Java's java.util.Random includes the following comment:

Okay sounds acceptable :)

int kk;

for (kk=0;kk<N-M;kk++) {
Expand All@@ -153,10 +155,10 @@ genrand_uint32(RandomObject *self)
y = (mt[N-1]&UPPER_MASK)|(mt[0]&LOWER_MASK);
mt[N-1] = mt[M-1] ^ (y >> 1) ^ mag01[y & 0x1U];

self->index = 0;
FT_ATOMIC_STORE_INT(self->index, 0);
}

y = mt[self->index++];
y = mt[FT_ATMOIC_INT_INCREMENT(self->index)];
y ^= (y >> 11);
y ^= (y << 7) & 0x9d2c5680U;
y ^= (y << 15) & 0xefc60000U;
Expand All@@ -175,7 +177,6 @@ genrand_uint32(RandomObject *self)
*/

/*[clinic input]
@critical_section
_random.Random.random

self: self(type="RandomObject *")
Expand All@@ -185,7 +186,7 @@ random() -> x in the interval [0, 1).

static PyObject *
_random_Random_random_impl(RandomObject *self)
/*[clinic end generated code: output=117ff99ee53d755c input=26492e52d26e8b7b]*/
/*[clinic end generated code: output=117ff99ee53d755c input=afb2a59cbbb00349]*/
{
uint32_t a=genrand_uint32(self)>>5, b=genrand_uint32(self)>>6;
return PyFloat_FromDouble((a*67108864.0+b)*(1.0/9007199254740992.0));
Expand DownExpand Up@@ -491,7 +492,6 @@ _random_Random_setstate_impl(RandomObject *self, PyObject *state)
}

/*[clinic input]
@critical_section
_random.Random.getrandbits

self: self(type="RandomObject *")
Expand All@@ -503,7 +503,7 @@ getrandbits(k) -> x. Generates an int with k random bits.

static PyObject *
_random_Random_getrandbits_impl(RandomObject *self, int k)
/*[clinic end generated code: output=b402f82a2158887f input=87603cd60f79f730]*/
/*[clinic end generated code: output=b402f82a2158887f input=abfd108bf3c91945]*/
{
int i, words;
uint32_t r;
Expand Down
12 changes: 2 additions & 10 deletions Modules/clinic/_randommodule.c.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.