`memcpy()` usage without `#include <string.h>` in `pwdmodule.c`

[fuhsnn]

Bug report

Bug description:

For a non-gcc/clang compiler, _Py_TYPEOF is not defined.

cpython/Include/pyport.h

Lines 548 to 551 in c08a302

	// The macro is only defined if GCC or clang compiler is used.
	#if defined(__GNUC__) || defined(__clang__)
	# define _Py_TYPEOF(expr) __typeof__(expr)
	#endif

Which leads to Py_CLEAR() fallback to an implementation with memcpy() call.

cpython/Include/object.h

Lines 1016 to 1027 in c08a302

	#else
	#define Py_CLEAR(op) \
	do { \
	PyObject **_tmp_op_ptr = _Py_CAST(PyObject**, &(op)); \
	PyObject *_tmp_old_op = (*_tmp_op_ptr); \
	if (_tmp_old_op != NULL) { \
	PyObject *_null_ptr = _Py_NULL; \
	memcpy(_tmp_op_ptr, &_null_ptr, sizeof(PyObject*)); \
	Py_DECREF(_tmp_old_op); \
	} \
	} while (0)
	#endif

Py_CLEAR() is used in ./Modules/pwdmodule.c

cpython/Modules/pwdmodule.c

Lines 355 to 358 in c08a302

	static int pwdmodule_clear(PyObject *m) {
	Py_CLEAR(get_pwd_state(m)->StructPwdType);
	return 0;
	}

without including string.h

cpython/Modules/pwdmodule.c

Lines 12 to 16 in c08a302

	#include <errno.h> // ERANGE
	#include <pwd.h> // getpwuid()
	#include <unistd.h> // sysconf()

For a non-gcc/clang compiler, this may fail due to missing memcpy() declaration.

./Include/object.h:1017: #define Py_CLEAR(op) do { PyObject **_tmp_op_ptr = _Py_CAST(PyObject**, &(op)); PyObject *_tmp_old_op = (*_tmp_op_ptr); if (_tmp_old_op != NULL) { PyObject *_null_ptr = _Py_NULL; memcpy(_tmp_op_ptr, &_null_ptr, sizeof(PyObject*)); Py_DECREF(_tmp_old_op); } } while (0)
 ^ implicit declaration of a function
./Modules/pwdmodule.c:356: Py_CLEAR(get_pwd_state(m)->StructPwdType);
 ^ in expansion of macro

On a side note, typeof is widely implemented among alternative C compilers like TinyCC and cproc, kefir, chibicc, and is standardized in C23. It could be beneficial to enable typeof usage through configure option or probing, instead of hard-coded off on non-gcc/clang compilers.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-121617: Include string.h in pwdmodule.c if _Py_TYPEOF is undefined #121643
• gh-121617: Include <string.h> for Py_CLEAR() macro #144666

[Zheaoli]

For now, GCC and Clang are the officially supported toolchains for Python. I'm not sure we need support extra compiler set.

But for now, this is expected behavior

[fuhsnn]

For now, GCC and Clang are the officially supported toolchains

From what I'm seeing, there are new code written as recent as 3.13 that explicitly provide support for 1. standard C11 compiler and 2. MSVC.

cpython/Include/cpython/pyatomic.h

Lines 488 to 498 in 2bd960b

	#if _Py_USE_GCC_BUILTIN_ATOMICS
	# define Py_ATOMIC_GCC_H
	# include "cpython/pyatomic_gcc.h"
	# undef Py_ATOMIC_GCC_H
	#elif __STDC_VERSION__ >= 201112L && !defined(__STDC_NO_ATOMICS__)
	# define Py_ATOMIC_STD_H
	# include "cpython/pyatomic_std.h"
	# undef Py_ATOMIC_STD_H
	#elif defined(_MSC_VER)
	# define Py_ATOMIC_MSC_H

The define logic of _Py_TYPEOF() clearly has "implementations other than GCC and Clang" in mind,

cpython/Include/pyport.h

Lines 548 to 551 in c08a302

	// The macro is only defined if GCC or clang compiler is used.
	#if defined(__GNUC__) || defined(__clang__)
	# define _Py_TYPEOF(expr) __typeof__(expr)
	#endif

even if the guarding condition only exists for MSVC, it would simply be !defined(_MSC_VER).

If fact, were it be !defined(_MSC_VER) this issue would not exist, since the tested compiler actually support __typeof__.

[encukou]

@vstinner, AFAIK you both rewrote Py_CLEAR and removed string.h. Are you interested in this issue?

I don't think fixing only pwdmodule.h is correct; Py_CLEAR should work for all users of Python.h.

[vstinner]

I don't think fixing only pwdmodule.h is correct; Py_CLEAR should work for all users of Python.h.

I agree. I wrote #144666 to include <string.h> header if the _Py_TYPEOF macro is not defined.

On a side note, typeof is widely implemented among alternative C compilers like TinyCC and cproc, kefir, chibicc, and is standardized in C23. It could be beneficial to enable typeof usage through configure option or probing, instead of hard-coded off on non-gcc/clang compilers.

My PR also modify _Py_TYPEOF() macro to use C23 typeof() if available.

I wrote a configure check for typeof() but it didn't work as expected. It says that typeof() is available, but then building Python fails because typeof() is undefined. The problem is that configure adds -std=c11 to CFLAGS_NODIST, and typeof() is only defined in C23 or newer. I abandoned the configure change, it's too tricky for me to get it right (run the checks with the same C flags than the ones used to build Python).

[vstinner]

On a side note, typeof is widely implemented among alternative C compilers like TinyCC and cproc, kefir, chibicc, and is standardized in C23. It could be beneficial to enable typeof usage through configure option or probing, instead of hard-coded off on non-gcc/clang compilers.

For fun, I tried building Python 3.15 (main branch) with tcc (TinyCC), but it failed to compile _Py_thread_local.

The following program fails to build with: error: _Thread_local is not implemented (using tcc -std=c11).

int main()
{
 _Thread_local int myvar;
 return 0;
}

[vstinner]

I fixed the Py_CLEAR() issue in the main branch: Python.h now always includes <string.h>.

I don't think that it's worth it to backport the change to 3.13 and 3.14 branches. I close the issue.

Thanks for the bug report @fuhsnn.