Limited API tests are now incorrectly skipped unconditionally

[andersk]

Commit 13a0007 (#108663, cc @vstinner) made all Python builds compatible with the Limited API, and removed the LIMITED_API_AVAILABLE flag. However, some tests are still checking for that flag, so they are now being incorrectly skipped.

cpython/Modules/_testcapimodule.c

Lines 4000 to 4010 in 3bfa24e

	#ifndef LIMITED_API_AVAILABLE
	PyModule_AddObjectRef(m, "LIMITED_API_AVAILABLE", Py_False);
	#else
	PyModule_AddObjectRef(m, "LIMITED_API_AVAILABLE", Py_True);
	if (_PyTestCapi_Init_VectorcallLimited(m) < 0) {
	return NULL;
	}
	if (_PyTestCapi_Init_HeaptypeRelative(m) < 0) {
	return NULL;
	}
	#endif

cpython/Lib/test/support/__init__.py

Lines 1088 to 1094 in 3bfa24e

	def requires_limited_api(test):
	try:
	import _testcapi
	except ImportError:
	return unittest.skip('needs _testcapi module')(test)
	return unittest.skipUnless(
	_testcapi.LIMITED_API_AVAILABLE, 'needs Limited API support')(test)

Linked PRs

• gh-109045: Remove remaining LIMITED_API_AVAILABLE checks in tests #109046

[vstinner]

I'm now curious. How did you notice that some tests were always skipped?

[vstinner]

Before:

./python -m test test_capi 
(...)
Total tests: run=668 skipped=13
Total test files: run=1/1
Result: SUCCESS

After:

./python -m test test_capi 
(...)
Total tests: run=669 skipped=6
Total test files: run=1/1
Result: SUCCESS

I can see that there are less skipped tests :-) By the way, remaining skips:

$ ./python -m test test_capi -v|grep 'skipped '
test_Z (test.test_capi.test_getargs.String_TestCase.test_Z) ... skipped 'requires legacy Unicode C API'
test_Z_hash (test.test_capi.test_getargs.String_TestCase.test_Z_hash) ... skipped 'requires legacy Unicode C API'
test_u (test.test_capi.test_getargs.String_TestCase.test_u) ... skipped 'requires legacy Unicode C API'
test_u_hash (test.test_capi.test_getargs.String_TestCase.test_u_hash) ... skipped 'requires legacy Unicode C API'
test_trashcan_python_class1 (test.test_capi.test_misc.CAPITest.test_trashcan_python_class1) ... skipped "resource 'cpu' is not enabled"
test_trashcan_python_class2 (test.test_capi.test_misc.CAPITest.test_trashcan_python_class2) ... skipped "resource 'cpu' is not enabled"

[vstinner]

I created issue #109050 for skipped 'requires legacy Unicode C API' tests.

[andersk]

I'm now curious. How did you notice that some tests were always skipped?

I actually found this from the source—during an unrelated investigation I noticed that changes to heaptype_relative.c and vectorcall_limited.c had no effect even when introducing deliberate compile errors.

(The unrelated investigation is into the feasibility of const-marking fields like tp_methods, tp_members, tp_getset, m_ml, m_methods, m_slots so that we can conveniently move a bunch of structs with function pointers to read-only memory for slightly improved security hardening.)

[vstinner]

had no effect even when introducing deliberate compile errors

It's always good to test the tests by injecting errors on purpose and making sure that... they fail :-) Especially during development!