gh-110012: Fix RuntimeWarning in test_socket

[sobolevn]

After:

Ran 483 tests in 27.474s
OK (skipped=245)
== Tests result: SUCCESS ==
1 test OK.
Total duration: 27.6 sec
Total tests: run=483 skipped=245
Total test files: run=1/1
Result: SUCCESS

I haven't checked if it also generates this warning on other platforms (different from macos sonoma), but in case it is not, I will just change quiet to True

• Issue: test_socket generates several warnings #110012

[AlexWaygood]

You mean "malformed" and "quiet", not "mailformed" and "quite" :-)

[sobolevn]

@AlexWaygood I am really into tpyos, thanks! :)

[AlexWaygood]

Yuo're wellcom!

[vstinner]

Does it catch them or ignore them? What is the behavior of quiet=False?

[sobolevn]

Yes

cpython/Lib/test/support/warnings_helper.py

Lines 152 to 188 in 8f324b7

	def_filterwarnings(filters, quiet=False):
	"""Catch the warnings, then check if all the expected
	warnings have been raised and re-raise unexpected warnings.
	If 'quiet' is True, only re-raise the unexpected warnings.
	"""
	# Clear the warning registry of the calling module
	# in order to re-raise the warnings.
	frame=sys._getframe(2)
	registry=frame.f_globals.get('__warningregistry__')
	ifregistry:
	registry.clear()
	withwarnings.catch_warnings(record=True) asw:
	# Set filter "always" to record all warnings. Because
	# test_warnings swap the module, we need to look up in
	# the sys.modules dictionary.
	sys.modules['warnings'].simplefilter("always")
	yieldWarningsRecorder(w)
	# Filter the recorded warnings
	reraise=list(w)
	missing= []
	formsg, catinfilters:
	seen=False
	forwinreraise[:]:
	warning=w.message
	# Filter out the matching messages
	if (re.match(msg, str(warning), re.I) and
	issubclass(warning.__class__, cat)):
	seen=True
	reraise.remove(w)
	ifnotseenandnotquiet:
	# This filter caught nothing
	missing.append((msg, cat.__name__))
	ifreraise:
	raiseAssertionError("unhandled warning %s"%reraise[0])
	ifmissing:
	raiseAssertionError("filter (%r, %s) did not catch any warning"%
	missing[0])

1. It catches warnings with warnings.catch_warnings(record=True)
2. Then it filters out ones that matched ("received malformed or improperly-truncated ancillary data", RuntimeWarning)
3. If there are any left, the test fails
4. quiet=True handles the cases where no warning was raised - just by ignoring the last check, if quiet=False we are required to have at least one matched warning

[serhiy-storchaka]

Is there a correlation between a warning and the emptiness of ancdata?

[vstinner]

"received malformed or improperly-truncated ancillary data" message comes from Python if get_cmsg_data_len() returns non-zero. Can you please check in which case your are?

staticintget_cmsg_data_len(structmsghdr*msg, structcmsghdr*cmsgh, size_t*data_len)
{
size_tspace, cmsg_data_len;
if (!cmsg_min_space(msg, cmsgh, CMSG_LEN(0)) ||cmsgh->cmsg_len<CMSG_LEN(0))
return-1; cmsg_data_len=cmsgh->cmsg_len-CMSG_LEN(0);
if (!get_cmsg_data_space(msg, cmsgh, &space))
return-1;
if (space >= cmsg_data_len) {
*data_len=cmsg_data_len;
return0;
}
*data_len=space;
return1;
} 

[sobolevn]

Local debug shows that I face

printf("3");
*data_len=space;
return1;

[sobolevn]

Hm, I agree that this does not look like a proper fix. There's something else going on inside, for some reason there's not enough space to write the data.

[serhiy-storchaka]

The code was added in 2011 (bpo-6560, #50809, 96fe56a). It does not emit a warning on Linux (at least I never see it). Were the warnings always on Windows, or did they appear in some version? Are they always expected in these tests, do they signal a problem? Shouldn't these tests fail if there was not a warning on Windows?

[sobolevn]

We should ask @ncoghlan for help, I guess :)

[serhiy-storchaka]

@sobolevn Could you test whether warnings were present in old Python versions (it was 3.2 or 3.3 I guess). Since most core developers do not use Windows, it might just slip past their notice.

[sobolevn]

I cannot build anything below 3.7 on mac sonora (m2) :(

3.7:

~/Desktop/cpython2 0f56adb8d7 ✗ » ./python.exe -m test test_socket 0:00:00 load avg: 5.09 Run tests sequentially
0:00:00 load avg: 5.09 [1/1] test_socket
/Users/sobolev/Desktop/cpython2/Lib/test/test_socket.py:2323: RuntimeWarning: received malformed or improperly-truncated ancillary data
result = sock.recvmsg(bufsize, *args)
/Users/sobolev/Desktop/cpython2/Lib/test/test_socket.py:2414: RuntimeWarning: received malformed or improperly-truncated ancillary data
result = sock.recvmsg_into([buf], *args)
== Tests result: SUCCESS ==
1 test OK.
Total duration: 24.0 sec
Tests result: SUCCESS

3.8:

~/Desktop/cpython2 3.8 ✔ » ./python.exe -m test test_socket 0:00:00 load avg: 6.11 Run tests sequentially
0:00:00 load avg: 6.11 [1/1] test_socket
/Users/sobolev/Desktop/cpython2/Lib/test/test_socket.py:2438: RuntimeWarning: received malformed or improperly-truncated ancillary data
result = sock.recvmsg(bufsize, *args)
/Users/sobolev/Desktop/cpython2/Lib/test/test_socket.py:2529: RuntimeWarning: received malformed or improperly-truncated ancillary data
result = sock.recvmsg_into([buf], *args)
== Tests result: SUCCESS ==
1 test OK.
Total duration: 23.2 sec
Tests result: SUCCESS

So, this is not new at least.

[vstinner]

@serhiy-storchaka: @sobolevn is trying to make warnings quiet on macOS, not on Windows. I don't know if these warnings are emitted on other platforms than macOS.

[AlexWaygood]

python -We -m test test_socket passes for me on Windows on a reasonably fresh build of Python 3.13, FWIW

[sobolevn]

@vstinner they appear both on Win and Mac, see my first commit:

• ubuntu (failing): https://github.com/python/cpython/actions/runs/6335548712/job/17206922402
• windows passing: https://github.com/python/cpython/actions/runs/6335548712/job/17206922528

[vstinner]

@vstinner they appear both on Win and Mac, see my first commit

Oh ok.

[hugovk]

@AlexWaygood@serhiy-storchaka@vstinner Any further comments on this PR?

[serhiy-storchaka]

I do not think that simply suppressing warnings here is the right solution. What is the warning about? What is the end user supposed to do with it? Non-zero result of get_cmsg_data_len() should be either an error or ignored (note that there are two non-zero returned values). If such situation is common, then perhaps the code just does not handle all cases correctly.

[graingert]

What we can do is set the warning filter to default for these warnings, add a TODO comment and not close this issue

[AlexWaygood]

@AlexWaygood@serhiy-storchaka@vstinner Any further comments on this PR?

I only ever suggested some typo fixes on this one; it's well out of my area of expertise :-)

[graingert]

What we can do is set the warning filter to default for these warnings, add a TODO comment and not close this issue

I've now done this in the warnings as errors PR

[github-actions]

This PR is stale because it has been open for 30 days with no activity.