gh-150114: Log the memory usage in regrtest on FreeBSD

[vstinner]

Add _testcapi.get_process_memory_usage().
On FreeBSD, _testcapi is now linked to libkvm.

• Issue: GitHub free threading CI failing with SIGTERM #150114

[vstinner]

!buildbot FreeBSD

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 2e19d41 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F150280%2Fmerge

The command will test the builders whose names match following regular expression: FreeBSD

The builders matched are:

• AMD64 FreeBSD16 PR
• AMD64 FreeBSD Refleaks PR
• AMD64 FreeBSD15 PR
• AMD64 FreeBSD14 PR

[vstinner]

buildbot/AMD64 FreeBSD Refleaks PR
buildbot/AMD64 FreeBSD14 PR
buildbot/AMD64 FreeBSD15 PR

The 3 FreeBSD buildbot succeeded.

But test_regrtest failed once on AMD64 FreeBSD14 PR (and passed when re-run in verbose mode):

FAIL: test_module_from_test_autotest (test.test_regrtest.ProgramsTestCase.test_module_from_test_autotest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/test_regrtest.py", line 952, in test_module_from_test_autotest
self.run_tests(args)
~~~~~~~~~~~~~~^^^^^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/test_regrtest.py", line 919, in run_tests
output = self.run_python(args, env=env, isolated=isolated)
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/test_regrtest.py", line 861, in run_python
proc = self.run_command(cmd, **kw)
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/test_regrtest.py", line 849, in run_command
self.fail(msg)
~~~~~~~~~^^^^^
AssertionError: Command ['/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/python', '-X', 'faulthandler', '-I', '-Wd', '-E', '-bb', '-c', 'from test import autotest', '-uall', '-rwW', '--testdir=/tmp/test_python_9cnxbofn/tmpugkn3ex0', '--timeout', '3600', '-j4', 'test_regrtest_noop58', 'test_regrtest_noop59', 'test_regrtest_noop60', 'test_regrtest_noop61'] failed with exit code 1, but exit code 0 expected!
stdout:
---
Using random seed: 3935093812
0:00:00 load avg: 3.86 mem: 28.0 MiB Run 4 tests in parallel using 4 worker processes (timeout: 1 hour, worker timeout: 1 hour 5 min)
Kill <WorkerThread #1 running test=test_regrtest_noop58 pid=76931 time=381 ms> process group
Kill <WorkerThread #2 running test=test_regrtest_noop60 pid=76932 time=381 ms> process group
Traceback (most recent call last):
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/run_workers.py", line 633, in run
result = self._process_result(item)
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/run_workers.py", line 593, in _process_result
self.display_result(mp_result)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/run_workers.py", line 577, in display_result
self.display_progress(self.test_index, text)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/logger.py", line 76, in display_progress
self.log(f"[{line}] {text}")
~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/logger.py", line 30, in log
mem = self.get_mem_usage()
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/run_workers.py", line 614, in get_mem_usage
worker_mem = worker.get_mem_usage()
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/run_workers.py", line 459, in get_mem_usage
return get_process_memory_usage(popen.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
from test import autotest
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/autotest.py", line 5, in <module>
main()
~~~~^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/main.py", line 801, in main
Regrtest(ns, _add_python_opts=_add_python_opts).main(tests=tests)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/main.py", line 793, in main
exitcode = self.run_tests(selected, tests)
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/main.py", line 600, in run_tests
return self._run_tests(selected, tests)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/main.py", line 563, in _run_tests
self._run_tests_mp(runtests, self.num_workers)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/main.py", line 457, in _run_tests_mp
RunWorkers(num_workers, runtests, self.logger, self.results).run()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/run_workers.py", line 646, in run
self.stop_workers()
~~~~~~~~~~~~~~~~~^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/run_workers.py", line 531, in stop_workers
worker.stop()
~~~~~~~~~~~^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/run_workers.py", line 176, in stop
self._kill()
~~~~~~~~~~^^
File "/home/buildbot/buildarea/pull_request.opsec-fbsd14/build/Lib/test/libregrtest/run_workers.py", line 151, in _kill
sid = os.getsid(popen.pid)
ProcessLookupError: [Errno 3] No such process