use pytest-xdist to utilize multiple workers for Python tests

[firewave]

No description provided.

[firewave]

__________________________ test_color_tty[env0-True] ___________________________
[gw3] darwin -- Python 3.12.5 /usr/local/bin/python3
tmpdir = local('/private/var/folders/3p/1bg9mx4130dgqr_85c65ptwh0000gn/T/pytest-of-runner/pytest-1/popen-gw3/test_color_tty_env0_True_0')
env = {}, color_expected = True
@pytest.mark.skipif(sys.platform == "win32", reason="TTY not supported in Windows")
@pytest.mark.parametrize("env,color_expected", [({}, True), ({"NO_COLOR": "1"}, False)])
def test_color_tty(tmpdir, env, color_expected):
test_file = os.path.join(tmpdir, 'test.c')
with open(test_file, 'wt') as f:
f.write('#error test\nx=1;\n')
exitcode, _, stderr = cppcheck([test_file], env=env, tty=True)
assert exitcode == 0
> assert stderr
E AssertionError: assert ''
other_test.py:119: AssertionError
----------------------------- Captured stdout call -----------------------------
using '/Users/runner/work/cppcheck/cppcheck/cppcheck'
=========================== short test summary info ============================
FAILED other_test.py::test_color_tty[env0-True] - AssertionError: assert ''

I cannot reproduce this on Linux. I assume this might be because the used file is not unique but I cannot be sure. Needs some looking into.

[firewave]

I cannot reproduce this on Linux. I assume this might be because the used file is not unique but I cannot be sure. Needs some looking into.

That is an obviously wrong observation since tmpdir is involved. It is more likely the using of multiple workers is interfering with the hand-rolled process spawning code on macOS.

[firewave]

__________________________ test_addon_absolute_path ___________________________
[gw0] win32 -- Python 3.13.0 C:\hostedtoolcache\windows\Python\3.13.0\x64\python.exe
def test_addon_absolute_path():
args = [
'--addon=misra',
'--enable=style',
'--template=cppcheck1',
__proj_dir
]
ret, stdout, stderr = cppcheck(args)
filename = os.path.join(__proj_dir, 'main.c')
assert ret == 0, stdout
> assert stderr == ('[%s:5]: (error) Division by zero.\n'
'[%s:1]: (style) misra violation (use --rule-texts=<file> to get proper output)\n' % (filename, filename))
E AssertionError: assert '[D:\\a\\cppcheck\\cppcheck\\test\\cli\\helloworld\\main.c:5]: (error) Division by zero.\n[D:\\a\\cppcheck\\cppcheck\\test\\cli\\helloworld\\main.c:1]: (style) misra violation (use --rule-texts=<file> to get proper output)\n[C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-1\\popen-gw1\\test_addon_misra0\\test.cpp:2]: (style) misra violation (use --rule-texts=<file> to get proper output)\n' == '[D:\\a\\cppcheck\\cppcheck\\test\\cli\\helloworld\\main.c:5]: (error) Division by zero.\n[D:\\a\\cppcheck\\cppcheck\\test\\cli\\helloworld\\main.c:1]: (style) misra violation (use --rule-texts=<file> to get proper output)\n'
E E [D:\a\cppcheck\cppcheck\test\cli\helloworld\main.c:5]: (error) Division by zero.
E [D:\a\cppcheck\cppcheck\test\cli\helloworld\main.c:1]: (style) misra violation (use --rule-texts=<file> to get proper output)
E + [C:\Users\runneradmin\AppData\Local\Temp\pytest-of-runneradmin\pytest-1\popen-gw1\test_addon_misra0\test.cpp:2]: (style) misra violation (use --rule-texts=<file> to get proper output)
test\cli\helloworld_test.py:93: AssertionError
---------------------------- Captured stdout call -----------------------------
using 'D:\a\cppcheck\cppcheck\cppcheck.exe'
______________________________ test_addon_misra _______________________________
[gw1] win32 -- Python 3.13.0 C:\hostedtoolcache\windows\Python\3.13.0\x64\python.exe
tmpdir = local('C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-1\\popen-gw1\\test_addon_misra0')
def test_addon_misra(tmpdir):
test_file = os.path.join(tmpdir, 'test.cpp')
with open(test_file, 'wt') as f:
f.write("""
typedef int MISRA_5_6_VIOLATION;
""")
args = ['--addon=misra', '--enable=all', '--disable=unusedFunction', '-j1', test_file]
exitcode, stdout, stderr = cppcheck(args)
> assert exitcode == 0
E assert 1 == 0
test\cli\other_test.py:284: AssertionError
---------------------------- Captured stdout call -----------------------------
using 'D:\a\cppcheck\cppcheck\cppcheck.exe'
=========================== short test summary info ===========================
FAILED test/cli/helloworld_test.py::test_addon_absolute_path - AssertionError: assert '[D:\\a\\cppcheck\\cppcheck\\test\\cli\\helloworld\\main.c:5]: (error) Division by zero.\n[D:\\a\\cppcheck\\cppcheck\\test\\cli\\helloworld\\main.c:1]: (style) misra violation (use --rule-texts=<file> to get proper output)\n[C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-1\\popen-gw1\\test_addon_misra0\\test.cpp:2]: (style) misra violation (use --rule-texts=<file> to get proper output)\n' == '[D:\\a\\cppcheck\\cppcheck\\test\\cli\\helloworld\\main.c:5]: (error) Division by zero.\n[D:\\a\\cppcheck\\cppcheck\\test\\cli\\helloworld\\main.c:1]: (style) misra violation (use --rule-texts=<file> to get proper output)\n'
[D:\a\cppcheck\cppcheck\test\cli\helloworld\main.c:5]: (error) Division by zero.
[D:\a\cppcheck\cppcheck\test\cli\helloworld\main.c:1]: (style) misra violation (use --rule-texts=<file> to get proper output)
+ [C:\Users\runneradmin\AppData\Local\Temp\pytest-of-runneradmin\pytest-1\popen-gw1\test_addon_misra0\test.cpp:2]: (style) misra violation (use --rule-texts=<file> to get proper output)
FAILED test/cli/other_test.py::test_addon_misra - assert 1 == 0

[firewave]

_______________________________ test_addon_misra _______________________________
[gw1] linux -- Python 3.8.10 /usr/bin/python3
tmpdir = local('/tmp/pytest-of-runner/pytest-0/popen-gw1/test_addon_misra0')
def test_addon_misra(tmpdir):
test_file = os.path.join(tmpdir, 'test.cpp')
with open(test_file, 'wt') as f:
f.write("""
typedef int MISRA_5_6_VIOLATION;
""")
args = ['--addon=misra', '--enable=all', '--disable=unusedFunction', '-j1', test_file]
exitcode, stdout, stderr = cppcheck(args)
> assert exitcode == 0
E assert -6 == 0
test/cli/other_test.py:284: AssertionError
----------------------------- Captured stdout call -----------------------------
using '/home/runner/work/cppcheck/cppcheck/cppcheck'
=========================== short test summary info ============================
FAILED test/cli/other_test.py::test_addon_misra - assert -6 == 0

[firewave]

The addons tests fail because of data being created in the source tree - see https://trac.cppcheck.net/ticket/13366.

[firewave]

FYI this saves about half the run-time (40s vs 77s) according to a CI run with otherwise comparable times.

This was missed in a few steps which were added after I opened this PR. I added those in #7106.