Improve test runner script with refactoring and prerequisite checks - #509

Merged

firewave merged 7 commits into

cppcheck-opensource:masterfrom

jcfr:improve-test-runner

Aug 30, 2025

jcfr commented Aug 26, 2025•
edited
Loading

Contributor

This pull request enhances the test runner script with two main improvements:

Refactors the script to introduce a run() function that centralizes the logic for executing compiler commands, capturing their output, and returning results.
Adds checks to ensure that required compilers (clang, gcc) are present before running tests.

jcfr force-pushed the improve-test-runner branch from 86f3774 to 3505c8fCompare

August 26, 2025 04:59

firewave reviewed

View reviewed changes

run-tests.py Outdated

firewave reviewed

View reviewed changes

run-tests.py Outdated

firewave reviewed

View reviewed changes

run-tests.py Outdated

firewave reviewed

View reviewed changes

run-tests.py Outdated

firewave reviewed

View reviewed changes

run-tests.py Outdated

firewave reviewed

View reviewed changes

run-tests.py Outdated

firewave reviewed

View reviewed changes

run-tests.py Outdated

firewave reviewed

View reviewed changes

run-tests.py Outdated

jcfr force-pushed the improve-test-runner branch 3 times, most recently from dc55734 to 129b151Compare

August 26, 2025 08:58

firewave reviewed

View reviewed changes

run-tests.py Outdated

firewave reviewed

View reviewed changes

run-tests.py Outdated

jcfr force-pushed the improve-test-runner branch 2 times, most recently from c6a0970 to 51a3803Compare

August 26, 2025 16:30

jcfr commented Aug 26, 2025

ContributorAuthor

Thanks for the taking the time to review 🙏

All the review comments have been addressed. Once the GitHub workflow have been enabled and the test pass, I believe this will be ready for integration 🚀

cc: @danmar @firewave

firewave reviewed

View reviewed changes

jcfr force-pushed the improve-test-runner branch from 51a3803 to d88f23fCompare

August 27, 2025 13:33

jcfr mentioned this pull request

refactor: modernize run-tests helpers with typing and safer subprocess handling #512

Closed

jcfr commented Aug 27, 2025

ContributorAuthor

For additional context, the initial goal of this pull request was to allow me to understand the missing prerequisites.

The proposed improvements fulfill this. This is now ready for integration 🙏

jcfr force-pushed the improve-test-runner branch from 2f1962a to 8203180Compare

August 28, 2025 18:10

hjmjohnson commented Aug 28, 2025

@jcfr Thank you for making this MUCH easier to understand.

firewave commented Aug 28, 2025

Collaborator

For additional context, the initial goal of this pull request was to allow me to understand the missing prerequisites.

Yeah - it is (was) a bit of a mess. I just started digging into it in the past few days myself.

danmar reviewed

View reviewed changes

run-tests.py Outdated

jcfr force-pushed the improve-test-runner branch from 8203180 to 2bc5572Compare

August 29, 2025 14:41

jcfr added 4 commits

August 29, 2025 12:00

chore: Simplify test runner by introducing "run()" function

9f8749b

This change refactors the test runner script by adding a `run()` function
for executing compiler commands, capturing their output, and returning the
results.

tests: Update test runner to fail early if prerequisites are missing

f2d0749

This change updates the test runner script to check for the presence of
required compilers (clang, gcc) before running tests.

chore: Update test runner script to explicitly set output of run()

5fecffc

Suggested-by: Oliver Stöneberg <firewave@users.noreply.github.com>

refactor: modernize run-tests helpers with typing and safer subproces…

645062b

…s handling
Suggested-by: Oliver Stöneberg <firewave@users.noreply.github.com>

jcfr added 3 commits

August 29, 2025 12:00

chore: simplify run-tests by using text mode for subprocess

8d9228d

chore: Remove misleading exit codes from run-tests.py

2a310b3

Suggested-by: Oliver Stöneberg <firewave@users.noreply.github.com>

chore: Drop unnecessary __future__.annotations import in test runner

89e6b31

`from __future__ import annotations` is useful in importable packages to
enable postponed evaluation of annotations. However, in a standalone
script like `run-tests.py`, it adds no value.

jcfr force-pushed the improve-test-runner branch from 2bc5572 to 89e6b31Compare

August 29, 2025 16:00

firewave approved these changes

View reviewed changes

firewave left a comment

Collaborator

LGTM. The stuff I wanted to check will need to be redone anyways, so no need to further look into that.

firewave merged commit 37fa4f4 into cppcheck-opensource:master

14 checks passed

jcfr deleted the improve-test-runner branch

August 30, 2025 21:54

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment