Skip to content

testrunner: added options -n (no summary) and -d (dry run) / added script to run test cases separately / adjusted some tests - #6049

Merged
firewave merged 9 commits into
cppcheck-opensource:mainfrom
firewave:testrunner-single-2
Mar 6, 2024
Merged

testrunner: added options -n (no summary) and -d (dry run) / added script to run test cases separately / adjusted some tests#6049
firewave merged 9 commits into
cppcheck-opensource:mainfrom
firewave:testrunner-single-2

Conversation

@firewave

Copy link
Copy Markdown
Collaborator

This script runs each test case separately so we can uncover cases where it depends on left-over data on the previous one. CTest is only running the fixtures separately so that does not.

@firewave

Copy link
Copy Markdown
CollaboratorAuthor

My first attempt was to use a grep script to get all the names but because some test files contain multiple fixtures that does not work easily. So I went with any easier way and just get the file of all the tests from the application itself.

We currently cannot add this into the CI since the start-up time of testrunner is too high since it always instantiates all the test fixtures. Even with a release build the run-time is substantially higher. This issue is being tracked in https://trac.cppcheck.net/ticket/12080.

We can probably also integrate this into CTest but I did not look into it (will file a ticket about it though). But that also requires the run-time issue being resolved first.

@firewavefirewave changed the title testrunner: add options -n (no dummary) and -d (dry run) / added script to run test case separatelytestrunner: added options -n (no summary) and -d (dry run) / added script to run test case separatelyFeb 27, 2024
@firewave

Copy link
Copy Markdown
CollaboratorAuthor

"no dummary" 😵‍💫

@firewavefirewave changed the title testrunner: added options -n (no summary) and -d (dry run) / added script to run test case separatelytestrunner: added options -n (no summary) and -d (dry run) / added script to run test case separately / adjusted some testsFeb 28, 2024
@firewave
firewave marked this pull request as ready for review February 28, 2024 00:30
@firewave
firewave marked this pull request as draft February 28, 2024 00:31
@firewave

Copy link
Copy Markdown
CollaboratorAuthor

Done - but script still lacks the executable flag.

@firewave
firewave marked this pull request as ready for review February 28, 2024 12:13
@chrchr-github

Copy link
Copy Markdown
Collaborator

"no dummary" 😵‍💫

image

@firewavefirewave changed the title testrunner: added options -n (no summary) and -d (dry run) / added script to run test case separately / adjusted some teststestrunner: added options -n (no summary) and -d (dry run) / added script to run test cases separately / adjusted some testsMar 4, 2024
@firewave
firewaveforce-pushed the testrunner-single-2 branch from fd7a753 to 4cc8d84CompareMarch 4, 2024 15:15
@firewave
firewaveforce-pushed the testrunner-single-2 branch from 4cc8d84 to d3d562eCompareMarch 5, 2024 15:28
@firewave

Copy link
Copy Markdown
CollaboratorAuthor

Beside the dummary almost a whole week without any feedback - merging.

@firewave
firewave merged commit 54c01cf into cppcheck-opensource:mainMar 6, 2024
@firewave
firewave deleted the testrunner-single-2 branch March 6, 2024 13:14
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@firewave@chrchr-github