Uh oh!
There was an error while loading. Please reload this page.
Run the API tests in the TPM CI job - #1165
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the TPM CI workflow to actually compile and run wolfSSH’s API test suite (tests/api.test) under a --enable-tpm build, ensuring TPM-guarded tests can gate merges.
Changes:
- Add a gated
make check TESTS=tests/api.teststep to run API tests in a single selected matrix cell. - Add a failure-only step to print
tests/api.log. - Archive
wolfssh/tests/api.logas a workflow artifact.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Uh oh!
There was an error while loading. Please reload this page.
wolfSSL-Fenrir-bot
left a comment
There was a problem hiding this comment.
Fenrir Automated Review — PR #1165
No scan targets match the changed files in this PR. Review skipped.
ejohnstown
left a comment
There was a problem hiding this comment.
Also, going forward, please be more descriptive in your commit messages. The commit messages are the real history that moves around. This commit is doing something non-obvious enough (why TESTS= is overridden, why one matrix cell, why the grep guard exists) that the reasoning belongs in the commit body, not only on the PR page.
Uh oh!
There was an error while loading. Please reload this page.
- No job compiled a wolfSSH test binary with WOLFSSH_TPM defined, so a test guarded on it compiled out everywhere and could not gate a merge. This job enables TPM but only builds; the jobs that run make check do not enable it. - Add a make check step. automake's check-am builds every check_PROGRAM regardless of the TESTS override, so this is the only job that compiles wolfSSH's tests with TPM support. - Override TESTS to run only tests/api.test, the one suite with TPM-specific tests. kex.test also aborts in the example client, which demands -K in a TPM build. - Restrict it to one matrix cell. The 2x2x2 matrix varies the simulator and the host key, neither of which these tests touch. - Assert -DWOLFSSH_TPM in AM_CPPFLAGS first. A build without it compiles the guarded tests out and still exits 0, a hollow pass. - Dump tests/api.log on failure and archive it.
Uh oh!
There was an error while loading. Please reload this page.
Problem
No CI job compiles a wolfSSH test binary with
WOLFSSH_TPMdefined, so any test guarded on it silently compiles out everywhere and cannot gate a merge.--enable-tpmmake checktpm-ssh.ymlmake+make installsanitizer.yml--enable-allexcludes TPM)The intersection is empty. Found while writing #1164, whose regression test is unreachable by CI for exactly this reason.
Fix (
.github/workflows/tpm-ssh.yml)Adds a
make check TESTS=tests/api.teststep afterBuild wolfSSH:check_PROGRAMs still compile —check-ambuilds them regardless of theTESTSoverride. That is the point: this is the only job compiling wolfSSH's tests with TPM support.api.testruns. It holds the only TPM-specific tests in the tree:grep -i tpmfinds 28 hits intests/api.cand zero in the other five test sources.ecc/ibmswtpm2/raw). The 2×2×2 matrix varies the simulator and the host key, neither of which these tests touch.-DWOLFSSH_TPMcompiles the guarded tests out and still exits 0, a hollow pass. The step grepsAM_CPPFLAGSfor the define, prints what it found, and aborts if absent.tests/api.logis dumped on failure and added to the archived artifacts.Verification
Against a scratch prefix reproducing CI's library layout (wolfSSL with this job's exact flags, wolfTPM rebuilt against it):
PASS: tests/api.test, and all five test binaries compile under--enable-tpm --enable-certs.--enable-tpm, the guard fails and the step aborts.unit.testandregress.testgive byte-identical output with and without the define (diffed);testsuite.testexits 0 having executed nothing, its body being entirely#ifdef WOLFSSH_SHELL/WOLFSSH_SFTP;kex.testfails;auth.testis not built.Not in this PR
kex.testfails because the example client aborts with "You must specify a password for the TPM key". Two separate defects sit behind that:-Kguard is too broad. The client aborts whenevertpmKeyAuth == NULLin a TPM build, butkex.testpasses no-i, soClientSetPrivateKey()takes the built-in key-buffer path wheretpmKeyAuthis unused. Fixing only this would letkex.testrun on a software key, adding no TPM coverage.#elifinClientSetPrivateKey()routes every-ithroughwolfSSH_TPM_InitKey(), leaving no path towolfSSH_ReadKey_file().Both point at the same gap: nothing anywhere exercises a client user-auth key held in the TPM. That deserves a purpose-built test rather than a retrofit of the KEX sweep, so it is left for a follow-up.