Skip to content

audit F4/F8/F9: boot_check asserts a real boot; cipher-suite doc fix; runnable no-REU KAT - #80

Merged
JC-000 merged 4 commits into
docs/benchmark-refreshfrom
fix/audit-f4-f8-f9-harness-docs
Aug 13, 2026
Merged

audit F4/F8/F9: boot_check asserts a real boot; cipher-suite doc fix; runnable no-REU KAT#80
JC-000 merged 4 commits into
docs/benchmark-refreshfrom
fix/audit-f4-f8-f9-harness-docs

Conversation

@JC-000

@JC-000JC-000 commented Aug 13, 2026

Copy link
Copy Markdown
Owner

Audit remediation, lane E: three findings — one harness oracle (F4) and two
claims that could not be checked from the repo (F8 doc error, F9 untestable
guarantee). Based on docs/benchmark-refresh, one commit per finding.


F4 — tools/uci/boot_check.py passed a failed boot

Defect. The pass criterion was "screen RAM contains printable text and at
least 3 distinct byte values", which distinguishes a booted machine from a
blank screen and nothing else.

Fix. The criterion is now backend-aware — the banner differs per backend
(net_banner_str: UCI NETWORKING under uci, RR-NET (CS8900A) ETHERNET
under ip65) — and asserts what a successful boot must produce: the PRG image on
disk carries the expected backend's banner (checked before the device is
touched), the screen shows the common banner and the expected backend line, the
other backend's line is absent, no FAILED appears anywhere, and the main menu
was reached. Boot now polls for the menu (default 60 s, BOOT_TIMEOUT) instead
of sleep(3) — shorter than boot's own reu_mul_init pass (15-18 s on the
U64E), so the old script routinely graded a mid-boot screen. enable_uci is
issued before reset when the expected backend is uci (as phase2_check.py
does); BACKEND selects the expectation and C64_PRG overrides the image.

Acceptance A/B — on hardware (U64E 10.43.23.81, DeviceLock + enable_uci):

Correct UCI build (make BACKEND=uci USE_NISTCURVES_ONCHIP=1, 62,977 B) ⇒ PASS:

--- PRG image checks ---
[PASS] image carries uci banner: expected bytes 'UCI NETWORKING' in the PRG
[PASS] image free of foreign banner: none present
...
00: c64-https client v0.1
02: tls 1.3 / chacha20-poly1305
03: uci networking
05: initializing network...
06: network ok
07: requesting dhcp...
08: dhcp ok - ip: 10.43.23.81
09: i=init h=http g=https q=quit
--- boot checks ---
[PASS] common banner: expected 'C64-HTTPS CLIENT V0.1'
[PASS] uci backend banner: expected 'UCI NETWORKING'
[PASS] no foreign backend banner: none present
[PASS] no FAILED on screen: no failure message
[PASS] main menu reached: expected 'Q=QUIT'
PASS: uci PRG booted cleanly to the menu
rc=0

ip65/RR-Net PRG on the same U64E (C64_PRG=…/ip65.prg BACKEND=uci) ⇒ FAIL:

--- PRG image checks ---
[FAIL] image carries uci banner: expected bytes 'UCI NETWORKING' in the PRG
[FAIL] image free of foreign banner: image looks like a ['ip65'] build
...
03: rr-net (cs8900a) ethernet
05: initializing network...
06: network init failed
07: i=init h=http g=https q=quit
--- boot checks ---
[PASS] common banner: expected 'C64-HTTPS CLIENT V0.1'
[FAIL] uci backend banner: expected 'UCI NETWORKING'
[FAIL] no foreign backend banner: found ['RR-NET (CS8900A) ETHERNET'] — wrong-backend PRG?
[FAIL] no FAILED on screen: screen reports a failure
[PASS] main menu reached: expected 'Q=QUIT'
FAIL: boot check failed for expected backend uci
rc=1

The old criterion replayed against that exact ip65 screen still returns
PASS (unique=35 any_text=True), which is the finding.

Blast radius. Note the ip65 image does reach the menu — do_net_init's
failure is non-fatal in boot.s — so "menu reached" alone would not have
caught it; the backend-banner identity and the FAILED scan are what do the
work. evaluate_screen / check_prg_image are pure functions so the criterion
can be replayed against a captured dump without hardware.


F8 — CLAUDE.md named the wrong cipher suite

Defect. CLAUDE.md:429 said the client "negotiates only
TLS_AES_128_GCM_SHA256". It offers exactly one suite, 0x1303
TLS_CHACHA20_POLY1305_SHA256 (src/tls_handshake.s:85, echo-verified against
ServerHello at :380), and there is no AES anywhere in src/crypto/. The
packaging validation record in the same file already said ChaCha20-Poly1305, so
the document contradicted itself.

Fix. The claim is load-bearing — it is the stated reason the P-384 path
keeps a SHA-256 transcript hash. That conclusion is unchanged and still correct
(RFC 8446 §4.4.1 binds the transcript hash to the negotiated suite's hash, and
0x1303's hash is also SHA-256); only the reason is corrected, with the source
reference attached. Repo-wide sweep fixed the same error in
src/crypto/ecdsa_verify_384.s (3 comment sites) and
tools/package/listener/{listener.py,README.md}.

tools/test_tls_handshake.py:674 keeps its mention and is correct there:
0x1301 genuinely is TLS_AES_128_GCM_SHA256, used as a suite the client must
reject — which is itself corroboration that the client does not speak it.

Acceptance.grep -rni "aes_128_gcm|aes-128|aes128" over the repo (minus
libs/, ip65/, build/) now returns exactly that one correct line. Build
re-run after the comment edits: make BACKEND=uci rc=0, ecdsa_verify_384.o
still assembles.


F9 — the "no REU required" shipped-artifact claim had no runnable test

Defect. CLAUDE.md's packaging record states the onchip PRG "passes the
3-vector ECDSA KAT in VICE without REU", but every VICE test routes through
tools/_vice_helpers.py::default_vice_config(), which hard-codes
-reu -reusize 512. The claim was true but only reproducible by
monkeypatching the helper.

Fix.C64_VICE_NO_REU=1 makes default_vice_config() drop the REU flags
and announce it on stderr. -reu stays the default, deliberately: a no-REU run
of a REU-profile build does not error, it silently returns wrong answers. The
invocation is documented where the claim is made (packaging validation record)
and cross-referenced from the "VICE harness gotcha" note.
tools/test_ecdsa_kat_oracle.py is untouched (lane C owns it for F7), so the
opt-out is available to every VICE test rather than one script.

Acceptance (onchip build make BACKEND=uci USE_NISTCURVES_ONCHIP=1,
sha256 c9e1740e…, no patching):

$ C64_SKIP_BUILD=1 C64_VICE_NO_REU=1 python3 tools/test_ecdsa_kat_oracle.py
[C64_VICE_NO_REU] VICE launching WITHOUT -reu — valid only for USE_NISTCURVES_ONCHIP
builds; any REU-profile image will silently compute wrong results.
[3e] PASS: ecdsa_verify returned C=0 (valid) [120s]
[3f] PASS: ecdsa_verify returned C=0 (valid) [120s]
[3g] PASS: ecdsa_verify returned C=0 (valid) [120s]
Passed: 3/3 Failed: 0/3 rc=0
$ C64_SKIP_BUILD=1 python3 tools/test_ecdsa_kat_oracle.py # control
Passed: 3/3 Failed: 0/3 rc=0

Process-level proof the flag reaches VICE (same PRG, concurrent runs):

37534 x64sc … -binarymonitoraddress ip4://127.0.0.1:6511 +sound -minimized
37619 x64sc … -binarymonitoraddress ip4://127.0.0.1:6513 +sound -minimized -reu -reusize 512

And the reason -reu stays the default — the same invocation against a
REU-profile build (make BACKEND=uci, sha256 2f43b6f3…):

$ C64_SKIP_BUILD=1 C64_VICE_NO_REU=1 python3 tools/test_ecdsa_kat_oracle.py
[3e] FAIL: ecdsa_verify returned C=1 (invalid) [60s]
[3f] FAIL: ecdsa_verify returned C=1 (invalid) [60s]
[3g] FAIL: ecdsa_verify returned C=1 (invalid) [60s]
Passed: 0/3 Failed: 3/3 rc=1

Three valid CAVP vectors rejected, no error, no diagnostic. Loud on stderr
and opt-in only, for that reason.

Blast radius. The env var is global to default_vice_config(), so any VICE
test honours it. That is intended (test_x509.py group 3 would be the next
consumer) and safe in the direction that matters: a wrongly-enabled no-REU run
makes valid signatures reject, i.e. tests fail loudly rather than pass
falsely.


Neither tools/uci/boot_check.py nor tools/_vice_helpers.py reads any
processor register by name, so the dead-"P"-key trap another lane found in
tools/test_tls_handshake.py does not apply here; test_ecdsa_kat_oracle.py
already latches carry through a 6502 trampoline (LDA #0 / ROL A / STA $0352)
and reads it back over DMA.


PRG-neutral (verified, clean builds)

This branch changes documentation, one comment block in
src/crypto/ecdsa_verify_384.s, and two test-side files. It moves no byte of
the shipped binary:

base f0127a0 make clean && make -> db31111031e2…7ba8f60, 47,105 B
this branch tip make clean && make -> db31111031e2…7ba8f60, 47,105 B

make clean is load-bearing in that check, not ceremony. An incremental build
across a BACKEND= switch produces a same-size, different-content PRG with no
warning — I hit it while running exactly this verification, and it is written
up with its mechanism in #86.

JC-000and others added 4 commits August 13, 2026 06:14
…ul boot
The pass criterion was "screen RAM contains any printable text and >= 3
distinct byte values". That distinguishes a booted machine from a blank
screen and nothing more: an ip65/RR-Net PRG booted on a U64E draws its
banner, prints NETWORK INIT FAILED, and still returned PASS (audit F4,
reproduced on hardware at 10.43.23.81).
boot_check now asserts what the boot actually has to produce, and knows
which build it is checking, because the banner is backend-aware
(`net_banner_str`: "UCI NETWORKING" under uci, "RR-NET (CS8900A)
ETHERNET" under ip65):
- `BACKEND` env (default `uci`) selects the expected banner.
- The PRG image on disk is checked for that banner before the device
is touched, so a stale artifact from a different `BACKEND=` is caught
without burning a hardware slot.
- Screen checks: common banner present, expected backend banner
present, the other backend's banner absent, no `FAILED` anywhere,
and the main menu (`Q=QUIT`) reached.
- Boot wait polls for the menu (default 60 s budget, `BOOT_TIMEOUT`)
instead of a fixed 3 s sleep, which was shorter than boot's own
reu_mul_init pass (~15-18 s on the U64E) — the old script routinely
read a mid-boot screen.
- `enable_uci` before reset when the expected backend is uci, matching
phase2_check.py; without it $DF1D never answers $C9 and net_init
reports NOT_PRESENT.
- `C64_PRG` overrides the PRG path so a deliberate wrong-backend run
can be staged.
`evaluate_screen` / `check_prg_image` are pure functions so the criterion
can be replayed against a captured screen dump.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CLAUDE.md:429 said the client "negotiates only TLS_AES_128_GCM_SHA256".
It offers exactly one cipher suite, 0x1303 TLS_CHACHA20_POLY1305_SHA256
(src/tls_handshake.s:85, echo-verified against ServerHello at :380), and
there is no AES implementation anywhere in src/crypto/. CLAUDE.md's
packaging validation record already stated it correctly, so the document
contradicted itself.
The claim is load-bearing — it is the stated reason the P-384 path keeps
a SHA-256 transcript hash. That conclusion is unchanged and still
correct: RFC 8446 §4.4.1 ties the transcript hash to the negotiated
suite's hash function, and 0x1303's hash is also SHA-256. Only the
reason is corrected.
Repo-wide sweep; same error fixed in:
- src/crypto/ecdsa_verify_384.s (3 comment sites)
- tools/package/listener/{listener.py,README.md}
tools/test_tls_handshake.py:674 keeps its mention: 0x1301 genuinely is
TLS_AES_128_GCM_SHA256 there, used as a suite the client must reject.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CLAUDE.md's packaging validation record says the shipped onchip PRG
"passes the 3-vector ECDSA KAT in VICE without REU", but every VICE test
routes through default_vice_config(), which hard-codes `-reu -reusize
512`. The claim was true and unreproducible from the repo: checking it
required monkeypatching the helper.
default_vice_config() now honours C64_VICE_NO_REU=1 by dropping the REU
flags and saying so on stderr. `-reu` stays the default on purpose — a
no-REU run of a REU-profile build does not error, it silently returns
wrong answers, because fp_mul's REU row fetch no-ops and mul_dma_lo/hi
keeps reu_mul_init's a=255 residue.
Implemented in the shared helper rather than in the KAT script so the
opt-out is available to every VICE test, and documented where the claim
is made (packaging validation record) plus a cross-reference from the
"VICE harness gotcha" note.
Evidence, onchip build (make BACKEND=uci USE_NISTCURVES_ONCHIP=1),
no patching:
C64_SKIP_BUILD=1 C64_VICE_NO_REU=1 tools/test_ecdsa_kat_oracle.py
3/3 pass, rc=0 (x64sc launched with no -reu)
C64_SKIP_BUILD=1 tools/test_ecdsa_kat_oracle.py
3/3 pass, rc=0 (control, x64sc has -reu -reusize 512)
and, on a REU-profile build, the same no-REU invocation rejects all
three valid vectors with C=1 — the silent-garbage mode that keeps REU
the default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CLAUDE.md's Known-issues entry claimed the UCI build's banner line says
"ULTIMATE 64 ELITE (UCI)". The string in src/net/uci/net.s is
"UCI NETWORKING", and has been throughout — verified against a real
U64E boot screen while validating boot_check.py. A maintainer following
that line would have written the banner assertion against a string that
never appears.
Also updates the boot_check.py entry in the UCI test-script list to
describe what it now asserts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@JC-000

Copy link
Copy Markdown
OwnerAuthor

Spot-checked by the supervising session. Two claims verified directly against the repo:

  • The banner string.src/net/uci/net.s:1015 is "UCI NETWORKING". CLAUDE.md asserted ULTIMATE 64 ELITE (UCI) — a string that appears nowhere in the tree. Anyone writing the F4 assertion from the documentation would have matched on text the machine never prints, so catching this was a prerequisite for the fix rather than a bonus.
  • The cipher-suite sweep. Five files named TLS_AES_128_GCM_SHA256 before this branch (CLAUDE.md, src/crypto/ecdsa_verify_384.s, tools/package/listener/listener.py, tools/package/listener/README.md, tools/test_tls_handshake.py). After it, only test_tls_handshake.py retains the name, and correctly so — it uses 0x1301 as a suite the client must reject, which corroborates rather than contradicts the correction.

Two judgement calls here that I think are right:

Putting C64_VICE_NO_REU in default_vice_config() rather than in the KAT script. It makes the opt-out available to every VICE test and keeps test_ecdsa_kat_oracle.py free of a second lane's edits. The negative control is the part that matters: the same flag on a REU-profile build gives 0/3, rc=1 — three valid CAVP vectors rejected with no error and no diagnostic. That is exactly why the flag has to be opt-in and loud, and why -reu stays the default.

The two structural fixes to boot_check.py. Sleeping 3 s when boot's own reu_mul_init takes 15-18 s on the U64E, and never issuing enable_uci, together meant the old check could not have observed a healthy UCI boot even in principle. And the note that the ip65 image still reaches the menu — do_net_init failure being non-fatal in boot.s — is the reason "menu reached" alone is not a sufficient criterion; the banner identity and the FAILED scan carry the weight.

Confirms audit findings F4, F8, F9.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@JC-000