Complete TLS 1.3 HTTPS handshake + GET on U64E — 8 bug fixes - #24
Merged
Conversation
6 tasks
JC-000 added a commit
that referenced
this pull request
May 21, 2026
…verlays, UCI bounded timeouts, P-384 e2e bumps, harness lock/health) (#51) * test(https_e2e): add P-384 cert profile to test listener (Phase 4c) Extends https_listener.py with a "p384" cert_profile that auto-generates a self-signed secp384r1 / ecdsa-with-SHA384 cert (mirrors the existing P-256 generator) and pins ECDH to the same curve. Profile selection via the new cert_profile= kwarg or HTTPS_LISTENER_CERT_PROFILE env var; default remains "p256" so existing tests are unaffected. Documents the openssl regeneration command in tools/https_e2e/certs/README and whitelists the README from the certs/ gitignore. Cert/key files themselves stay gitignored. Verified end-to-end with openssl 3.6.2 s_server using the README's openssl command + s_client -tls1_3 -groups secp384r1: handshake negotiates "Signature type: ecdsa_secp384r1_sha384" and "Peer Temp Key: ECDH, secp384r1, 384 bits". No src/ or C64-side changes; Phase 5 will add the matching tools/uci/ driver sibling. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(submodules): bump libs/nistcurves to 90830c9 (post-PR #23 + #24) Brings in upstream PR #23 (SHA-384 streaming hash + ecdsa_verify_with_message_384 wrapper) and PR #24 (API.md + CLAUDE.md doc cleanup). No code in c64-https references the new SHA-384 surface yet — those exports are pulled into the build only when the upcoming P-384 integration links them. P-256 path verified unchanged: tools/test_x509.py PASSES, PRG size 47105 B unchanged, CRYPTO_RESIDENT 24576 B unchanged on both backends. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(p384-overlay): wire SHA-384 + ecdsa_verify_with_message_384 wrapper into overlay image Phase 1b extension to the c64-nist-curves P-384 overlay archive. Stages sha384.s and ecdsa384.s from the sibling, plus curve384.s for the G generator constants. Overlay link overflows OVERLAY_REGION (cfg pinned at 8 KB) by 4644 B; the archive (.a) builds clean and per-segment data is captured in build/lib/overlay-p384.sizes.txt for the Phase 1.5 cfg-restructure agent. Concrete changes: (a) Source additions to the staging step: * sha384.s -> sha384_raw.s (streaming SHA-384, ~5.5 KB code+RODATA inc. K[80] round constants) * ecdsa384.s -> ecdsa384_raw.s (packaged ecdsa_verify_384 + ecdsa_verify_with_message_384 wrapper + test trampoline + fp_reverse48) * curve384.s -> curve384_raw.s (ec_a384/b384/gx384/gy384, kept for the ec_scalar_mul_384 shim's G access) * ec_scalar_mul_384_shim_raw.s (heredoc-emitted Option A shim) (b) data_raw.s extended with the 13 new BSS / DATA exports the SHA-384 + ecdsa384 modules need: sha_state(64), sha_w(640), sha_abcdefgh(64), sha_t(16), sha_scratch(64), sha_block_buf(128), sha_block_len(1), sha_total_len(16), sha384_digest(48), ecdsa384_msg_struct_ptr(2), ecdsa_result_msg_384(1), ecdsa_inputs_384(240), ec_base384_x/y(48 each) Plus all ecdsa384_* slots ecdsa_verify_384 imports (ecdsa384_r/s/h/qx/qy/w/u1/u2/u1_be/u2_be/u1g_x/u1g_y, fp_rev_buf_384). sha384_msg_buf (1 KB test scratch) is OMITTED; the .import lines in sha384.s and ecdsa384.s are stripped via sed. (c) Option A taken for ec_scalar_mul_384: STRIP the Lim-Lee body (lines 787-1488 of points384.s) and provide ec_scalar_mul_384 via an in-staging shim that copies G into ec_base384_x/y and tail-calls ec_scalar_mul_var_384. Mirrors the Phase C.4 P-256 dispatcher pattern in src/crypto/ecdsa_verify.s::ec_scalar_mul. Avoids the ~24 KB REU bank-2 anchor table + ~100 s ec_precompute_384 boot drag at the cost of a slower per-call fixed-base scalar mult (double-and-add vs. h=8 windowed comb). (d) BSD-sed compat: every `sed -i 'pattern'` invocation in both build_nistcurves_p384.sh and build_nistcurves_p384_bin.sh now uses `sed -i '' 'pattern'`. macOS BSD sed requires the empty extension; GNU sed accepts both. (e) ZP slots used for SHA-384's streaming pointers, passed via -D to ca65 over zp_config.s defaults: sha_src = $04 sha_len = $06 sha_w_ptr = $08 sha_w_ptr2 = $0a These match the sibling's defaults and DO collide with c64-https's canonical map (w32_* at $04-$09, sha_temp1 at $0a-$0d) per src/crypto/shared/zp_canon.inc. Acceptable for Phase 1b because the overlay is harness-time-only (Phase C.3b) and never linked into the production PRG. Phase 2 must resolve the collision if SHA-384 / ECDSA-with-message is wired into the production handshake -- either relocate sha_src/len/w_ptr/w_ptr2 into c64-https's free range, or repurpose $04-$0B during the brief ECDSA verify window. Documented inline in the script. (f) Forbidden-symbol guard relaxed to drop ec_gx384 / ec_gy384 (now legit via the shim) but retain cm_k_384 / ec_anchor[0-9]_384 / ec384_sc_byte/mask / ec384_precomp_i bans -- those bodies are physically removed under Option A. Build status: * nistcurves-p384.a: builds clean (157 KB archive). * overlay-p384.bin: NOT PRODUCED -- ld65 reports OVERLAY_REGION overflow by 4644 B (OVERLAY_P384 = 12836 B, slot = 8192 B). Per-segment from ld65 .map: fp384 1054 B mod384 3625 B points384 1630 B (post Lim-Lee strip) curve384 288 B sha384 5456 B (code + IV/K[80] RODATA) ecdsa384 758 B ec_scalar_mul_384_shim 25 B ---------------------- OVERLAY_P384 12836 B (overflow: +4644 B over 8192-byte slot) DATA 3541 B (RESIDENT, fits) BSS 53 B (RESIDENT, fits) Phase 1.5 must either grow OVERLAY_REGION to >= 16 KB or split OVERLAY_P384 into two halves with a runtime swap dispatcher. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(p384-overlay): split overlay into sha384 + curve halves to fit 7.5 KB slot Phase 1.5. Phase 1b's monolithic OVERLAY_P384 segment was 12,836 B and overflowed the live UCI CRYPTO_OVERLAY slot (7,680 B at $4200-$5FFF). The slot cannot grow without colliding with CRYPTO_RESIDENT at $6000. Fix is functional: split the P-384 image into two halves, each fitting the slot; load one at a time on demand. The TLS path will drive them in sequence: load sha384 overlay -> hash transcript -> load curve overlay -> call ecdsa_verify_384 with the digest pre-spliced into ecdsa_inputs_384[96..143] in resident DATA. Split rationale: * sha384 half: just sha384.s code + IV/K[80] RODATA + the SHA-384 streaming state buffers (sha_state, sha_w, sha_block_buf, ...). Unpadded 5,456 B; 2,224 B headroom. * curve half: fp384 + mod384 + points384 (Lim-Lee stripped) + curve384 + ecdsa384 (verify_384 ONLY -- the verify_with_message_384 + verify_with_msg_384_tramp wrappers were physically deleted from ecdsa384_raw.s by the build script since they import sha384_init/update/final from the OTHER overlay half) + the ec_scalar_mul_384 -> ec_scalar_mul_var_384 shim. Unpadded 7,317 B; 363 B headroom. * Both halves fit the 7,680 B live slot. ZP allocation: Sibling defaults sha_src=$04 / sha_len=$06 / sha_w_ptr=$08 / sha_w_ptr2=$0a collide with c64-https canonical use ($04-$09 = w32_* ChaCha20/Poly1305, $0A-$0D = sha_temp1 SHA-256). Phase 1.5 moves them to $3D-$44 (lowest 8-byte contiguous free block above the canonical crypto ZP map; ec_scalar_ptr ends at $3C). Verified free during the SHA-384 call window; no save/restore needed. Inheritance for Phase 4a documented in the comment block at the top of src/crypto/shared/crypto_swap.s. REU layout (src/crypto/shared/reu_layout.inc): REU_OVERLAY_P384_SHA384 = $60000 (bank 6; 8 KB image, 56 KB headroom) REU_OVERLAY_P384_CURVE = $70000 (bank 7; 8 KB image, 56 KB headroom) Banks 4-5 retained for nominal P-384 precompute reservation; not used since Phase 1b stripped the Lim-Lee body. Resident DATA growth: SHA archive resident: 1,041 B (sha_state + sha_w + sha_abcdefgh + sha_t + sha_scratch + sha_block_buf + sha_block_len + sha_total_len + sha384_digest). Curve archive resident: 2,498 B DATA + 53 B BSS = 2,551 B (fp384_*, ec384_*, ecdsa384_*, ec_base384_*, fp_rev_buf_384, ecdsa_inputs_384, ecdsa_result_msg_384). Combined: 3,592 B -- byte-for-byte the same as Phase 1b's combined data_raw.s (3,541 B + 53 B = 3,594 B; the 2 B delta is the data_*.s split-off ecdsa384_msg_struct_ptr bytes which the curve archive no longer needs after the wrapper-strip). What changed: * cfg/p384-overlay.cfg (Phase 1b monolithic) DELETED. * cfg/p384-overlay-sha384.cfg + cfg/p384-overlay-curve.cfg ADDED; each pins OVERLAY_REGION at $4200 size $1E00 to match the live UCI slot, plus DATA / BSS at $C000 for label correctness. * tools/integration/build_nistcurves_p384.sh now produces build/lib/nistcurves-p384-sha384.a + nistcurves-p384-curve.a. Adds wrapper-strip + new forbidden-symbol guard for sha384_init/update/final references in the curve-half ecdsa384_raw.s. * tools/integration/build_nistcurves_p384_bin.sh now produces build/lib/overlay-p384-sha384.bin + overlay-p384-curve.bin (each padded to 7,680 B to match the live slot) plus per-half labels and sizes reports. Fixed macOS awk strtonum-portability issue along the way. * Makefile p384-overlay target rewired for the dual-archive output. * src/crypto/shared/crypto_swap.s comment block ONLY: documents the four overlay states (none / X25519 / P-384-sha384 / P-384-curve), the TLS-side call sequence, the resident DATA invariants, and the ZP save/restore obligation Phase 4a inherits. The swap code itself is untouched (Phase 3 will add the two new entry points). Outstanding for downstream phases: * Phase 3: extend crypto_swap.s with crypto_swap_to_p384_sha384 + crypto_swap_to_p384_curve entry points; remove the now-stale crypto_swap_to_p384 (only consumer is tools/test_p384_symbols.py which Phase 3 will rewrite). * Phase 4a: implement the TLS-side dispatcher (load sha384 -> hash -> splice digest -> load curve -> verify); see the comment block at the top of crypto_swap.s for the call sequence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(tls): negotiate ecdsa_secp384r1_sha384 (0x0503) in addition to P-256/SHA-256 Phase 4b: extends the c64-https TLS layer to advertise and accept the ecdsa_secp384r1_sha384 signature scheme alongside the existing ecdsa_secp256r1_sha256, so that a future P-384 verify (Phase 4a) drops into the existing curve_id-switched ecdsa_verify dispatcher. ClientHello (src/tls_handshake.s): - signature_algorithms extension (0x000D) now lists 0x0403 + 0x0503. Inner list length bumped 2 -> 4, ext data length 4 -> 6. Payload moved to a small RODATA table + 7-instruction copy loop, because emitting two more LDA/STA/INY triples in CODE would overflow LOADER by 9 B (the Phase-6 fit-up left ~50 B of slack and CRYPTO is full). CertificateVerify (src/tls_cert.s): - Replaces the strict 0x0403-only check with a high-byte-in-{$04,$05} test (high byte minus $04 -> cv_sig_scheme: 0 = P-256, 1 = P-384). - On 0x0503, takes a P-384 short-circuit: sets ecdsa_curve_id = 1 and tail-jumps to ecdsa_verify, bypassing the P-256-specific 32-byte DER parse and SHA-256 transcript hashing (running those against a 48-byte / SHA-384 CertificateVerify would mis-parse the signature and feed the wrong digest in). ecdsa_verify currently returns C=1 for curve_id != 0 (sec/rts stub); Phase 4a replaces that branch with the real P-384 verify and the short-circuit then returns C=0 for valid signatures with no further changes here. - cv_sig_scheme exported for the negotiation test below. Test (tools/test_tls_p384_negotiation.py): - [1a] Drives tls_build_client_hello in VICE and asserts both 0x0403 and 0x0503 appear in the signature_algorithms extension payload. - [1b] Synthesizes a CertificateVerify handshake message with signature_scheme = 0x0503, calls tls_handle_cert_verify, and asserts cv_sig_scheme = 1, ecdsa_curve_id = 1, and C = 1 (the expected stub-dispatcher rejection — proves the routing reached the ECDSA layer without touching the still-stubbed P-384 verify). Phase 4a will need to flip this assertion to C = 0 against a real signature once the dispatcher branch is filled in. Verification: - tools/test_x509.py: 11/11 P-256 assertions still pass (3c valid signature in 60 s under VICE warp; no regression). - make and make BACKEND=uci both build cleanly with no new warnings. - New negotiation test: 2/2 PASS under VICE warp (-reu). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(crypto): activate dual-overlay swap dispatcher (P-384 SHA + curve, X25519 sibling) Phase 3. Wires the four-state crypto overlay dispatcher to the split P-384 build (Phase 1.5) and populates REU banks 6 / 7 from PRG-embedded images at boot. State machine (src/crypto/shared/crypto_swap.s): - OV_NONE = 0 (boot default) - OV_X25519_SIBLING = 1 (state-only marker; no DMA -- the sibling rodata occupies CRYPTO_OVERLAY at PRG load time when USE_X25519_SIBLING=1, so a real REU stash is deferred to a follow-up) - OV_P384_SHA384 = 4 (REU bank 6 -> $4200, idempotent) - OV_P384_CURVE = 5 (REU bank 7 -> $4200, idempotent) The legacy crypto_swap_to_p256 / _p384 entries (single-image, stale post-Phase-1.5) and the OV_P256 / OV_P384 IDs are dropped. Boot REU population (src/boot.s::reu_p384_overlay_init): - Strategy chosen: ".incbin into a fixed RAM region" per the task spec, with one twist forced by the C64 memory map: * SHA blob ($1E00 = 7,680 B) -> CRYPTO_OVERLAY ($4200-$5FFF) at PRG load time; STASH'd to REU bank 6 in one DMA. * CURVE blob -> under-KERNAL RAM at $E000-$FDFF at PRG load time. Direct STASH from $E000 was empirically broken under VICE's REU emulator (returned an undefined fill pattern even with $01 banked off; verified vs every other source address). Workaround: CPU-copy curve blob from $E000 (KERNAL banked off) to $4200 (now-free SHA staging slot), then STASH from $4200 to REU bank 7. ~38 K cycles at 1 MHz / ~0.8 ms at 48 MHz. - Inert under USE_X25519_SIBLING=1 / BACKEND=ip65 (no main-RAM headroom; gated via USE_OVERLAY_P384_EMBED in the Makefile). Cfg + Makefile: - cfg/c64-https-uci.cfg: drop the vestigial TCP_BUF MEMORY region (tcp_recv_buf is just an equate at $C000); add OVERLAY_FILE_PAD ($C000-$DFFF, 8 KB zero-fill so KERNAL LOAD lands the OVERLAY_BLOB_CURVE_RAM bytes at the right address) and OVERLAY_BLOB_CURVE_RAM ($E000-$FDFF). Add OVERLAY_BLOB_SHA384 + OVERLAY_BLOB_CURVE segments. PRG grows from 47 KB -> 62 KB. - cfg/c64-https-ip65.cfg: add the same segment names but anchored to zero-size aliases (no embed under ip65). PRG stays 47105 B. - Makefile: define USE_OVERLAY_P384_EMBED under BACKEND=uci + !USE_X25519_SIBLING; pull build/lib/overlay-p384-{sha384,curve}.bin into PRG_DEPS so the .incbin in p384_overlay_blobs.s sees them. reu_layout.inc: trim OVERLAY_SIZE from $2000 to $1E00 to match the actual live slot + .bin sizes. The previous $2000 caused crypto_swap.s to DMA 512 bytes past the slot into the start of CRYPTO_RESIDENT RODATA -- silently corrupting it on every swap. tools/test_p384_symbols.py: rewritten end-to-end for the dual-overlay flow. Verifies both .bin sizes, asserts boot leaves current_overlay = OV_NONE, JSRs each swap entry in turn while reading back current_overlay and the first 16 B at $4200. Idempotent re-swap + direction-reversal round-trip both pass. Uses VICE extra_args=["-reu", "-reusize", "512"] per the documented harness gotcha (sibling P-256 fp_mul fetches REU mul rows; without -reu the boot path silently no-ops). 10/10 PASS on a clean run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(crypto): TLS-side P-384 verify dispatcher (dual-overlay swap + sha384 + ecdsa_verify_384) Phase 4a wires the TLS layer's CertificateVerify P-384 path through to the sibling c64-nist-curves ecdsa_verify_384 entry. The new dispatcher composes the Phase 3 dual-overlay swap dance with sha384 hashing of the TLS 1.3 §4.4.3 signed-content blob: 1. Parse DER-encoded ECDSA signature out of tls_rec_buf into ecdsa_inputs_384[0..47] (r) and [48..95] (s) (BE, right-aligned, leading-zero pad tolerated). The Phase 4b short-circuit skipped the in-tree P-256 DER parser, so the dispatcher does its own. 2. Copy ecdsa_pubkey_x/y into ecdsa_inputs_384[144..239] (Qx, Qy). 3. Build the 146 B signed-content blob at $CA00 in tcp_recv_buf scratch RAM: 64 spaces || "TLS 1.3, server CertificateVerify" (33 B) || 0x00 separator || 48 B transcript hash placeholder (.assert pins the byte count.) tcp_recv_buf is idle during crypto and the chosen window sits above both overlays' resident DATA so it survives the swaps. 4. crypto_swap_to_p384_sha384 -> sha384_init / update / final. 5. Splice sha384_digest ($C3E1, 48 B BE) into ecdsa_inputs_384[96..143]. 6. crypto_swap_to_p384_curve -> ecdsa_verify_384 ($5BDD entry). Carry returns through the dispatcher to ecdsa_verify and the TLS handler. Hookup: src/crypto/ecdsa_verify.s P-384 branch (was sec/rts stub) now jumps to ecdsa_verify_384_tls. Tail-call so the carry propagates naturally. Overlay-resident symbols (sha384_init/update/final, sha384_digest, ecdsa_verify_384, ecdsa_inputs_384) are NOT linked from the main PRG -- the overlay images are DMA'd in at runtime. The dispatcher declares them as numeric equates sourced from build/labels-p384-{sha384,curve}.txt; if those addresses move when the overlay images are rebuilt the equates must be re-synced. ZP usage $3D-$44 (sha_src/sha_len/sha_w_ptr/sha_w_ptr2) per Phase 1.5: demonstrably free across the SHA-384 window in c64-https, no save/restore. Phase 4a CAVEATs (will be addressed in Phase 5): - SHA-384 transcript: c64-https only runs a SHA-256 transcript (tls_transcript, 32 B). The dispatcher zero-pads it to 48 B for shape -- the swap + verify mechanism executes end-to-end but a real signature will return C=1 against any server until tls_transcript_384 lands. - ecdsa_pubkey_x/y are 32 B in data.s for the P-256 packed-struct invariant required by ecdsa_verify_256. Resizing them breaks the P-256 tail-call. The dispatcher copies 48 B from each (walks into adjacent BSS for P-384), which is no worse than the SHA-384 transcript placeholder; both fix together in Phase 5. Test: tools/test_tls_p384_negotiation.py subtest [1b] still PASSes end-to-end. Pre-Phase-4a it asserted C=1 from the sec/rts stub; post-Phase-4a it asserts C=1 from the dispatcher's DER parse rejecting the 48-zero-byte synthetic signature (first byte must be 0x30 SEQUENCE). The negotiation contract under test is unchanged. PRG size: 62977 B (UCI), unchanged -- the dispatcher (309 B) lands in CRYPTO_AUX_CODE which rides NET_CODE under UCI; cfg's fill=yes makes the PRG file size constant regardless of code added in NET_CODE. ip65 backend (47105 B) also builds clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(p384): KAT smoke test for ecdsa_verify_384 dual-overlay flow Adds tools/test_ecdsa_p384_kat.py — bare-metal driver for the Phase-3 dual-overlay verify path (crypto_swap_to_p384_sha384 → sha384_init/ update/final → splice digest into ecdsa_inputs_384[96..143] → crypto_swap_to_p384_curve → jsr ecdsa_verify_384). Independent of the TLS-side dispatcher (Phase 4a) so it exercises the swap + curve overlay in isolation against NIST CAVP / RFC 6979 P-384 vectors. Per-step run_subroutine invocations (one jsr per crypto step) with tight per-step timeouts make build-order and field-arithmetic hangs surface at the failing step rather than as opaque "stub never returned" timeouts. Two small 6502 stubs (12 B splice + 15 B verify) live in the OVERLAY_FILE_PAD tail past the curve overlay's resident DATA at $C9F7. Backend-agnostic via UnifiedManager; --u64 enables hardware runs (skip-by-default). Sub-modes: --sha-only Skip the (slow) ecdsa_verify_384 step but exercise the full dual-overlay swap dispatch + SHA-384 + splice path. All 4 vectors structurally PASS in ~0.1 s/vector under VICE warp; confirms the test infrastructure + Phase 3 swap dispatcher are wired up correctly even when the verify itself is too slow to complete in-session on a constrained host. --full 4 vectors (RFC 6979 positive + LSB-flip-r negative + first CAVP P + first CAVP F). Default is the RFC 6979 positive only. Build-order trap discovered + worked around: the overlay-bin link script (tools/integration/build_nistcurves_p384_bin.sh) reads build/labels.txt to resolve mul_dma_lo / mul_dma_hi / mul_cached_a / reu_fetch_mul_row at the SAME runtime addresses the main PRG uses. On a clean build labels.txt does not exist when the overlay-bin step runs, those symbols stub out to $0000, and the curve overlay's fp_mul_384 reads/writes $0000 — the verify hangs in field arithmetic with no obvious symptom. The test does a two-pass build internally (make → touch the script → make again) to land a self-consistent overlay; an explicit canary asserts the curve overlay's labels are non-zero post-build. Cleaner upstream fix: add build/labels.txt as an order-only dep on the overlay-bin Make target. Verify wall-clock under VICE warp on the development Mac was >22 min per vector — likely host-specific (the upstream tools/test_ecdsa_verify.py runs P-384 verify under 600 s on its CI machine) — so the ecdsa_verify_384 path is not validated end-to-end in this commit on this host. --sha-only PASSes 4/4 vectors at 0.05-0.10 s overall_dt, proving the dual-overlay flow up to and including the second swap is correct. Validation of the verify itself is best done on a faster VICE host or on real U64E hardware (--u64 path, expected 3-5 min per verify per the bench data). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(p384): shrink CertificateVerify signed blob 146 -> 130 B (Phase 5 Fix A) The TLS 1.3 transcript-hash function is bound to the cipher suite's hash (RFC 8446 §4.4.1), not to the signature_algorithm. c64-https only negotiates TLS_AES_128_GCM_SHA256, so the transcript embedded in the CertificateVerify signed-content blob is always 32 B SHA-256, even when the signature is ECDSA-P384/SHA-384. The Phase 4a draft built a 146 B blob with the 32 B SHA-256 transcript zero-padded out to 48 B (SHA-384 digest width). That feeds the verifier a different message than the one the server signed, so any real server signature would be rejected. This commit: - SIGNED_BLOB_LEN: 146 -> 130 (64 + 33 + 1 + 32). - .assert pin updated to (64 + 33 + 1 + 32) = 130. - Step 3 transcript copy: copy 32 B verbatim, drop the 16 B zero-pad. - Trim the documented blob window from \$CA00..\$CA91 to \$CA00..\$CA81; tcp_recv_buf still has plenty of slack. SHA-384 still produces the 48 B digest spliced into the h slot - the hash function and digest size used to derive the message hash are independent from the transcript-hash function. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(p384): separate 48 B P-384 pubkey BSS slots (Phase 5 Fix B) ecdsa_pubkey_x and ecdsa_pubkey_y in src/data.s are 32 B each, sized for P-256 as part of the contiguous r|s|h|Qx|Qy packed struct that ecdsa_verify_256 reads verbatim. The cert handler was writing 48 B per coordinate when ecdsa_sig_len = 48 (P-384), overrunning the slots and clobbering the next BSS variables. The dispatcher then read partially- corrupt pubkey bytes. Option (a) (per Phase 5 task brief — lighter than widening the P-256 struct): add separate ecdsa_pubkey_x_384 / ecdsa_pubkey_y_384 slots in CRYPTO_BSS, dispatch in the cert handler on ecdsa_curve_id, and have the P-384 dispatcher read from the new slots. Changes: - src/data.s: declare and export ecdsa_pubkey_x_384 / _y_384 (48 B each, +96 B in CRYPTO_BSS — well within the slack reclaimed by Phase 6's tls_hs_buf removal). - src/tls_cert.s: x509_extract_pubkey's X / Y copy loops dispatch on ecdsa_curve_id, writing P-256 pubkeys into the legacy 32 B slots (preserving the sibling ecdsa_verify_256 packed-struct invariant) and P-384 pubkeys into the new 48 B _384 slots. - src/crypto/ecdsa_verify_384.s: import _384 slots; copy_qx / copy_qy redirected to read from them; doc-block updated to reference Fix B. P-256 verify path is unchanged; only the P-384 cert + dispatcher path is affected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(p384): generate overlay-resident equates from labels at build time (Phase 5 Fix C) Phase 4a hand-pasted six addresses (sha384_init, sha384_update, sha384_final, sha384_digest, ecdsa_verify_384, ecdsa_inputs_384) into src/crypto/ecdsa_verify_384.s, sourced from build/labels-p384-*.txt. If the overlay images are rebuilt and any address moves, no link-time check fires - the dispatcher silently calls stale addresses. This commit: - tools/integration/gen_p384_overlay_equates.sh: extracts the six addresses from build/labels-p384-{sha384,curve}.txt and emits build/p384_overlay_equates.inc with ca65-syntax equates. Atomic write via mktemp + mv. Fails loudly if any required label is absent from the overlay labels. - Makefile: regenerate build/p384_overlay_equates.inc whenever either labels file or the generator script changes; declare build/crypto/ecdsa_verify_384.o as dependent on it; add to PRG_DEPS under USE_OVERLAY_P384_EMBED. Add -I build to CA65FLAGS so .include "p384_overlay_equates.inc" resolves. - src/crypto/ecdsa_verify_384.s: replace the hand-pasted equates block with .include "p384_overlay_equates.inc"; add six .assert pins that catch drift if the overlay slot ($4200..$5FFF) or the overlay-resident DATA window ($C000..$CFFF) invariants ever break. Build flow: bumping libs/nistcurves or restructuring an overlay cfg now propagates address changes through to the dispatcher .o (with a build-time .assert failure if the symbols land outside their slots), rather than producing a silently-wrong PRG. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(p384): order-only labels.txt dep on overlay-bin + script hardening (Phase 5 Fix D) Phase 2 finding: tools/integration/build_nistcurves_p384_bin.sh looks up four main-PRG addresses (mul_dma_lo, mul_dma_hi, mul_cached_a, reu_fetch_mul_row) in build/labels.txt to resolve them to the runtime locations the curve overlay's fp_mul_384 reads/writes. On a clean build labels.txt didn't exist when the overlay-bin step ran, the lookup_label() fallback stubbed all four to \$0000, and the curve overlay silently produced an image whose fp_mul_384 read/wrote \$0000 instead of \$BA00 / \$BB00 - silent corruption with no obvious symptom downstream. Changes: - Makefile: declare build/labels.txt as an ORDER-ONLY ('|') dependency on the overlay-bin rule. Order-only ensures labels.txt exists before the script runs without triggering an overlay rebuild on labels.txt mtime changes alone. - Makefile: respect a command-line USE_OVERLAY_P384_EMBED=0 (was forced to 1 under UCI). The two-step bootstrap workflow for a clean tree is documented inline: make BACKEND=uci USE_OVERLAY_P384_EMBED=0 # produce labels.txt make BACKEND=uci # real link - Makefile: gate the build/crypto/ecdsa_verify_384.o dep on the generated overlay-equates .inc by USE_OVERLAY_P384_EMBED so the bootstrap link doesn't chase the labels-p384-* -> overlay-bins -> labels.txt cycle. - src/crypto/ecdsa_verify_384.s: gate the .include on USE_OVERLAY_P384_EMBED with stub equates ($4200/$C000 within legal range) for the bootstrap path. - tools/integration/build_nistcurves_p384_bin.sh: replace the silent \$0000 fallback with a hard error pointing at the bootstrap workflow. lookup_label() now exits 4 if a required symbol is missing from build/labels.txt; missing build/labels.txt exits 3. After bootstrap, plain 'make BACKEND=uci' rebuilds incrementally without intervention. The order-only dep prevents spurious overlay rebuilds when labels.txt mtime changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(p384): tools/uci/test_https_local_p384.py + CLAUDE.md update (Phase 5) Phase 5 e2e wiring for the P-384 TLS handshake against a local listener serving an ECDSA secp384r1 cert. - tools/uci/test_https_local_p384.py: thin wrapper around tools/uci/test_https_local.py that swaps CERT_PATH / KEY_PATH to tools/https_e2e/certs/server-p384.{pem,key} and bumps default SENTINEL_POLL_TIMEOUT / ACCEPT_TIMEOUT to 30 min (the ECDSA-P384 verify is the dominant cost; expect 4-7 min per handshake at U64E 48 MHz turbo). Same flow / artifact pattern as the P-256 sibling - per-run dir under /tmp/uci_https_debug, 6510 bus capture, TLS state DMA snapshot, server-side result JSON. - CLAUDE.md: extend "End-to-end HTTPS status" with the P-384 wiring summary (negotiation + cert handler dispatch + Phase 5 Fix A 130 B signed blob + Fix B separate _384 pubkey slots) and add an "ECDSA P-384 verify wall-clock" subsection documenting the 4-7 min handshake expectation. Wall-clock number itself is "not yet measured end-to-end" - Phase 5's U64E test host was unreachable from the dev machine when this landed (DeviceLock unavailable; ping/TCP both timed out to the default 192.168.1.81). The four upstream Phase 5 fixes (A/B/C/D) and the negotiation plumbing test (2/2 PASS) all landed clean; running tools/uci/test_https_local_p384.py from a host with U64E LAN access will capture the wall-clock number and complete the e2e validation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(uci): bounded uci_wait_not_busy (TOD-budgeted, mirrors uci_wait_idle) Phase 5 e2e on U64E at 10.43.23.81 wedged in `uci_wait_not_busy` while fetching the record after Certificate (CertVerify recv). The unbounded spin polled $DF1C for 268K cycles with no progress; the test sentinel took 1843 s to fire. This is the exact hazard CLAUDE.md predicted in "Design note - bounded timeouts must use wall-clock time": `uci_push_wait` and `uci_wait_not_busy` were still unbounded and converted FPGA wedges into wall-clock timeouts. Conversion mirrors the issue #37 template for `uci_wait_idle`: * CIA1 TOD sampled at entry ($DC0B HOUR latch, $DC08 TENTHS unlatch). * Re-read TENTHS on each spin pass; bail with C=1 + net_last_error = UCI_ERR_WAIT_TIMEOUT after 50 transitions (~5 s wall-clock, CPU-speed-independent). * State lives in two SMC bytes inside the routine (no ZP, matches the file's no-zero-page convention). `uci_push_wait` inherits the bound automatically via its tail-`jmp` into `uci_wait_not_busy` (no separate conversion needed; same C return flag, same error code). No `uci_end_cmd` exists in code - the CLAUDE.md primitives list had a stale reference, now corrected. Caller audit - all six `jsr uci_wait_not_busy` / `jsr uci_push_wait` sites in src/net/uci/net.s now `bcs` out on C=1 to surface the timeout: * net_poll (wait_not_busy + push_wait): force tcp_state=ERROR * net_dhcp_acquire (push_wait): bail via @dhcp_wait_to * net_tcp_connect (push_wait): force tcp_state=CONNECT_FAIL * net_tcp_send (push_wait): sec + rts * net_tcp_close (push_wait): force tcp_state=CLOSED PRG sizes: ip65 backend unchanged (47105 B - none of UCI code linked); UCI backend PRG size also unchanged at 62977 B because UCI_CODE has slack to LOADER_OVERFLOW. Segment delta: UCI_CODE grew by 87 bytes ($6E8 -> $73F) for the bounded-helper logic + caller bcs trampolines. CLAUDE.md updated: * "UCI command primitives" section now lists `uci_wait_not_busy` as TOD-bounded and removes the stale `uci_end_cmd` reference. * Design note's "still unbounded" callout for `uci_push_wait` / `uci_wait_not_busy` removed; added a sub-paragraph describing the Phase 5 wedge that drove the conversion. Not tested on real U64E hardware (per the Phase 5b worker constraint); the bound semantics rely on CIA1 TOD which works identically on VICE and U64E. ip65 + UCI builds clean; tools/test_entropy.py 7/7 PASS on the ip65 build to confirm general build health. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tools/uci): factor P-384 arbiter monkey-patch into shared helper Phase 5's OVERLAY_BLOB_SHA384 (P-384 build) fully occupies CRYPTO_OVERLAY ($4200-$5FFF), so the default arbiter window in build_policy_and_arbiter() finds zero free bytes and raises MemoryArbiterError. The P-384 wrapper had a private workaround (_p384_build_policy_and_arbiter, ~80 LOC) that carved scratch from the NET_CODE zero-fill tail; that workaround now applies to the P-256 baseline too because more overlay landings push CRYPTO_OVERLAY's free tail below the 387 B the e2e tests need. Promote the carveout to a shared helper in _memory_policy.py (build_policy_and_arbiter_with_overlay_carveout), make test_https_local.py call it by default, and delete the inline monkey- patch from test_https_local_p384.py (it just inherits via the test_https_local.main() delegation now). The original build_policy_and_arbiter is kept as-is for back-compat with any tools/uci/* script that's happy with the CRYPTO_OVERLAY window. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(uci): adopt new c64-test-harness lock/health pattern in test_https_local c64-test-harness PR #88 added two opt-in helpers that pre-detect the two failure modes that previously cost ~10 min per occurrence: - DeviceLock.acquire_or_raise(timeout) -> raises DeviceLockTimeout with structured diagnostics (holder PID + liveness, lockfile age, REST reachability) instead of the legacy bool-return that left the caller guessing whether to wait, kill the holder, or call a human. - runner_health_check(client) -> raises Ultimate64RunnerStuckError when the firmware's "Cannot open file" wedged-runner state is detected, instead of letting subsequent run_prg() calls silently fail and the test time out at the sentinel poll. Adopt both in tools/uci/test_https_local.py (which the P-384 sibling inherits via the test_https_local.main() delegation). On wedge or timeout we surface the diagnostic and exit non-zero -- we never call recover() or any other state-changing action automatically (that requires explicit user authorization). Also surface lock.read_info() on the kickoff line so the supervisor log has visibility into queue state from the very first message. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(uci): bump SENTINEL_POLL_TIMEOUT to 5400s for P-384 e2e P-384 handshake on U64E 48 MHz takes longer than the 30-min budget — handshake actually progresses through Server Finished decrypt within 1812s (per Phase 5i diagnostic of artifact /tmp/uci_https_debug/20260516_152824/, read_seq=4 + tls_rec_buf shows freshly-written Finished). Bump to 5400s to allow Client Finished + HTTP exchange to complete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(uci): bound uci_drain_resp + uci_drain_status (TOD, mirrors uci_wait_not_busy) Phase 5j secondary-risk fix per CLAUDE.md brief. The two response-drain primitives in `src/net/uci/uci_cmd.s` were the last unbounded `jmp <self>` loops in the UCI adapter. Call chain that motivated the fix: tls_send_finished -> tls_record_write -> net_tcp_send -> uci_drain_resp (post-SOCKET_WRITE response drain) -> uci_drain_status (post-SOCKET_WRITE status drain) If firmware ever left DATA_AV / STAT_AV asserted after a SOCKET_WRITE response, control would wedge in the drain with no wall-clock escape - the same hazard CLAUDE.md predicted in "Design note - bounded timeouts must use wall-clock time" and that drove the issue #37 / Phase 5b conversions of `uci_wait_idle` / `uci_wait_not_busy`. Conversion mirrors the Phase 5b template: * CIA1 TOD sampled at entry ($DC0B HOUR latch, $DC08 TENTHS unlatch). * Re-read TENTHS on each iteration; bail with C=1 + net_last_error = UCI_ERR_WAIT_TIMEOUT after 50 transitions (~5 s wall-clock, CPU-speed-independent). * State lives in two SMC bytes inside each routine (no ZP, matches the file's no-zero-page convention). * Normal exit (FIFO drained) now returns C=0 explicitly (was: fall through to RTS with carry undefined - works because all callers discarded the flag, but the new BCS check makes the contract explicit). Caller audit - all 13 `jsr uci_drain_resp` / `jsr uci_drain_status` sites in `src/net/uci/net.s` now `bcs` out on C=1 to skip the companion drain + ack and force the appropriate exit state: * net_poll error path (line 183): tcp_state already ERROR, RTS. * net_poll @hdr_done_short (line 218): force tcp_state = ERROR, RTS. * net_poll @have_data zero-len (line 233): force tcp_state = ERROR, RTS. * net_poll @done_data (line 314): force tcp_state = ERROR, RTS. * net_dhcp_acquire (line 392): bail via @dhcp_wait_to (sec + rts). * net_tcp_connect err path (line 501): sec + rts; net_last_error preserved as UCI_ERR_WAIT_TIMEOUT from the drain. * net_tcp_connect ok path (line 524): force tcp_state = CONNECT_FAIL, sec + rts. * net_tcp_send err path (line 682): sec + rts; SEND_FAIL already set. * net_tcp_send ok path (line 701): sec + rts; net_last_error preserved as UCI_ERR_WAIT_TIMEOUT. * net_tcp_close (line 746): force tcp_state = CLOSED, RTS (best-effort semantics preserved). Build verification: `make BACKEND=uci` clean on a freshly bootstrapped tree. PRG size unchanged at 62977 B (UCI_CODE has slack to LOADER_OVERFLOW). Segment delta: UCI_CODE grew by 169 bytes ($73F -> $7E8) for the bounded-helper logic + caller bcs trampolines. CLAUDE.md updated: * "UCI command primitives" section now lists `uci_drain_resp` / `uci_drain_status` as TOD-bounded and describes the caller bcs pattern. * Design note section adds a paragraph describing the Phase 5j drain conversion and the call chain that motivated it. Not tested on real U64E hardware in this commit (the e2e wall-clock update follows as a separate commit on PASS). The bound semantics rely on CIA1 TOD, which works identically on VICE and U64E. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * build(make): emit cc65 debug info (-g + --dbgfile) for PRG + overlays Add cc65 debug-info generation across the build pipeline as a purely additive sidecar — PRG bytes are unchanged (verified bit-for-bit identical SHA-256 across before/after incremental builds). Rationale: better symbol / source-line mapping in VICE's monitor and for diagnostic agents that walk crash dumps, REU traces, and DMA captures. The cc65 `.dbg` format pairs cleanly with VICE's `-moncommands` and the c64-test-harness label readers; without it, the only out-of-PRG metadata is `build/labels.txt` (symbol-only, no source mapping). Changes: * Makefile: add `--dbgfile build/c64-https.dbg` to `LD65FLAGS`. `CA65FLAGS` already carried `--debug-info`, so all ca65 invocations driven from the Makefile already embedded per-source debug records. * tools/integration/build_nistcurves_p256.sh, tools/integration/build_nistcurves_p384.sh, tools/integration/build_x25519.sh: add `-g` to every ca65 call that produces a `.o` contributing to the final PRG. These were the only ca65 invocations that did NOT inherit `CA65FLAGS`. * tools/integration/build_nistcurves_p384_bin.sh: add `--dbgfile` pointing at `build/lib/overlay-p384-{sha384,curve}.dbg` for the two overlay-image link steps, derived from the .bin path. `make clean` already removes `build/` recursively so no separate sweep for the new `.dbg` sidecars is needed. Verification: * `make BACKEND=uci` (incremental, no clean — test in flight on PID 38329) rebuilt only the dispatcher + overlay-blob .o and re-linked. `sha256sum build/c64-https.prg` matches pre-edit: d72f733dc15ed1e7508f496a2daeda10433c6f3d864b05d8a7c584788ffb0b79 * `build/c64-https.dbg` is 1.44 MB, 31113 lines, valid cc65 format (`version major=2,minor=0` / `info csym=… file=49 mod=42 sym=5181`). * `build/lib/overlay-p384-sha384.dbg` (632 lines) and `build/lib/overlay-p384-curve.dbg` (2882 lines) generated alongside the padded .bin images. * Second `make BACKEND=uci` is a no-op (idempotent). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(CLAUDE.md): note cc65 .dbg sidecars in Build section Build now emits build/c64-https.dbg (and overlay .dbgs) per commit 28e042e. Diagnostic tools and VICE monitor can consume these for PC->symbol/source mapping instead of relying on labels.txt alone. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
HTTPS TLS 1.3 handshake + HTTP GET now works end-to-end on Ultimate 64 Elite hardware at 48 MHz turbo against the local Python TLS listener (ECDSA-P256 cert under
tools/https_e2e/certs/). The full flow — ClientHello → ServerHello → EncryptedExt → Certificate → CertificateVerify → Server Finished → Client Finished → app-keys → HTTP GET → 200 response — completes in ~145 s wall-clock and returns body"HELLO FROM TLS SERVER".8 latent bugs were found and fixed (all previously dormant because the handshake had never progressed this far), plus a Content-Length-driven body termination that closes the long-standing
http_getpoll-counter issue. A newLOADER_OVERFLOWsegment carries ~125 B ofhttp.sgrowth that did not fit in LOADER's slack.Bug fixes
src/tls13.s(2bb014a) — CertVerify and Finished were reading the transcript hash after their own bytes had been folded in; fix snapshots before dispatch and folds after.fp_zeroY-clobber across calls —src/crypto/ecdsa_verify.s(f006f10) —ecdsa_parse_der_siglost Y acrossfp_zero, mis-parsing DER signatures.src/tls_cert.s(4a2c4f3) — signed-digest length field was off by one, causing CertVerify hash mismatch.tls_derive_traffic_keyssuccess path —src/tls_keyschedule.s(cf8d326) — success path did not set CLC, so callers saw a spurious error.src/tls13.s(4130356) — Client Finished must be sent before deriving app traffic keys; also reloadhkdf_prkfromtls_c_hs_secretbefore the Finished HMAC.src/tls13.s(a18f324) — RFC 8446 §5.3:tls_write_seq/tls_read_seqmust reset to 0 at the key-schedule transition.http.sstate_body early-exit on empty ring —src/http.s(7821ffb) — body-read returned "done" on the first empty ring tick instead of polling.http_recv_responsereturned@not_doneon state transitions —src/http.s(fbc7d10) — fall through on 0→1 and 1→2 so a single dispatch can walk status + headers + body when they all fit in one TLS record.Bonus: Content-Length-driven body termination —
src/http.s+src/data.s+ both cfg files (768717a,36ce8bc) — adds a 16-bithttp_content_lengthsentinel in SHADOW_BSS, populated by a case-insensitiveContent-Length:header match.http_getnow exits cleanly whenhttp_resp_lenreaches the declared length (was previously waiting for the 40-min poll counter to expire).Supporting work
zp_countfix —src/tls_transcript.s(2f01c0d) — cherry-picked from an earlier branch;tls_transcript_updatewas truncating lengths to 8 bits.tools/uci/bench_ecdsa_u64e.py(9fc5021) — confirmsecdsa_verifycorrectness on 5 vectors and measures ~85 s median wall-clock at 48 MHz turbo on real U64E hardware.9f54f77— addsecdsa_*state totls_state_dump.jsonfor post-mortem verify.a43b42f— bumped to 600 s to accommodate ECDSA wall-clock.350956a— UCI backend, end-to-end HTTPS status, hardware test scripts.Memory layout change
A new
LOADER_OVERFLOWsegment loads into the tail of NET_CODE for ~125 B ofhttp.sgrowth (Content-Length parser + digit pattern) beyond LOADER's ~50 B of slack. It rides after the ip65 blob under the ip65 backend and after the UCI adapter under the UCI backend. Declared in bothcfg/c64-https-ip65.cfgandcfg/c64-https-uci.cfg. Reachable via JSR from LOADER-resident code.Verification
make BACKEND=ip65andmake BACKEND=uciproduce clean PRGs.tools/uci/test_https_local.pyPASS —http_status=200,http_resp_len=21, body"HELLO FROM TLS SERVER", 145 s wall-clock at 48 MHz turbo.Known follow-ups (not in this PR)
c64-x25519) — drop-in via the Crypto ABI.tools/uci/test_http_live.py) is untested with the currentuci_fenceconfiguration.Test plan
test_http,test_tls_handshake,test_x509,test_transcript_large,test_hkdf,test_chained_hmac,test_keyschedule_steps,test_entropyall greenmake BACKEND=ip65andmake BACKEND=uciboth cleantools/uci/test_https_local.pyPASS withDEBUG_CAPTURE=1LOADER_OVERFLOWsegment) for the ip65 build path🤖 Generated with Claude Code
Originally posted by @JC-000 on 2026-04-18