Phase 3 WIP: layout fix + TLS receive-path fixes (hits architectural limits)

[JC-000]

Summary

• End-to-end bridge test framework for c64-https running in VICE over RR-Net (Phase 1 DHCP and Phase 2 plain HTTP GET pass on this branch).
• Load-bearing memory-layout fix: relocate the entire crypto blob above ip65's BSS window to eliminate a CPU JAM caused by DHCP writing packet bytes on top of sha256_rotr8 code.
• TLS 1.3 receive-path chain-of-bugs repaired (plaintext/encrypted dispatch, X-wrap in TCP recv callback, missing clc, never-called ECDH shared-secret, CCS filter).
• TCP receive buffer grown to 4 KB with 16-bit indices, aead_data_len widened to 16-bit so TLS 1.3 Certificate records no longer overflow or truncate.
• Phase 3 diagnostic instrumentation (screen markers, 16-bit net_poll entry/return counters, dynamic label lookup in the test harness) preserved for the next session.

What's included

5feb0ce — Add end-to-end bridge tests for DHCP and HTTP GET

Vendors the c64-test-harness bridge networking scripts (setup + cleanup) and extends them with dnsmasq (DHCP + DNS overrides). Builds a reusable tools/https_e2e/ library with a BridgeEnv context manager, single-VICE launcher, boot-menu helpers, and HTTP listener. Two passing e2e tests drive the real c64-https binary inside VICE over RR-Net at normal speed: Phase 1 verifies DHCP, Phase 2 verifies a plain HTTP GET to a local listener. This is the scaffolding every later phase rides on.

d973531 — Relocate crypto above ip65 BSS + multiple TLS receive-path fixes

The original CPU JAM at $4DE0 was caused by c64-https crypto code streaming from $3B28 straight through ip65's BSS window at $4000-$5FFF: when DHCP ran, it overwrote sha256_rotr8 with packet bytes. Inserting * = $6000 before crypto/word32.asm in src/main.asm moves every crypto unit (word32/chacha/poly1305/aead/sha256/hmac_drbg/fe25519/x25519/ecdsa/ecdsa_verify/der/tls_cert/tls_ecdh) above the BSS. To fit in the space below $7C00 three compromises were required: sqtab_lo/sqtab_hi moved from hardcoded $7800/$7A00 equates to labels inside data.asm BSS; P-384 ECDSA was stubbed in crypto/ecdsa_verify.asm (tech debt tracked in project_p384_stubbed, must be restored before real CA chains); and the old * = $7C00 barrier was removed. With the JAM gone, a chain of follow-on TLS receive-path bugs became diagnosable and were fixed in the same commit (see "TLS receive-path fixes" below). Also renames the demo URL from www.apple.com to www.foo.bar throughout boot.asm, dnsmasq overrides, HTTPS listener cert subject/SAN, and test screen text, so a stray packet can never escape the bridge to a real host. The bridge setup/cleanup scripts are hardened (no blanket pkill, specific PID tracking, idempotent re-entry, stale-rc-file handling). Adds tests/test_phase3_https.py with granular post-mortem diagnostics via dynamic label lookup from build/labels.txt.

c8ca348 — Grow tcp_recv_buf to 4KB + widen aead_data_len to 16-bit

Two coordinated architectural changes that unblock the TLS 1.3 handshake past ServerHello. The previous 256-byte ring buffer with 8-bit head/tail silently overwrote unread data whenever ip65 delivered a TCP segment larger than the remaining space — the post-ServerHello flight (CCS + EncryptedExtensions + Certificate + CertVerify + Finished, 600–800 bytes) always overflowed. tcp_recv_buf is now a 4096-byte buffer pinned to $C000-$CFFF (always-RAM region on the C64 with BASIC ROM banked out) declared as an equate in constants.asm, so it consumes zero BSS. Head/tail are 16-bit with mask-on-access ($0FFF), and a tcp_recv_overflow sticky flag is set by the producer in net_tcp_recv_cb if it would lap the consumer. net.asm consumer/producer paths and http.asm's feed loop are updated to 16-bit indexing. Separately, aead_data_len in data.asm is widened from 1 byte to 16-bit because the previous 8-bit cap truncated any AEAD record larger than 255 bytes — TLS 1.3 Certificate ciphertext is ~353 bytes, so every Poly1305 tag check failed. crypto/aead.asm now uses 16-bit counter semantics throughout aead_encrypt/aead_decrypt/aead_compute_tag, the length block stores both bytes, chacha20_encrypt's block loop runs on a 16-bit cc20_remain:cc20_remain_hi, and tls_record.asm copies both bytes of tls_enc_aead_len into aead_data_len.

7eb8b8a — Phase 3 screen markers + net_poll counters + diagnostic instrumentation

Adds on-screen progress markers (CH, SH, HK1, KEYS, ENC1, RX, GOT2, DEC, PROC, EE, CERT, CV, FIN, CFIN) at each TLS handshake state transition in tls_connect, plus 16-bit entry/exit counters on net_poll so the gap net_poll_entry_count - net_poll_return_count directly exposes how many ip65_process frames are parked on the stack. The branch-target trampoline pattern (bcc @okN / jmp @error / @okN:) is used where the inserted print calls would push @error out of bcs range. tests/test_phase3_https.py is extended with dynamic label lookup via _label_addr() against build/labels.txt (so diagnostic reads always hit the current build regardless of data.asm layout shifts), a file-logged post-mortem at /tmp/c64-https-phase3-diag.log with explicit flush that survives hard-kill, per-heartbeat sampling of tcp_recv_head/tcp_recv_tail/net_poll_*, CPU-register reads via transport.read_registers(), and a top-of-stack return-address chain with label resolution. This instrumentation is what localized the remaining stall to ip65, not c64-https (see "What still doesn't work").

Memory-layout fixes (the load-bearing part)

The * = $6000 directive inserted before crypto/word32.asm in src/main.asm is the single most important change in this PR. Before it, the c64-https binary had crypto code mapped linearly from $3B28 upward, which means sha256_rotr8 and its neighbors sat inside ip65's BSS region at $4000-$5FFF. ip65 allocates DHCP/ARP/IP/TCP packet buffers in that window, so the first DHCP request issued from the boot menu wrote packet payload bytes directly on top of sha256_rotr8, which then took a $02 opcode on next entry and CPU-JAMed at $4DE0. Relocating the crypto blob above the BSS ends that. The three adjustments that come with it — sqtab_lo/sqtab_hi moved from hardcoded $7800/$7A00 equates into data.asm BSS labels, P-384 ECDSA stubbed in crypto/ecdsa_verify.asm, and the * = $7C00 barrier removed — are all necessary to fit everything below $7C00 and are tracked in project_p384_stubbed as known tech debt that must be restored before the client can validate real CA chains.

tcp_recv_buf is separately pinned to $C000 as an equate in constants.asm. That region ($C000-$CFFF) is always RAM when the $01 processor port is set to $36 (BASIC ROM out, KERNAL in) as boot.asm does, so it's 4 KB of completely free buffer space that costs zero BSS. The buffer's 16-bit head/tail live in data.asm BSS (tcp_recv_head, tcp_recv_tail, tcp_recv_overflow), so no data.asm label shifts affect the ring's backing store. Widening aead_data_len to a !word in data.asm is the matching fix on the crypto side: without it, the 8-bit counter truncated AEAD decrypt at 255 bytes and the ~353-byte Certificate ciphertext body always failed Poly1305.

TLS receive-path fixes

All landed in d973531:

• src/net.asm — net_tcp_recv_cb had an 8-bit X register wrap when ip65 delivered packets >255 bytes. cb_remaining is now clamped to 255 per invocation so the callback re-enters cleanly on the next slice.
• src/tls_record_io.asm — plaintext/encrypted dispatch was comparing against TLS_STATE_SERVER_HELLO ($02), but tls_state is already $02 when the ServerHello is being received, so the code was trying to AEAD-decrypt a plaintext record. Changed to TLS_STATE_ENCRYPTED_EXT ($03). Also adds a CCS filter: RFC 8446 §5 requires TLS 1.3 clients to ignore ChangeCipherSpec records during handshake, and the record receive loop now retries past them.
• src/tls_keyschedule.asm — tls_derive_handshake_keys fell off its final loop without a clc, leaving carry set from the last hmac_sha256. The caller's bcs @error then fired on success. Added clc before rts.
• src/tls13.asm (tls_recv_server_hello) — tls_ecdh_compute_shared was defined in tls_ecdh.asm but never called anywhere in the codebase, so tls_shared_secret stayed all zeros and HKDF derived handshake keys from zeros. Added the missing jsr tls_ecdh_compute_shared right after tls_parse_server_hello succeeds.

What still doesn't work

The Phase 3 HTTPS end-to-end test (tests/test_phase3_https.py) still fails at runtime, but the failure is inside ip65, not c64-https. The on-screen markers reach CH → SH → HK1 → KEYS → ENC1 → RX cleanly (ClientHello sent, ServerHello parsed, X25519 shared secret computed, HKDF handshake keys derived, entered tls_recv_encrypted, first net_poll call inside the encrypted-wait loop) and then hang. The net_poll_entry_count / net_poll_return_count 16-bit counters show a gap of exactly 1 — one ip65_process call on the stack, never returning — and tcp_recv_tail sits frozen at $0198 (408 bytes delivered) across 55+ heartbeats / 1800 seconds. PC samples during the hang hit different addresses inside ip65 code ($20BE, $9E8D), so ip65 is executing, just caught in an inner loop that doesn't yield — most likely the CS8900a driver or TCP reassembly path choking on a burst of retransmits that arrive after ~3.6 minutes of CPU silence during the X25519 shared-secret computation.

This is tracked separately. Four candidate mitigations are written up in the project_phase3_handoff session memory (ip65 upstream patch, c64-https-side close/reopen workaround, server-side pacing, or warp mode during X25519 to eliminate the long silence). None of them require reverting anything in this PR.

Phase 1 (DHCP) and Phase 2 (HTTP GET) end-to-end tests pass on this branch. The memory-layout fix, the TLS receive-path fixes, and the buffer-widening changes are all independently load-bearing for future progress, and nothing in them depends on the ip65 stall being resolved first.

Testing

• PASS — sudo PYTHONPATH=tools python3 tests/test_phase1_dhcp.py (DHCP end-to-end over bridge + RR-Net)
• PASS — sudo PYTHONPATH=tools python3 tests/test_phase2_http.py (HTTP GET to local listener)
• KNOWN FAILURE — sudo PYTHONPATH=tools python3 tests/test_phase3_https.py hangs after the RX marker inside ip65_process; see "What still doesn't work" above. The failure is preserved at the ip65 boundary by design — all c64-https markers light up cleanly first.
• Unit tests still pass: tools/test_crypto.py, tools/test_sha256.py, tools/test_x25519.py, tools/test_hkdf.py, tools/test_keyschedule_steps.py, tools/test_tls_handshake.py, tools/test_tls_record.py, tools/test_net.py, tools/test_http.py, tools/test_dns.py, tools/test_x509.py, tools/test_entropy.py, tools/test_chained_hmac.py.

Review notes for merger

• tools/https_e2e/ (the bridge test library) is checked in on this branch: __init__.py, env.py, c64_menu.py, http_listener.py, https_listener.py, vice_on_bridge.py. The only other tracked diagnostic scaffold is tools/_diag_carry.py (a small carry-flag probe from the tls_derive_handshake_keys bug hunt).
• Ad-hoc tools/diag_4de0_*.py / tools/diag_read_live.py scripts that appeared during the CPU-JAM post-mortem are not tracked anywhere — they live only in working trees and are not part of this PR.
• project_phase3_handoff.md in the session memory is the primary session-handoff document for whoever picks up the ip65 investigation next. It has the full list of addresses, the four mitigation options, and the gotchas (X25519 timing, $01 processor port and BASIC ROM shadowing, label shifts in data.asm, buffered-stdout pitfalls with tee).
• * = $6000 in src/main.asm is load-bearing. Do not move it.
• P-384 ECDSA is stubbed (crypto/ecdsa_verify.asm returns error). Tracked in project_p384_stubbed; must be restored before the client can validate any real CA chain.

🤖 Generated with Claude Code

---

Originally posted by @JC-000 on 2026-04-14