feat(crypto): c64-x25519 v0.4.0 optional sibling integration (Phase C.5) - #41
Merged
Conversation
Adds `libs/x25519` submodule at v0.4.0 and a `USE_X25519_SIBLING=1`
Makefile flag that links against the sibling X25519 implementation
instead of the in-tree `src/crypto/{fe25519,x25519}.s`. Default is
OFF; nothing changes for `make` / `make BACKEND=uci` and the shipped
PRG is byte-identical to the pre-branch baseline. Flag is opt-in
pending the supervisor's decision to flip the default.
## Why now
Two prior integration attempts (Phase C.1 commit 6c9d2a3 against
v0.2.0-era source, and a v0.3.0 retry) rolled back due to wrong-
result symptoms under BACKEND=uci at 48 MHz. Trace analysis later
showed the symptom was NOT a Montgomery-ladder hang but a wrong-
output cascade caused by REU register residue ($DF04, $DF0A) left
by c64-https's `do_swap` dispatcher before the sibling's first
`fe25519_mul`. Upstream issue c64-x25519#33 closed this with PR #36
(defensive REU register init at `x25519_scalarmult` entry), extended
in v0.4.0 H2 to every `fe25519_{mul,sqr,mul_a24,inv}` entry point.
PR #36's adversarial harness explicitly simulates c64-https `do_swap`
residue and passes 4/4 on real U64E hardware. The v0.4.0 release is
strictly safer than v0.3.0 for our composition.
## What's in this commit
- `libs/x25519@v0.4.0` submodule (commit 47c0ad2)
- `Makefile`: `USE_X25519_SIBLING ?= 0` flag. When `=1`, drops
`src/crypto/fe25519.o` + `src/crypto/x25519.o` from the link line
and links `build/lib/x25519.a` instead.
- `tools/integration/build_x25519.sh`: stages sibling .s files,
emits two custom data files (BSS for zero-init buffers, RODATA
for x25_basepoint + fe_p + mul_by_38 / sqr / a24 LUTs), assembles
to `build/lib/x25519.a`. Critical detail: x25_basepoint + fe_p
MUST live in a `type = ro` segment, not `type = bss` — ld65 drops
init bytes from bss and the constants will be zero at runtime.
- `cfg/c64-https-uci.cfg`, `cfg/c64-https-ip65.cfg`: new
`X25519_RODATA` (type = ro) and `X25519_BSS` (type = bss)
segments routing to `CRYPTO_OVERLAY`. ip65 cfg has the segments
declared but the backend overflows `CRYPTO_RESIDENT` by ~1 KB
under the flag — UCI is the only supported flagged path until
a separate cfg restructure lands.
- `src/boot.s`: `.ifdef USE_X25519_SIBLING` guards. Under the flag,
in-tree `reu_mul_init` / `reu_fetch_mul_row` / `reu_init_a/b`
state are dropped; sibling's `x25519_init.s` provides them.
In-tree `sqtab_init` + `mul_8x8` (from src/crypto/poly1305.s) are
retained — calling-convention compatible with the sibling.
- `src/data.s`: in-tree `fe_*` / `x25_*` buffers dropped under the
flag (sibling exports its own from `data_x25519_bss_raw.s`).
- `src/crypto/shared/reu_layout.inc`: comment note about overlap
between REU_P{256,384}_PRECOMPUTE_BASE (banks 3-5, reserved-
but-unused) and the sibling's per-row multiply tables.
- `tools/uci/test_https_local.py`: **load-bearing fix**. The
hardcoded `ROUTINE_ADDR = 0x4200` (+ 5 sibling scratch addresses
through $4542) DMA-injected a 6502 trampoline + harness data
directly on top of the new X25519_RODATA segment ($4200-$4AFF
under the flag), silently clobbering x25_basepoint, fe_p, and
the multiply / square lookup tables. Every fe25519_mul/sqr then
read garbage and X25519 emitted wrong-but-deterministic output;
the symptom surfaced 2 layers downstream as `tls_state=$FF` /
AEAD decrypt fail and cost a 12-hour bisection. Relocated the
trampoline + scratch to $5100-$5442 (past the X25519_BSS tail at
$50FF, still inside CRYPTO_OVERLAY). Comment block at the
declaration explains the constraint for future maintainers. An
upstream issue (JC-000/c64-test-harness#93) + draft PR #94
propose a `MemoryArbiter` API to surface this class of collision
at harness level rather than via cryptographic mystery.
- `CLAUDE.md`: updated public-symbol section, Known issues entry,
and CRYPTO_OVERLAY collision lesson.
## Validation
- Default builds (USE_X25519_SIBLING=0): PRG md5 byte-identical to
be57811 baseline, both BACKEND=ip65 and BACKEND=uci. Zero impact
on the shipped binary.
- Default-build regression: `tools/test_x509.py` 11/11 PASS,
`tools/test_entropy.py` 7/7 PASS, U64E HTTPS e2e PASS at 48 MHz.
- Sibling build (USE_X25519_SIBLING=1) BACKEND=uci:
- U64E HTTPS e2e at TURBO_MHZ=48: PASS, http_status=200, body =
"HELLO FROM TLS SERVER", handshake wall-clock 101.4 s (vs the
87 s in-tree baseline; +14 s is consistent with v0.4.0's
documented +27 % scalarmult cost from the L1-L29 CT closures
and PR #36 defensive REU init).
- U64E HTTPS e2e at TURBO_MHZ=1: PASS, body = "HELLO FROM TLS
SERVER", handshake wall-clock 1280.5 s (~21 min) — stock-CPU
baseline.
- Sibling build BACKEND=ip65: overflows CRYPTO_RESIDENT by ~1 KB —
documented blocker, follow-up cfg work.
## Known follow-ups (not in this commit)
- Flip the default to `USE_X25519_SIBLING=1` after a reviewing pass
on the cleanliness of the build script + a second e2e validation
pass.
- Restructure ip65 cfg to make room for the sibling (~1 KB short).
- Sweep `tools/uci/` for other `ROUTINE_ADDR=$4200`-class addresses
(`bench_ecdsa_u64e.py:80` spotted; doesn't exercise X25519 today
so unblocked, should be moved before any default flip).
- `tools/test_x25519.py` references in-tree `fe_*` label names; a
shim or rewrite is needed before that test can validate the
sibling implementation directly. Today the U64E HTTPS e2e is the
load-bearing validation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>Replaces hardcoded scratch DMA addresses across five tools/uci/ scripts with arbiter-driven allocation backed by a c64-https-aware MemoryPolicy factory. Builds on c64-test-harness PR #95 (released 2026-05-14) which adds a transport-level write guard: every `transport.write_memory()` call now passes through `policy.check_write()` before any byte crosses the wire, raising `MemoryPolicyError` on a collision. ## Why this exists The Phase C.5 X25519 integration in this branch surfaced a silent memory-corruption class: `tools/uci/test_https_local.py` had `ROUTINE_ADDR = 0x4200` hardcoded; under `USE_X25519_SIBLING=1` that sat inside `X25519_RODATA`, every DMA-inject quietly clobbered `x25_basepoint` + `fe_p` + multiply/sqr LUTs, X25519 emitted wrong- but-deterministic output, and the failure surfaced two layers down the TLS state machine as `tls_state=$FF`. The PR #41 prior commit relocated the constants to `$5100-$5442` as a defensive hack, but the antipattern (hardcoded harness scratch in a moving consumer memory layout) is the actual class of bug. The new MemoryPolicy / MemoryArbiter API addresses it at the source. ## What changed New `tools/uci/_memory_policy.py` (295 LOC) — shared factory: - `build_policy_and_arbiter(labels_path, prg_path)` parses ld65's `__<SEGMENT>_START__` / `__<SEGMENT>_LAST__` / `__<SEGMENT>_SIZE__` markers from `build/labels.txt` to derive a MemoryPolicy reflecting the *current* build's memory map (BACKEND ip65/uci, with/without USE_X25519_SIBLING). Reserves the *declared* area of each segment (conservative — runtime tails are protected against silent growth). - Returns a paired MemoryArbiter ready to `.alloc(size, name=...)` inside the CRYPTO_OVERLAY tail (or wherever the layout has slack). - `unknown_policy = WARN` — stray writes outside declared regions surface as UserWarning, not failure. Tighten to DENY in a follow-up once the existing call sites (notably `c64_test_harness.keyboard` KERNAL kbd-buf writes) are declared safe. Five tools/uci/ scripts migrated (28 hardcoded addresses replaced): - `test_https_local.py` (6 addrs) - `bench_ecdsa_u64e.py` (5 addrs) - `test_http_local.py` (6 addrs) - `test_http_live.py` (6 addrs) - `phase3_tcp_echo.py` (11 addrs incl. RECV_BYTES alias) Each script: module-scope constants become `-1` sentinels populated in `main()` after labels load; `transport.memory_policy = policy` attaches the policy right after `Ultimate64Transport(...)` construction; the arbiter's per-allocation list is printed to test output for debugging. ## Validation Real U64E hardware @ 10.43.23.81, 48 MHz turbo, both build configs: - `make BACKEND=uci USE_X25519_SIBLING=1`: HTTPS PASS, 98.3 s wall- clock, http_status=200, body="HELLO FROM TLS SERVER". Trampoline allocated at $5100 (matches the prior defensive hack location). - `make BACKEND=uci` (default, in-tree X25519): HTTPS PASS, 86.7 s wallclock, http_status=200, body OK. Trampoline at $4200 — the historical hardcoded value, now arbiter-derived because $4200 is genuinely free in the default build. - First-attempt firmware flakiness (UCI_ERR_NO_SOCKET on cold socket) observed on BOTH builds — orthogonal to migration; the second attempt passes with zero code change. Pattern is identical across configurations. - Surfaced UserWarnings (expected, signal-bearing): 8 per run — 2 from `c64_test_harness.keyboard.send_text` writes to KERNAL kbd-buf state ($00C0-$00CF, $0277-$0280), 6 from the arbiter's own allocations because the policy doesn't auto-add arbiter claims as safe regions. Both are follow-up scope. ## Follow-ups (not in this commit) - Promote arbiter allocations to `safe_regions` via `arbiter.policy_with_allocations()` (silences 6 of 8 warnings). - Declare KERNAL kbd-buf state as safe (probably belongs in the harness's `c64_test_harness.keyboard` rather than per-test). - Tighten `unknown_policy = WARN` → `DENY` once the above lands. - Migrate the VICE-side `tools/test_*.py` similarly when convenient — the same factory works for ip65 builds (segment-marker driven). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 task
JC-000 added a commit
that referenced
this pull request
May 15, 2026
Two entries described work that has now landed on master: 1. The MEMORY requirements paragraph claimed "the sibling overlay integration in Phase C.1 was rolled back — see Known issues." That's historical; Phase C.5 (PR #41) landed a working flag-gated sibling integration. Replaced with the current dual-state description (default build leaves banks 0-1 free; sibling build populates banks 0-5). 2. The "CRYPTO_OVERLAY address collision lesson" entry described relocating ROUTINE_ADDR to $5100 as the fix. That defensive hack was superseded by the MemoryPolicy + MemoryArbiter adoption in PR #41's second commit (tools/uci/_memory_policy.py). Rewrote the entry to point at the now-active safety mechanism: every tools/uci/* script derives its scratch addresses from the arbiter, and the c64-test-harness PR #95 transport-layer write guard catches any future collision before a byte crosses the wire. No behaviour change — documentation only. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
libs/x25519@v0.4.0as an optional sibling library and aUSE_X25519_SIBLING=1Makefile flag. Default is OFF — the shipped PRG is byte-identical to master atbe57811. Flag is opt-in pending a supervisor decision to flip the default.Closes the two prior Phase C.1 / v0.3.0 rollback dead-ends. Root cause was upstream
c64-x25519#33— REU register residue from c64-https'sdo_swapclobbering$DF04/$DF0Abefore the sibling's firstfe25519_mul. Upstream PR #36 + v0.4.0 H2 add defensive REU register init at every publicfe25519_*/x25519_scalarmultentry. PR #36's adversarialreu_full_dirtyharness explicitly simulates ourdo_swapresidue and passes 4/4 on real U64E.Second commit (added 2026-05-14): MemoryPolicy adoption. Replaces the defensive
ROUTINE_ADDR=$5100hack intools/uci/test_https_local.py(and 4 sibling scripts) withMemoryArbiter-driven allocation backed by a c64-https-awareMemoryPolicy. Builds on the just-merged c64-test-harness PR #95 (transport-layer write guard). The arbiter readsbuild/labels.txtfor segment boundaries — same scratch addresses derived dynamically per build config (BACKEND × USE_X25519_SIBLING), no more hardcoded constants for future layouts to collide with.Validation
BACKEND=ip65andBACKEND=uci.tools/test_x509.py11/11,tools/test_entropy.py7/7. U64E HTTPS e2e green at 48 MHz (86.7 s).USE_X25519_SIBLING=1BACKEND=uci):tools/uci/test_https_local.pyatTURBO_MHZ=48: PASS, handshake 98.3 s (post-MemoryPolicy commit; was 101.4 s pre-adopt — variance within run-to-run noise).tools/uci/test_https_local.pyatTURBO_MHZ=1: PASS, handshake 1280.5 s (~21 min stock-CPU).HELLO FROM TLS SERVER.BACKEND=ip65: overflowsCRYPTO_RESIDENTby ~1 KB — documented limitation, separate cfg-restructure follow-up.Investigation highlights
Two consecutive bugs in the prototype were caught before this lands:
x25_basepoint+fe_pinitially routed toX25519_BSS(type = bss). ld65 silently drops init bytes from bss segments; both constants were zero at runtime. Symptom: everyfe25519modular reduction sawfe_p=0, everyx25519_basesawbasepoint=0. Fix: route toX25519_RODATA(type = ro).tools/uci/test_https_local.pyhad a hardcodedROUTINE_ADDR = 0x4200that DMA-injected a 117-byte 6502 trampoline directly on top of the new X25519_RODATA segment. Silently clobberedx25_basepoint,fe_p,mul38_*_tab,sqr_*— everyfe25519_mul/sqrthen read garbage tables, X25519 emitted wrong-but-deterministic output, surfaced 2 layers downstream astls_state=$FFAEAD decrypt fail. 12-hour bisection. This PR's second commit replaces the defensive$5100relocation with properMemoryArbiter-driven allocation, so future layout changes can't recur this class of bug silently.The harness-level fix is now upstream in c64-test-harness#93 / #95 (merged). c64-https adopts that API in commit 2 here.
Test plan
USE_X25519_SIBLING=1 BACKEND=uci: builds clean, zero ld65 warningsUSE_X25519_SIBLING=1(out of scope for this PR)safe_regions(silences 6 of 8 expected UserWarnings; follow-up)keyboardmodule (follow-up)unknown_policy=WARN→DENYonce the above two land (follow-up)tools/test_x25519.pyshim/rewrite to validate sibling primitives directly under VICE (follow-up)Commits
feat(crypto): c64-x25519 v0.4.0 optional sibling integration (Phase C.5)— submodule, build script, Makefile flag, cfg, src wiring, CLAUDE.md, defensive$5100ROUTINE_ADDR relocation.chore(tests): adopt MemoryPolicy + MemoryArbiter for tools/uci/ scratch— replaces commit 1's defensive$5100hack with proper transport-layer policy-driven allocation; sweeps 5 scripts (28 hardcoded addresses → 0).🤖 Generated with Claude Code