feat: TDD refactoring workflows for runner.ts + main.rs decomposition

[khaliqgant]

Summary

TDD refactoring workflows for the two largest files in the codebase, executed end-to-end with full verification.

Targets

• runner.ts (6,878 lines) — decomposed into 5 focused modules
• main.rs (7,023 lines) — decomposition planned, broker/worker extraction attempted

E2E Results (full run with refactored code)

Wave	Workflow	Steps	Status
1	runner-decomposition-plan	7/7	✅
1	main-rs-decomposition-plan	6/6	✅
2	extract-verification	6/6	✅
2	extract-template-channel	6/6	✅
2	extract-broker-worker	❌	See below
3	extract-step-executor	8/8	✅

extract-broker-worker failure details

This workflow extracts BrokerState and WorkerRegistry from main.rs (Rust). It failed on both attempts:

Attempt 1 (6 passed, 1 failed, 2 skipped):

• All agent steps succeeded — Codex extracted broker.rs and worker.rs
• cargo-gate (deterministic cargo test verification) failed with exit code 1 — the extracted Rust modules broke compilation
• cargo-review and downstream steps skipped

Attempt 2 (1 passed, 2 failed, 6 skipped):

• read-broker and read-worker failed — the deterministic steps tried to read the extracted files but they were in a broken state from attempt 1
• All downstream steps skipped

Root cause: Codex's Rust extraction produced code that didn't compile. This is an agent quality issue (the Codex worker made incorrect edits to the Rust source), not a workflow runner or infrastructure issue. The runner correctly detected the failure via the cargo-gate verification step and reported it.

Not a regression — no runner code paths were affected. The workflow SDK correctly:

1. Spawned agents ✅
2. Collected output ✅
3. Ran verification gates ✅
4. Detected the failure ✅
5. Retried the wave ✅
6. Reported final status ✅

Extracted Modules

Module	Lines	Tests
verification.ts	143	11 tests
template-resolver.ts	87	tests included
channel-messenger.ts	151	tests included
step-executor.ts	571	27+ characterization tests
process-spawner.ts	96	—

Test Results

• SDK workflow tests: 88/88 pass ✅
• Full repo tests: 672/672 pass ✅
• TypeScript build: clean ✅
• Rust tests: pass ✅

Bug Fixes (found during E2E)

• stripInjectedTaskEcho missing in extracted verification.ts — Codex extraction lost the task-echo-stripping logic that prevents false positive verification. Restored.
• agent.release() broker 400 race condition — broker returns HTTP 400 when releasing an already-exited agent. Was causing steps to fail even when the agent completed successfully. Added .catch() guards.
• Legacy summary test assertion — pre-existing test expected old format the code no longer produces. Updated.
• ESM imports — all 6 workflow files used require() in an ESM project. Fixed to import.
• Executor exit code — run-refactor.ts didn't exit non-zero on single-wave failures. Fixed.

Tooling

• run-continuous.sh — runs all waves sequentially, retries up to 3x per wave, auto-commits
• Rebuilt broker binary from main (old binary lacked HTTP API server)

[barryonthecape]

PR #675 Review — Strong Work, One Blocking Issue

What's Impressive

• Decomposition of runner.ts (6,878 lines → 5 focused modules) with full test coverage across all extracted units
• 672/672 full repo tests pass, 88/88 SDK workflow tests pass, TypeScript clean, Rust tests pass
• Honest failure reporting — the extract-broker-worker wave failed cleanly because Codex's Rust extraction broke compilation, and the workflow runner correctly caught it and retried
• Good bug fixes caught during E2E: stripInjectedTaskEcho restoration, broker 400 race condition guards, ESM require() → import fixes

---

Blocking: CodeQL ReDoS in channel-messenger.ts (line 87)

Severity: High — ReDoS vulnerability in secret scrubbing

The problematic pattern:

/-----BEGIN\s+(?:RSA\s+)?PRIVATE\s+KEY-----[\s\S]*?-----END/g

The non-greedy [\s\S]*? still causes polynomial backtracking on adversarial input (e.g., many repetitions of -----BEGIN\tPRIVATE\tKEY----- with no closing delimiter).

Suggested fix — use a tempered greedy token:

/-----BEGIN\s+(?:RSA\s+)?PRIVATE\s+KEY-----(?:(?!-----END)[\s\S])*-----END/g

This prevents backtracking by checking ahead at each position. Alternatively, since scrubForChannel already iterates line-by-line, you could use indexOf/slice for the private key block stripping instead of regex entirely.

---

Non-Blocking: Devin Reviews

Multiple Devin Review passes are showing findings (5 + 14 + 17 + 18 + 26 + etc. across passes). The PR body doesn't address whether these are resolved. Worth confirming all findings are addressed before merge.

---

Merge Readiness

Check	Status
CI (excluding CodeQL)	All passing
Tests	672/672
TypeScript	Clean
Rust clippy/build	Passing
CodeQL	1 high (ReDoS)

One fix needed before merge — the ReDoS in scrubSecrets. Once that's addressed, this is a solid approve. Massive refactor executed well.