Uh oh!
There was an error while loading. Please reload this page.
fix: chat→solve QA — permission-hang blocker + multi-qubit/regime guidance - #22
Conversation
Rchari1
commented
Jun 20, 2026
…rectory hang)
The instruction-injection fix made the agent read the bundled template at an
absolute path outside the session cwd + write /tmp/amicode-work scratch.
opencode defaults external_directory to 'ask' → the turn hangs forever with no
interactive approver (headless) and nags on every solve (GUI). buildOpencodeConfigContent
now injects permission:{bash,edit,webfetch,external_directory: allow} so the
controlled solve workflow proceeds without prompts.
Found via headless chat QA (qa-20260620): baseline X-gate chat stalled on the
external_directory ask; with the permission block the agent reads the template,
mkdirs scratch, authors solve.jl, and launches amico-run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>…2/F3/F4) QA (qa-20260620) found the agent over-reaches on out-of-scope requests: - F4: a 2-qubit (CNOT) request → the agent HALLUCINATES a multi-qubit API (TransmonSystem(ω=[..],g=..)) → TypeError crash, while claiming 'solve launched'. - F3: a 30 ns gate with the default N=50 is under-resolved → F=0.914 reported as success. - F2: levels=5 stalls (integrator stiffness) at f~69. AGENTS.md now: declares the build single-qubit only and tells the agent to decline multi-qubit gates instead of authoring a crashing script; and gives regime guidance (avoid 5+ levels; keep ~5-10 timesteps/ns so longer gates scale N up; bump max_iter for harder cases). Guard tests added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
b39dcd4 to
9b287adComparebb34e3d to
b6bd96dComparejack-champagne
commented
Jun 26, 2026
Correct root-cause:
Note this doesn't make chat→solve work end-to-end on its own — the #11 author/run path mismatch was still live here and is reconciled in #23 (now validated). |
jack-champagne
left a comment
There was a problem hiding this comment.
The external_directory fix is correctly diagnosed and lands the blocker. Three changes (inline): drop the unused webfetch allow; scope external_directory to the two roots the agent actually touches (or document the trust boundary if the schema won't scope); and verify the nested permission block deep-merges rather than replacing the user's permission config (+ a test).
bash/edit are redundant-as-defaults but harmless — leave them; scoping bash is fiddly given the compound mkdir … && nohup amico-run … launch.
AGENTS.md guidance is solid and the N / steps-per-ns numbers are consistent — two prose fixes: "do not hand-roll a coupled multi-transmon system (it will not construct)" — MultiTransmonSystem does exist and would construct; say "out of scope for this single-lab build." And "the integrator stiffens" for levels ≥5 is loose — it's conditioning/leakage/cost.
Tests are content/string guards — they assert the keys/words are emitted, not that opencode honors them, and nothing covers the permission merge.
| permission: { | ||
| bash: "allow", | ||
| edit: "allow", | ||
| webfetch: "allow", |
There was a problem hiding this comment.
[minor] webfetch isn't used by the solve workflow (it never fetches a URL) and already defaults to allow, so this line only forces egress back on for a user who'd restricted it globally. The new test omits it too. Drop it:
| webfetch: "allow", |
There was a problem hiding this comment.
Dropped in 4942a2b — webfetch is gone (and edit/bash left as you suggested).
| bash: "allow", | ||
| edit: "allow", | ||
| webfetch: "allow", | ||
| external_directory: "allow", |
There was a problem hiding this comment.
[important] This is the line that fixes the hang — correct, external_directory defaults to "ask" in 1.17.x. But "allow" grants the agent's file tools the entire filesystem outside the session cwd. The agent itself needs only two roots: it reads the bundled template (opts.templateSrc) and writes scratch to /tmp/amicode-work (amico-run's writes to ~/.amico/runs / ~/.amico/julia are the subprocess's, not the agent's). If opencode 1.17.3 supports path-scoped external_directory, scope it:
external_directory: {
"/tmp/amicode-work/**": "allow",
"<extension templates dir>/**": "allow",
},
If it only takes a flat string, keep "allow" but document the trust boundary.
There was a problem hiding this comment.
Scoped in 4942a2b. external_directory is now an object keyed to the two roots the agent's file tools touch — the bundled templates dir (path.dirname(templatePath) + the exact template file) and /tmp/amicode-work (plus /private/tmp/amicode-work for macOS's /tmp→/private/tmp symlink). amico-run's own ~/.amico writes are the subprocess's, not the agent's, so they're not granted. The path-scoped object form is accepted by 1.17.3 — verified via opencode debug config (resolved config preserves the per-path keys). Threading templatePath through buildOpencodeConfigContent so the dir is computed at runtime (correct for both dev and the installed VSIX).
| return JSON.stringify({ | ||
| $schema: "https://opencode.ai/config.json", | ||
| instructions: [agentsPath], | ||
| permission: { |
There was a problem hiding this comment.
[important] Verify the merge for this nested object. The doc-comment says opencode merges the injected config over the user's — but that was only confirmed for top-level model/provider and the instructions array (#12). If opencode shallow-replaces rather than deep-merges, injecting permission here wipes any other permission keys the user set globally (doom_loop, per-tool patterns). Nothing tests this — confirm against 1.17.3 and add a test that the user's other permission keys survive.
There was a problem hiding this comment.
Verified — opencode 1.17.3 deep-merges the nested permission object; it does not shallow-replace. Tested with opencode debug config: a global permission.doom_loop:"deny" survives alongside our injected external_directory keys. Added an integration test (opencode_config.test.ts → 'opencode permission merge (1.17.3)') that runs the vendored binary's debug config with a global config + our injected content and asserts the user's key survives; skipIf the binary is absent. So injecting permission adds keys without clobbering the user's other permission settings.
…drop webfetch, verify deep-merge - Scope `external_directory` (least privilege) to the two roots the agent's file tools actually touch: the bundled templates dir (reads the solve template) and /tmp/amicode-work (writes solve.jl). Path-scoped object form verified accepted by opencode 1.17.3 via `opencode debug config`. Includes /private/tmp/** for macOS's /tmp symlink. amico-run's own ~/.amico writes are the subprocess's, not the agent's, so they need no grant. - Drop `webfetch` (unused by the solve flow; already defaults to allow). Keep bash/edit per Jack (redundant-as-defaults but harmless; bash launch not worth scoping). - Verify the permission merge: opencode 1.17.3 DEEP-merges the injected `permission` object over the user's global config (a global permission.doom_loop survives). Added an integration test (runs the vendored binary's `debug config`, skipped when absent) so this can't regress to a clobber. - AGENTS.md prose fixes: multi-qubit is "out of scope for this single-lab build" (MultiTransmonSystem does exist/construct), and levels≥5 degrades via conditioning/leakage/cost, not "integrator stiffens". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rchari1
commented
Jun 26, 2026
Thanks @jack-champagne — all addressed in 4942a2b:
44 tests green (incl. the new merge test). One note: the runtime enforcement of the scoped globs (vs schema-acceptance, which is verified) I'll smoke-test on the dev host with a live solve before this merges — if a path canonicalization edge re-introduces the prompt, the documented- |
jack-champagne
left a comment
There was a problem hiding this comment.
All three addressed in 4942a2b — verified against the diff:
- webfetch dropped; bash/edit left at allow as agreed.
- external_directory scoped to the template file + its dir + /tmp/amicode-work (and /private/tmp for the macOS symlink). Blanket allow gone, amico-run's own ~/.amico writes correctly ungranted.
- deep-merge confirmed empirically via
opencode debug config+ integration test.
One follow-up, not blocking: the merge test is describe.skipIf(!existsSync(OC_BIN)) — it silently skips when the vendored binary is absent, so in CI before fetch:opencode it's green-without-verifying (same false-green class as #25). Sequence that test job behind fetch:opencode or the guard is decorative.
Good to merge.
Uh oh!
There was an error while loading. Please reload this page.
Findings + fixes from an autonomous QA campaign on the chat→solve path (drove the real opencode backend headlessly with a matrix of user requests; full notes in vault
qa-20260620-amicode-chat-solves.md). Stacked on #14; baserchari/template-plot-pulse. Two real bugs fixed, two limitations mitigated by guidance.F1 (BLOCKER) — chat hung on a permission prompt for every solve
The instruction-injection fix (#12) made the agent read the bundled template at an absolute path outside the session dir + write
/tmp/amicode-work. opencode defaultsexternal_directoryto ask, so with no interactive approver the turn hangs forever (headless) and nags on every solve (GUI) — the "trouble running gates" symptom returning. It only worked before because the GUI popup was clicked (always-remembered).Fix:
buildOpencodeConfigContentinjectspermission: { bash, edit, webfetch, external_directory: allow }.Verified: baseline X-gate chat → authors
solve.jl→ launchesamico-run→ F=0.99994.F4 — multi-qubit requests: agent hallucinated an API and crashed
A CNOT request → the agent didn't decline; it hand-rolled
TransmonSystem(ω=[ω1,ω2], g=…)(a multi-qubit API that doesn't exist) →TypeError: keyword ω expected Float64, got Vector{Float64}, while claiming "solve launched" with a tidy params table.Fix: AGENTS.md now scopes the build to single-qubit only and tells the agent to decline multi-qubit gates.
Verified: re-asking for a CNOT now returns "…not supported in this build. TransmonSystem models a single qubit only… If you have a single-qubit gate (X, H, T, …) I can help" — no crash, no solve launched.
F3 — long gates were silently under-resolved
T=30 nswith the defaultN=50(0.6 ns/step) → objective plateaus → F=0.914 reported as success.Fix: AGENTS.md guidance to keep ~5–10 timesteps/ns and scale
Nwith gate time.Verified: asking for a 30 ns gate now → agent authors N=200 (~6.7 steps/ns).
F2 — levels ≥5 is init/iteration-sensitive (soft)
levels=5stalled (f≈69) on one seed with the defaultmax_iter=60, but converged to F=0.99999 when the agent bumpedmax_iter=200. Not broken — the template default is too low for 5 levels, and the agent already compensates. AGENTS.md adds regime guidance (prefer 3–4 levels; warn on 5+). Deeper physics fixes (energy-shift / DRAG warm-start / fixed seed) deferred.What's already good (no change needed)
Correct gate translation (H/Y/Z), sane defaults on underspecified prompts, and genuinely smart parameter adaptation — fast 4 ns gate → raised N + drive_max + max_iter (F≈1.0); 5 levels → raised max_iter (F=0.99999). Template gates X/Y/Z/H + tight drive bound all converge F > 0.999.
Tests
40 extension tests green; new guards on the permission block + the single-qubit scoping + regime guidance.
🤖 Generated with Claude Code