Stream foreground exec output and retain it only for explicit journals

[taras]

Story

As an xmd run user, I want a foreground executable block to behave like a foreground child process, so long-running commands show progress immediately and XMD does not accumulate output nobody asked it to retain.

Motivation

PR #436 exposed the mismatch through the repository's own README:

./dist/xmd run README.md#Development/Build

The build runs, but the terminal is empty. The proposed README then needs a root-level self-closing <Output /> to obtain fail-fast behavior, silent exec blocks to suppress captured rendering, and a test that requires successful output to be empty. Those are workarounds around the core process contract, not behavior an executable README should teach.

A root <Output> declaration selects rendered document content. It is not a process-status switch. Foreground process I/O, structural capture, and durable retention are three separate concerns.

Product contract

Foreground execution

An ordinary executable block is a foreground child process:

```bash exec
deno task build
```

• stdout and stderr are forwarded progressively through the host's corresponding observability channels;
• the first chunk is observable before the child exits;
• each byte is displayed once, with no captured copy rendered again at completion;
• a nonzero exit fails the executable block, stops later document work, and makes xmd run exit nonzero without requiring <Output>; and
• <Output> continues to govern rendered document selection only.

A document that explicitly handles or prints a process failure may do so through the language's explicit error-handling constructs. A text root's ordinary printed-error mode does not silently turn a checked foreground-command failure into exit 0.

Structural capture

Do not add a capture exec modifier. Existing document structure expresses capture:

<Capture as="buildOutput">

```bash exec
deno task build
```

</Capture>

Inside <Capture as>:

• stdout is collected into the captured rendered value instead of being forwarded live;
• stderr remains diagnostic output;
• capture is scoped to the structural region and may include prose, components, or several executable blocks;
• a nonzero exit still fails rather than becoming a successful binding; and
• streaming and capturing simultaneously are not implied. A future tee construct requires its own product case.

silent exec remains the explicit way to display neither stdout nor stderr. Silence does not convert nonzero exit into success.

Retention belongs to the host

Live routing and retention are independent:

Invocation/context	Live routing	Retained process result
xmd run, ordinary exec	stdout/stderr stream	exit status only
xmd run, <Capture as>	stdout captured; stderr streams	exit status only; captured stdout lives only as long as its binding requires
xmd run, silent exec	neither stream	exit status only
xmd run --journal <path>	same document-selected live routing	exit status, stdout, and stderr

Without --journal, XMD must not build complete stdout/stderr strings merely because execution internally uses a transient event stream. The only exception is output a document explicitly needs in memory for a structural capture, and that buffer has the capture's scope.

With --journal, the host explicitly requests a diagnostic record. Process stdout and stderr are teed into the journal in addition to whatever live routing the document selected. This includes silent and structurally captured blocks: display policy does not weaken an explicit audit record.

Core receives an explicit host retention choice. It does not infer persistence by checking a concrete stream implementation. Existing programmatic callers that supply a durable journal retain their current full process-result behavior unless they explicitly select transient execution.

Secret detection

Secret detection continues to guard persistent journal content:

• with --journal, retained stdout and stderr cross the existing pre-persistence secret gate;
• a finding leaves the offending process-result event absent and follows the existing journal-policy failure behavior;
• --journal --no-secret-detection remains the explicit warned opt-out; and
• without --journal, child bytes do not enter a retained event or get accumulated merely so a persistence scanner can inspect data that will not be persisted.

This changes no scanning rule for other event fields.

Host channel compatibility

• A text root's foreground stdout and stderr keep their corresponding host channels.
• A value root's stdout remains reserved for its successful JSON result; foreground process observability must not corrupt that contract.
• Piped and interactive executions both receive progressive foreground output.
• Deno, Node, Bun, and the compiled binary expose the same behavior.

Architecture boundary

The transient path requires a genuinely streaming process operation. It cannot call a convenience helper that unconditionally collects stdout and stderr and discard those strings afterwards.

Process acquisition, forwarding, completion, cancellation, and teardown remain owned by the executable block's Effection scope. Cancellation stops the child and its forwarders before the block settles; no output arrives after teardown.

Default exec, silent exec, structural capture, and journal retention are visible in document structure or host invocation. No root-level presentation declaration, runtime detection in shared code, or README-specific behavior selects them.

Keep the existing journal result readable. A journaled exec continues to carry exit status, stdout, and stderr. A transient status-only execution creates no persistence format that a later process can replay.

Acceptance criteria

• A deterministic foreground-process test observes an early stdout chunk before releasing the child to exit.
• stdout and stderr preserve their channels and appear exactly once.
• A large no-journal command does not accumulate output proportional to its total stdout/stderr.
• --journal streams output live and records exact stdout, stderr, and exit status.
• Secret-shaped child output succeeds without a journal, while the same output with --journal is refused before the offending event persists.
• <Capture as> captures stdout without live duplication, leaves stderr diagnostic, and reconstructs the binding correctly during the live run.
• silent exec displays nothing; with --journal, its output is nevertheless retained.
• Nonzero exit stops later executable blocks and produces nonzero CLI status without any <Output> declaration.
• Value-root stdout remains JSON-only.
• Cancellation tears down the child and forwarding tasks and emits no later bytes.
• Existing journaled exec records remain readable.
• Focused coverage exercises the shared contract under Deno, Node, and Bun, plus the compiled CLI surface that exposed the problem.
• architecture.md and the executable-MDX specification describe the new process, capture, error, and retention boundaries in the same PR.

Scope

Included:

• ordinary host-process execution under xmd run;
• stdout/stderr routing, structural capture, silence, and host-selected retention;
• checked nonzero-exit behavior independent of <Output>;
• CLI and programmatic retention selection;
• secret-gate interaction for process output; and
• specification, architecture, runtime, core, CLI, and discriminating test changes required by that contract.

Excluded:

• README target implementation from Dogfood hierarchical README targets in developer tooling #414 / PR 📝 Make the README the development guide, and run it #436, which consumes this contract after it lands;
• a process-specific capture modifier;
• a tee construct;
• workflow Workspace Worker Shell behavior from Run shell blocks in xmd workflows #363;
• changing <Output> rendering semantics;
• changing journal file reuse or workflow-resume behavior; and
• unrelated repair of the upstream process acquisition issue in exec can orphan the child process tree when halted during acquisition #423.

Dependencies and consumers

• PR 📝 Make the README the development guide, and run it #436 remains draft and is blocked on this story. After this lands, 📝 Make the README the development guide, and run it #436 removes the root <Output />, uses ordinary exec blocks, removes output-discard workarounds, and tests progressive output plus fail-fast status.
• Issue Dogfood hierarchical README targets in developer tooling #414 remains the README dogfood story and consumes this core contract.
• @effectionx/process has no exit-settled join — runtime exec spawns directly #345 records the existing exit-settled process seam.
• exec can orphan the child process tree when halted during acquisition #423 remains the separate upstream acquisition/teardown defect.
• Run shell blocks in xmd workflows #363 owns workflow Workspace shell execution and is unchanged.

[taras]

Accepted limitation: pump-tail delivery (effectionx #244)

The acceptance criterion above says --journal "records exact stdout, stderr, and exit status". That is bounded by a limitation approved for #442, recorded here so the criterion is not read as stronger than what shipped.

• The per-exec Stdio boundary is installed before process acquisition, so a chunk forwarded while the child is starting is received like any other.
• What reaches that boundary is routed and retained exactly as the contract states — "exact output" means exactly the bytes received there, never what the child wrote to its pipe.
• Until effectionx #244 is integrated, Process.join() may settle before the stdout/stderr pumps and their Stdio middleware finish. Output written as the pumps settle may therefore fail to reach the boundary before the retained or captured result is snapshotted.

#442 proceeds with that limitation and claims no pump-complete tail delivery or retention. architecture.md, specs/executable-mdx-spec.md §3.6, packages/core/src/foreground.ts and packages/runtime/apis.ts all state it.