Skip to content

fix(sandbox): complete successful main processes - #2884

Open
drew wants to merge 3 commits into
mainfrom
codex/2879-completed-main-process
Open

fix(sandbox): complete successful main processes#2884
drew wants to merge 3 commits into
mainfrom
codex/2879-completed-main-process

Conversation

@drew

@drewdrew commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Restore ephemeral canonical-command semantics for sandbox create. The canonical main process controls the sandbox lifecycle, and foreground create streams output and returns the command exit code.

Related Issue

Closes#2879

Changes

  • Add the public Completed sandbox phase across protobuf, SDK, CLI, and TUI surfaces.
  • Map exit code 0 to Completed/MainProcessCompleted.
  • Map nonzero and signal-normalized exits to Error/MainProcessFailed, preserving the exact exit code. For example, SIGINT is reported as 130.
  • Keep explicit sandbox stop as Stopped/SandboxStopped.
  • Distinguish infrastructure errors from command failures by condition reason and the absence of a main-process exit result.
  • Keep the supervisor relay available long enough to attach to fast commands and drain retained output.
  • Send the SSH exit result only after the gateway durably records the terminal phase and exit code.
  • Make explicit create commands foreground by default; --detach remains available for long-running workloads.
  • Delete --no-keep sandboxes only after stdout, stderr, and the command result have drained.
  • Preserve restart and stale-instance fencing for retained terminal sandboxes.
  • Update lifecycle tests, architecture documentation, user documentation, and the OpenShell CLI skill.

Testing

  • mise run pre-commit
  • cargo test -p openshell-server --lib -q — 1,415 passed, 8 ignored
  • OPENSHELL_E2E_DOCKER_TEST=sandbox_lifecycle mise run e2e:docker — 7 passed
  • Supervisor and CLI lifecycle tests
  • Python type checking and tests — 89 passed
  • mise run sdk:ts:ci
  • Go SDK tests and generated-protobuf checks
  • GitHub branch checks on Linux amd64, Linux arm64, and macOS

Checklist

  • Follows Conventional Commits
  • Commits are signed off for DCO
  • User-facing and architecture documentation updated
  • Sandbox infrastructure E2E exercised

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@github-actions

Copy link
Copy Markdown

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
krishicks
krishicks previously approved these changes Aug 21, 2026
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@krishicks

Copy link
Copy Markdown
Collaborator

I'm trying out a new format for agent review. gpt-5.6-sol medium found some issues, which after some back and forth was reshaped into the following output, which I think is much more human-readable and understandable.


Gateway cleanup can preempt SSH exit delivery

Before ef29680, Ctrl-D ended an ordinary SSH shell. SSH delivered the shell’s exit status directly, while the sandbox remained Ready; no gateway lifecycle cleanup was triggered by that shell exit.

After ef29680, Ctrl-D ended the canonical main process. Its result was reported to the gateway, which transitioned the sandbox to terminal Error and could tear down its runtime connectivity.

In this PR, the supervisor attempts to coordinate durable result reporting with SSH delivery, but it marks the terminal event as “delivered” before actually sending the SSH exit status. It then waits for the gateway to acknowledge the result. That acknowledgement can trigger session cleanup and disconnect the relay before the pending SSH exit-status frame is sent. As a user, the command may finish and its result may be stored, but the CLI can see an interrupted SSH connection instead of the command’s exit status.

Details: crates/openshell-supervisor-process/src/ssh.rs:643-647.

Infrastructure failures can be presented as command failures

Before ef29680, the foreground command’s SSH exit status was the command result, while failures to create, connect to, or run the sandbox were separate CLI/transport errors. There was no durable canonical-main exit code on the sandbox record to confuse the two.

After ef29680, canonical-main exits and sandbox runtime failures both resulted in terminal Error. The stored condition could describe the cause, but the phase itself did not distinguish workload failure from infrastructure failure.

In this PR, the server introduces the intended distinction:

  • MainProcessFailed means the user’s command returned nonzero.
  • Other Error reasons mean the sandbox runtime or infrastructure failed.

However, an infrastructure Error may still be enriched with an observed exit_code, and the CLI currently treats any terminal Error containing an exit code as a main-process result. As a user, a sandbox runtime failure could therefore be reported as though their command simply exited nonzero, hiding the actual infrastructure error.

Details: crates/openshell-server/src/compute/mod.rs:3359-3369, :3398; crates/openshell-cli/src/run.rs:764-771.

SSH credentials can cross a sandbox runtime generation

Before ef29680, Ctrl-D ended only an SSH shell and did not create a new sandbox runtime generation. The SSH session could remain valid for its configured TTL, but reconnecting reached the same still-running sandbox.

After ef29680, Ctrl-D ended the canonical main process and left the sandbox terminal. Restarting an Error sandbox directly was rejected. To create a fresh runtime generation, the user had to stop and start the sandbox, and the stop path disconnected the supervisor and deleted the sandbox’s SSH sessions before restart.

In this PR, Completed and MainProcessFailed sandboxes can be started directly without passing through the stop cleanup path. As a result, an SSH credential issued for the previous runtime generation can remain valid against the newly started generation. As a user—or anyone holding that credential—the old credential can access a fresh workload instance that previously would have required newly issued credentials.

Details: pre-PR cleanup at ef29680^:crates/openshell-server/src/compute/mod.rs:1125-1137 and :2960-2967; new direct-start eligibility at crates/openshell-server/src/compute/mod.rs:1183-1194.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: support ephemeral job-style sandboxes

2 participants

@drew@krishicks