Skip to content

refactor(cli): unify MakaRunOutcome classification - #3092

Merged
Astro-Han merged 10 commits into
mainfrom
refactor/3088-unify-run-outcomes
Aug 21, 2026
Merged

refactor(cli): unify MakaRunOutcome classification#3092
Astro-Han merged 10 commits into
mainfrom
refactor/3088-unify-run-outcomes

Conversation

@Astro-Han

@Astro-HanAstro-Han commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Normalize live SessionEvent and durable StoredMessage inputs through one internal outcome reducer while preserving their distinct incomplete-terminal policies.
  • Align cancellation, failure, complete-stop, and sandbox recovery semantics across maka run and maka run --graph.
  • Remove InvocationResult-era sandbox helpers and unused CLI wrappers and exports.
  • Pin ordinary and Graph success/failure exit codes at the public CLI boundary.

Fixes#3088

Verification

  • npm --workspace maka-agent run build
  • node --test packages/cli/dist/__tests__/runtime-host-run-command.test.js — 25 passed
  • npm --workspace maka-agent run typecheck
  • npm run format:check
  • npx biome check on all changed files
  • Full-repository tests were not run; the focused CLI suite covers the changed execution seam.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Codex analyzed the duplicate outcome paths, implemented the reducer and dead-code cleanup, authored and ran the focused tests, and drafted this PR description. The commits include Generated-by: Codex trailers.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@coderabbitai

coderabbitaiBot commented Aug 15, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5112463d-5afa-4688-9de8-06aaad0d57cf

📥 Commits

Reviewing files that changed from the base of the PR and between 26c60c0 and c865cd0.

📒 Files selected for processing (6)
  • packages/cli/src/__tests__/runtime-host-run-command.test.ts
  • packages/cli/src/cli-core.ts
  • packages/cli/src/runtime-host-run-command.ts
  • packages/cli/src/sandbox-boundary-failure.ts
  • packages/cli/src/session-recap.ts
  • packages/cli/src/tui-diff.ts
💤 Files with no reviewable changes (2)
  • packages/cli/src/cli-core.ts
  • packages/cli/src/session-recap.ts
🚧 Files skipped from review as they are similar to previous changes (3)
  • packages/cli/src/tui-diff.ts
  • packages/cli/src/runtime-host-run-command.ts
  • packages/cli/src/tests/runtime-host-run-command.test.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 2 remain after this review.


📝 Walkthrough

What this PR solves

This PR unifies MakaRunOutcome classification for live SessionEvent and durable StoredMessage inputs. One internal reducer now handles cancellation, failure, completion, missing terminal events, and sandbox-boundary recovery for both maka run and maka run --graph.

The PR preserves distinct incomplete-terminal policies for live and stored execution. It also pins ordinary and graph success and failure exit codes at the CLI boundary.

The PR removes obsolete InvocationResult sandbox helpers and unused exports, including formatResumeHint, cleanRecapText, RECAP_INSTRUCTION, runMakaTextCli, and colorDiffRow.

Source of truth

The PR extends the existing outcome-handling path. It does not create a parallel classifier. Live events and stored messages are normalized and passed through the same internal reducer.

The reducer also serves Agent Graph terminal turns. This removes the previous event accumulator and stored sandbox-boundary helper.

Scope and complexity

The solution is the smallest coherent change described by the objectives:

  • It centralizes outcome classification.
  • It keeps incomplete-terminal behavior explicit for each input type.
  • It retains session-event sandbox failure-reason normalization.
  • It removes invocation-level detection and recovery logic that is no longer needed.
  • It adds focused tests for each changed outcome class and exit-code rule.

The added normalization and reducer branches are necessary to support two input representations and their different terminal policies.

The removed helpers and exports reduce dead code and public surface. No further deletion is apparent without weakening regression coverage. The new test fixtures and helpers simplify repeated setup and keep the 24 focused CLI cases maintainable.

Complexity delta

  • Authorities: Removes separate live-event and stored-message outcome logic. Adds one shared reducer as the authority.
  • States: Adds explicit outcome states for completion, runtime failure, user abort, missing terminal events, step-cap failure, and recovered sandbox boundaries.
  • Branches: Adds normalization and recovery branches. Removes duplicated invocation-level sandbox branches and the prior event accumulator.
  • Configuration: No configuration surface is added.
  • Public surface: Removes obsolete exports and makes SandboxBoundaryFailureReason private.
  • Test-maintenance burden: Adds focused fixtures and outcome-observation helpers. These replace duplicated setup and cover the new shared behavior.

Total maintenance complexity decreases. The reducer adds local state complexity, but it removes duplicated authorities and dead public APIs.

Validation

The PR reports validation for build, typecheck, formatting, Biome checks, and 24 focused CLI tests.

The test changes cover Host Turn failures, graph completion and failure exit codes, recovered sandbox boundaries, cancellations, step-cap failures, and later completion replacing a failed graph outcome. Final check status remains unverified without direct check results.

Review-relevant risks

The PR changes user-visible CLI exit codes and outcome classification for maka run and maka run --graph. These changes require independent human review under repository policy.

The PR removes exported functions, aliases, and a public type. Consumers outside the reviewed call sites may be affected. These public-contract changes require independent human review under repository policy.

No security, licensing, release, or governance effect was identified in the current diff.

The person performing the merge must review the final diff. A maintainer makes the final determination.

Walkthrough

The CLI now uses one classifier for live session events and stored messages. It reports explicit completion and failure outcomes, tracks sandbox recovery, validates Host and Graph exit codes, and removes unused exports and helpers.

Changes

Runtime outcome handling

Layer / File(s)Summary
Normalize terminal observations
packages/cli/src/runtime-host-run-command.ts, packages/cli/src/sandbox-boundary-failure.ts
Session events, tool results, and sandbox failures are normalized into typed observations for TurnOutcomeClassifier.
Apply shared classification
packages/cli/src/runtime-host-run-command.ts
Live and stored turns use the shared classifier. Incomplete turns fail with a missing-terminal failure.
Validate CLI outcomes and remove dead exports
packages/cli/src/__tests__/runtime-host-run-command.test.ts, packages/cli/src/cli-core.ts, packages/cli/src/session-recap.ts, packages/cli/src/tui-diff.ts
Tests cover failure, completion, abort, recovery, step-cap, and replacement outcomes. Unused exports and helpers are removed.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk:⚪ Minimal · up to c865c

The change unifies CLI outcome handling and removes unused internal helpers; no actionable merge-blocking correctness or production risk is identified.

Suggested reviewers:jackwener

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Linked Issues check⚠️ WarningThe PR meets the linked issue objectives except that removal of the dead runMakaTextCli export is not shown.Remove the runMakaTextCli export, or provide evidence that it was already removed and remains covered by this change.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly identifies the main change: unified MakaRunOutcome classification in the CLI.
Description check✅ PassedThe description includes the required summary, verification, AI-use declaration, checklist, behavior change, and linked issue.
Out of Scope Changes check✅ PassedThe changes align with the linked issue by unifying outcome classification and removing obsolete helpers and exports.
Ai Use Disclosure✅ PassedThe description selects generative tooling, names Codex and its scope, and all four PR commits contain the standalone consistent trailer Generated-by: Codex.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/3088-unify-run-outcomes

Comment @coderabbitai help to get the list of available commands.

@Astro-Han
Astro-Hanforce-pushed the refactor/3088-unify-run-outcomes branch 2 times, most recently from dfc300b to c865cd0CompareAugust 18, 2026 06:43
@coderabbitai

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

Comment threadpackages/cli/src/__tests__/runtime-host-run-command.test.ts
Comment threadpackages/cli/src/runtime-host-run-command.ts Outdated
@Astro-Han
Astro-Hanforce-pushed the refactor/3088-unify-run-outcomes branch from c865cd0 to 0f410bfCompareAugust 20, 2026 15:16

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One material finding is inline.

Codex-assisted review performed under the maintainer-approved review workflow.

Comment threadpackages/cli/src/runtime-host-run-command.ts Outdated
@Astro-Han
Astro-Hanforce-pushed the refactor/3088-unify-run-outcomes branch from 0f410bf to aff982dCompareAugust 20, 2026 20:22

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two material findings are inline.

Codex-assisted review performed under the maintainer-approved review workflow.

Comment threadpackages/cli/src/runtime-host-run-command.ts Outdated
Comment threadpackages/cli/src/runtime-host-run-command.ts

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One material finding is inline.

Codex-assisted review performed under the maintainer-approved review workflow.

Comment threadpackages/runtime-host/src/adapter/session-projector.ts
hqhq1025
hqhq1025 previously approved these changes Aug 21, 2026

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the current head against the previous denied-boundary-request finding. Live and durable classification now retain each tool name with its provider step, so a successful request_sandbox_boundary settlement cannot clear the unresolved failure; the ordinary and Graph public CLI regressions both return failure as required. The complete build, related CLI/Runtime Host/ToolRuntime suites, formatting, lint, and required CI passed. No actionable findings remain.

Codex-assisted review performed under the maintainer-approved review workflow.

@Astro-Han

Copy link
Copy Markdown
ContributorAuthor

@hqhq1025 The branch has been rebased onto the current main; the new head is fa3028627. The denied-boundary-request fix and its ordinary/Graph regressions are preserved, the Runtime Host compatibility epoch now follows main at 32, and the required test check passes. Could you please re-review the current head?

Automated request by Codex on behalf of @Astro-Han.

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on head fa30286270fd07dbb51538f0540a9c04092cea8d.

Problem reconstruction

This PR correctly identifies the duplicated live/durable outcome classifiers as the root maintenance problem. Normalizing both sources into one reducer, preserving their different incomplete-terminal policies, deleting InvocationResult-era helpers, and pinning public CLI exit codes is the right direction. The latest rebase also correctly carries the bounded sandbox failure reason over the Runtime Host wire and allocates compatibility epoch 32.

Findings

  1. P2: reconnect recovery can erase an unresolved sandbox failure from the active root Turn. The Session channel intentionally cuts queued events and rebuilds from the canonical transcript after a reconnect, but the transcript-replacement listener feeds only Graph terminal handling. The root classifier created in #observeTurn() consumes only live events. If the sandbox failure is durable but its live frame is lost before the sequence cut, the replacement can later seed only the terminal completion; the classifier returns completed and maka run can exit 0.

  2. P2: the reducer collapses multiple sandbox failures into one state. Two blocked calls in one provider step both set the same { status: 'unresolved', failedStepId }. A later provider step that successfully retries only one call flips the whole Turn to recovered, even though the sibling operation never ran. The public command can consequently exit 0 with unresolved blocked work.

First-principles and Occam assessment

The shared reducer is simpler than the previous duplicated authorities, but its sandbox state is not yet rich enough for the behavior it owns. The final structure should preserve unresolved failures individually, or consume an explicit Runtime recovery fact, and it must be rebuildable from the canonical replacement transcript after a sequence cut. A single mutable boundary flag cannot represent multiple failures or reconnect recovery correctly.

Deletion and tests

The dead exports and InvocationResult helpers are correctly removed; I found no additional production deletion required for merge. The new CLI tests are behavior-focused rather than coverage-only. Add public ordinary-command reconnect coverage and a two-failures/one-retry case for both ordinary and Graph paths. Also strengthen the epoch regression: RUNTIME_HOST_COMPATIBILITY_EPOCH > 29 would still pass at the previous epoch 31, so it does not prove this wire change owns epoch 32.

Merge readiness

Not ready to merge. GitHub CI is green. Locally, 110 focused CLI/Runtime Host tests passed, CLI and Runtime Host typechecks passed, formatting, Astryx inventory, and git diff --check passed. The affected packages built successfully after serializing the workspace build; an initial full build:test attempt was invalidated by local concurrent/dist-generation behavior and was not used as evidence.

Residual gaps after the two blockers are fixed: packaged-binary smoke was not run, and the recovery semantics still rely on inferred provider-step progression rather than an explicit operation-recovery identity.

Codex-assisted adversarial re-review; I verified the exact live head, current-main integration, prior denied-boundary fix, Runtime Host producer/wire/projector chain, reconnect sequence-cut path, and public CLI tests before publication.

Comment threadpackages/cli/src/runtime-host-run-command.ts Outdated
Comment threadpackages/cli/src/runtime-host-run-command.ts Outdated
Normalize live SessionEvent and durable StoredMessage inputs through one outcome reducer. Keep incomplete live streams fail-closed while durable graph projections remain pending until their terminal Turn is stored, and pin ordinary and graph exit codes.
Generated-by: Codex
Remove InvocationResult-era sandbox helpers and unused CLI wrappers and aliases while retaining the live SessionEvent boundary classifier and internal diff row renderer.
Generated-by: Codex
Let the latest normalized terminal observation determine the CLI outcome, matching the append-only durable turn-state authority. Remove coverage and branches for terminal sequences that the Runtime Host projector cannot emit, and pin failed-to-completed recovery at the public adapter seam.
Generated-by: Codex
Remove the stale recap-cleaner documentation and keep the sandbox failure reason type private to its only module.
Generated-by: Codex
Treat raw stream boundary failures as unclassified until the corresponding outcome records recovery or an unresolved failure. Keep unresolved outcomes aggregated by outcome id so a later successful Graph result cannot overwrite an earlier boundary decision.
Generated-by: Codex
Keep the established human-readable failure message when the final Agent Graph Turn is classified from durable messages, and pin the public CLI stderr contract.
Generated-by: Codex
Track live and durable tool calls in the unified outcome classifier so a sandbox boundary failure is recovered only by a successful tool result from a later provider step. Same-step sibling successes and unpaired legacy history remain unresolved, while a later failure restores the unresolved state.
Add public CLI regressions for ordinary and Graph runs plus coverage for later-step recovery, missing step identity, and repeated failures.
Generated-by: Codex
Carry only the bounded sandbox failure reason across the Runtime Host live tool-result projection so CLI outcome classification does not depend on stripped result content. Bump the compatibility epoch because older closed-frame decoders reject the new field.
Distinguish live terminal fallbacks from durable terminal replacement so trailing completion events cannot erase earlier errors or cancellations, while standalone failure-bearing stop reasons retain the core failure taxonomy.
Generated-by: Codex
Track the tool name alongside provider-step identity in the unified run outcome classifier. A successful request_sandbox_boundary settlement is control-plane completion, not evidence that the blocked work recovered, so it cannot clear an unresolved sandbox failure.
Pin ordinary and Graph public CLI behavior for sandbox failure followed by a denied boundary request and final output.
Generated-by: Codex
Rebuild the active Turn outcome from canonical transcript replacements and track unresolved sandbox failures independently so ambiguous partial recovery fails closed. Cover ordinary reconnect and multi-failure behavior across ordinary and Graph commands, and pin compatibility epoch 33 after integrating the latest main.
Generated-by: Codex
@Astro-Han
Astro-Hanforce-pushed the refactor/3088-unify-run-outcomes branch from 90cf955 to e86ef4fCompareAugust 21, 2026 09:51
@Astro-Han

Copy link
Copy Markdown
ContributorAuthor

@hqhq1025 The two requested changes are addressed in e86ef4fe6 on the rebased head. The active root outcome is now replayed from canonical transcript replacements, and unresolved sandbox failures remain individually represented with ambiguous recovery failing closed. Public ordinary reconnect and ordinary/Graph multi-failure regressions pass, together with the focused CLI and Runtime Host protocol suites, affected typechecks, lint, formatting, and git diff --check. CI is running on the exact head. Please re-review when ready.

Automated message by Codex.

@hqhq1025hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the current head against both previous blockers. No actionable findings remain.

Problem and mechanism: the PR correctly removes the duplicated live/durable CLI outcome authorities and normalizes both representations through one reducer while preserving their different incomplete-terminal policies. The latest revision now rebuilds the active root classifier from canonical transcript replacements after a sequence cut, so reconnect recovery cannot erase a durable sandbox failure. It also tracks unresolved failures independently by tool identity and fails closed when the current wire cannot correlate a partial retry.

First principles and Occam: one reducer is the simpler final architecture. Canonical transcript replacement is the right reconnect authority, and a collection of unresolved failures accurately represents the state that a single mutable flag could not. The conservative ambiguous-recovery rule is appropriate until the wire carries explicit retry identity.

Optimality, deletion, and tests: the InvocationResult-era helpers and unused exports are correctly removed. I found no additional production code or low-value test that should be deleted, and no deeper refactor is required for this PR. The new ordinary and Graph regressions directly cover reconnect restoration and two-failures/one-retry behavior.

Merge verdict: ready to merge. Residual risk is intentionally conservative false failure when multiple blocked operations are all retried but cannot be correlated; that is safer than returning exit code 0 with unresolved work and can be improved later with explicit recovery identity.

Verification: reviewed head e86ef4f; required CI is green; npm run build:test passed; 158 focused CLI, session-driver, Runtime Host protocol, and continuity tests passed; git diff --check passed.

@Astro-Han
Astro-Han merged commit c1e02d8 into mainAug 21, 2026
1 check passed
@Astro-Han
Astro-Han deleted the refactor/3088-unify-run-outcomes branch August 21, 2026 10:26
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(cli): remove dead outcome helpers and unify MakaRunOutcome classification

3 participants

@Astro-Han@MicroGery@hqhq1025