Background
The flow-error-handling checklist item records a FAIL, but like its sibling it is not a bug and not a regression — it is a checklist-vs-implementation contract disagreement with explicit implementation-side rationale. The QA run states this needs a maintainer decision rather than a fix ticket.
On a caught failure, the run's step log is exactly:
[ start, guarded_push (try_catch, success), record_failure (catch) ]
Nothing carries regionKind: 'try'. Nothing carries status: 'failure'. The persistence layer agrees with the in-memory log — this is not a serialisation gap. Reproduced 2×.
Note what the try_catch container's own step says: success. From the run log alone, a caught failure is indistinguishable from a clean run that happened to also touch the catch path — except that the catch's side effects exist.
Everything else in the item passes, and passes well: $error interpolation into the record; the retry ladder measured at 7.13s / 7.16s against the expected 0+1+2+4s; an unhandled failure terminating status=failed with both run-level and step-level errors; the designer Runs panel rendering region nesting; the loud ERROR log for a trigger-fired unheard failure. The "catch must not run when try succeeds" negative was proven with a purpose-built probe. The error handling works — the question is only what it records.
Premises
Each premise is independently re-checkable; the command to re-check it is on its own line.
P1 — A failed region's partial steps are deliberately not surfaced.runRegion() documents it in as many words: on failure the region throws as before (preserving try_catch retry semantics); "a failed attempt's partial steps are not surfaced".
rg -n -B12 "partial steps are not surfaced" packages/services/service-automation/src/engine.ts
P2 — try-catch-node.ts returns childSteps only from a region that succeeded. The success arm returns childSteps: trySteps; the catch arm returns childSteps: catchSteps. The failed try attempts return nothing — their steps are discarded in the catch (err) path.
rg -n -A4 "regionKind: 'try'" packages/services/service-automation/src/builtin/try-catch-node.ts
P3 — The engine splices childSteps into the parent log, so anything returned would appear. The plumbing to surface these steps already exists (#1479) and works for every region kind that succeeds; the failed-attempt steps are absent because they are never returned, not because the log cannot hold them.
rg -n -B4 -A4 "result.childSteps" packages/services/service-automation/src/engine.ts
P4 — Run summaries fold over the FLAT step log, so absent steps are absent from every downstream aggregate too. A container's body steps are spliced into the same log and folded from there; steps that were never spliced cannot be counted, so failure counts and per-node metrics silently omit failed try attempts.
rg -n -B8 "childSteps" packages/services/service-automation/src/run-summary.ts
P5 — There is prior art on region-nested nodes being invisible to other subsystems.#4380 (closed) recorded that nodes inside nested regions were invisible to flow lint rules, silently disabling two gating errors. Different subsystem, same underlying shape: region interiors not being surfaced to the layer that needs them.
gh issue view 4380 --repo objectstack-ai/objectstack
The question
Is a caught failure allowed to leave no forensic trace?
Today, after a try_catch run an operator cannot tell:
- what failed — no step records the error;
- how many attempts ran — the retry ladder is invisible in the log even though it demonstrably executed (measured at 7.13s);
- which node threw — the failing node produced no step at all.
The only evidence a failure occurred is the catch's side effects. If the catch is a bare notification, or if the catch's own write is what you are trying to explain, there is nothing.
Should the failed attempt's steps be surfaced — and if so, at what cost to log volume and to the "a container that recovered reports success" model?
Options
Option A — Keep the current behaviour; revise the checklist clause
Accept "a recovered failure is not an incident" as the contract, and rewrite the flow-error-handling clause to assert the steps that do appear rather than the ones it currently expects.
- 实际业务需求 — no measured pull recorded either way. Nobody has filed an operator complaint about this; the QA run is the first party to notice. But that is weak evidence of absence: the failure mode is silence, and silence does not generate reports. An operator who cannot explain a run generally concludes they misread it, not that the log is lossy.
- 项目长远合理性 — defensible as a model (the region recovered; its interior is an implementation detail) and it is the cheapest option. Its long-run cost is that the run log stops being a faithful record of execution: things demonstrably happened — four attempts, one throw — that the log denies. A log that omits events on purpose is one an operator learns not to trust, and that distrust generalises beyond
try_catch. - 防 AI 写代码/元数据犯错 — poor. An agent debugging a flow reads the run log; here the log actively misleads, since the container step reads
success. The agent's most likely conclusion is that the try region never ran, and its most likely next action is to "fix" a region that was working as designed.
Option B — Surface failed-attempt steps, tagged as such
Return the failed attempts' steps from runRegion() alongside the throw, tag them regionKind: 'try' with status: 'failure' and their attempt index, and splice them in ahead of the catch's steps.
- 实际业务需求 — the pull is inferred rather than measured, but the capability gap is concrete and stated plainly by the run: what failed, how many attempts, which node. Those are the three questions any post-incident review of a retrying flow starts with, and none is currently answerable.
- 项目长远合理性 — the most defensible end state. The step log becomes a faithful record: everything that executed appears, with its real status. It also composes correctly with what already works — the splicing plumbing (P3), the region tagging vocabulary (already correct for
loop-body/try/catch/parallel-branch), and the flat-fold summaries (P4) all handle these steps for free once they are returned. The real cost is log volume: a region retried 4 times emits 4× its body's steps, and a retry ladder inside a loop multiplies. That needs a cap or a per-flow opt-out, and the run-summary fold needs checking so retried attempts do not inflate per-node runs counts in a misleading way. - 防 AI 写代码/元数据犯错 — strongest. Declared = enforced applies to observability too: if the engine claims a step log, the log should contain the steps. An agent reading a complete log reaches the right conclusion without a human explaining the omission.
Option C — Surface only a failure summary, not the full steps
Instead of every failed attempt's steps, have the try_catch container's own step carry a structured failure record: attempt count, the failing node id, and the terminal error — and flip the container's status away from a bare success (e.g. recovered).
- 实际业务需求 — answers all three of the operator's questions at a fraction of Option B's volume. It does not give a step-by-step replay of a failed attempt, which matters when the failure is order-dependent or when a partial write landed before the throw.
- 项目长远合理性 — a good cost/benefit compromise and a small, contained change. The
recovered status is a genuine improvement on its own: success on a container that swallowed four failures is the single most misleading field in the current output. The downside is that it introduces a second, summary-shaped channel for information the step log is otherwise the home for. - 防 AI 写代码/元数据犯错 — good, and notably better than A: the container step stops lying. Slightly weaker than B, since an agent still cannot see what a failed attempt actually did.
Recommendation
Option C now, with Option B's shape kept open as the follow-on — and in either case, stop reporting success on a container that caught a failure.
Weighing the axes:
- 实际业务需求 has no measured pull on any side, so it should not decide this — but it does rule out gold-plating. That argues against doing B's full volume work on inference alone.
- 项目长远合理性 favours B in the abstract and C in practice: C buys the great majority of B's value (the three unanswerable questions become answerable) for a small, contained change, and it does not foreclose B later — the plumbing in P3 stays available.
- 防 AI 写代码/元数据犯错 rules out A. The decisive detail is P2/P3 combined with the container reporting
success: an agent or operator reading the run log today is not merely under-informed, they are actively pointed at the wrong conclusion. Declared = enforced cuts against a log that omits events it recorded, and consumer-side tolerance ("the operator can go read the ERROR log") is exactly the tolerance this principle says not to rely on.
Concretely, for Option C: give the try_catch container step a distinct status (recovered, or whatever this repo's vocabulary prefers) plus a structured record of attempt count, failing node id, and terminal error. That single change converts the run log from misleading to merely summary-level, which is the whole gap on the 防错 axis.
Then, if operator demand for a full replay materialises — that is the measured pull B currently lacks — do B on top, with a per-flow cap on retained failed-attempt steps and a check that run-summary's per-node runs fold counts retried attempts sensibly rather than inflating them.
Whichever is chosen, the flow-error-handling clause needs rewriting to match, since it currently asserts a contract the engine has never promised.
Source
Extracted from the QA run #7516 (framework a86db17). Checklist item flow-error-handling, recorded there as one of the two FAILs that are contract decisions rather than defects. Reproduced 2×.
Background
The
flow-error-handlingchecklist item records a FAIL, but like its sibling it is not a bug and not a regression — it is a checklist-vs-implementation contract disagreement with explicit implementation-side rationale. The QA run states this needs a maintainer decision rather than a fix ticket.On a caught failure, the run's step log is exactly:
Nothing carries
regionKind: 'try'. Nothing carriesstatus: 'failure'. The persistence layer agrees with the in-memory log — this is not a serialisation gap. Reproduced 2×.Note what the
try_catchcontainer's own step says:success. From the run log alone, a caught failure is indistinguishable from a clean run that happened to also touch the catch path — except that the catch's side effects exist.Everything else in the item passes, and passes well:
$errorinterpolation into the record; the retry ladder measured at 7.13s / 7.16s against the expected 0+1+2+4s; an unhandled failure terminatingstatus=failedwith both run-level and step-level errors; the designer Runs panel rendering region nesting; the loud ERROR log for a trigger-fired unheard failure. The "catch must not run when try succeeds" negative was proven with a purpose-built probe. The error handling works — the question is only what it records.Premises
Each premise is independently re-checkable; the command to re-check it is on its own line.
P1 — A failed region's partial steps are deliberately not surfaced.
runRegion()documents it in as many words: on failure the region throws as before (preservingtry_catchretry semantics); "a failed attempt's partial steps are not surfaced".P2 —
try-catch-node.tsreturnschildStepsonly from a region that succeeded. The success arm returnschildSteps: trySteps; the catch arm returnschildSteps: catchSteps. The failed try attempts return nothing — their steps are discarded in thecatch (err)path.P3 — The engine splices
childStepsinto the parent log, so anything returned would appear. The plumbing to surface these steps already exists (#1479) and works for every region kind that succeeds; the failed-attempt steps are absent because they are never returned, not because the log cannot hold them.P4 — Run summaries fold over the FLAT step log, so absent steps are absent from every downstream aggregate too. A container's body steps are spliced into the same log and folded from there; steps that were never spliced cannot be counted, so failure counts and per-node metrics silently omit failed try attempts.
P5 — There is prior art on region-nested nodes being invisible to other subsystems.#4380 (closed) recorded that nodes inside nested regions were invisible to flow lint rules, silently disabling two gating errors. Different subsystem, same underlying shape: region interiors not being surfaced to the layer that needs them.
The question
Is a caught failure allowed to leave no forensic trace?
Today, after a
try_catchrun an operator cannot tell:The only evidence a failure occurred is the catch's side effects. If the catch is a bare notification, or if the catch's own write is what you are trying to explain, there is nothing.
Should the failed attempt's steps be surfaced — and if so, at what cost to log volume and to the "a container that recovered reports success" model?
Options
Option A — Keep the current behaviour; revise the checklist clause
Accept "a recovered failure is not an incident" as the contract, and rewrite the
flow-error-handlingclause to assert the steps that do appear rather than the ones it currently expects.try_catch.success. The agent's most likely conclusion is that the try region never ran, and its most likely next action is to "fix" a region that was working as designed.Option B — Surface failed-attempt steps, tagged as such
Return the failed attempts' steps from
runRegion()alongside the throw, tag themregionKind: 'try'withstatus: 'failure'and their attempt index, and splice them in ahead of the catch's steps.loop-body/try/catch/parallel-branch), and the flat-fold summaries (P4) all handle these steps for free once they are returned. The real cost is log volume: a region retried 4 times emits 4× its body's steps, and a retry ladder inside a loop multiplies. That needs a cap or a per-flow opt-out, and therun-summaryfold needs checking so retried attempts do not inflate per-noderunscounts in a misleading way.Option C — Surface only a failure summary, not the full steps
Instead of every failed attempt's steps, have the
try_catchcontainer's own step carry a structured failure record: attempt count, the failing node id, and the terminal error — and flip the container's status away from a baresuccess(e.g.recovered).recoveredstatus is a genuine improvement on its own:successon a container that swallowed four failures is the single most misleading field in the current output. The downside is that it introduces a second, summary-shaped channel for information the step log is otherwise the home for.Recommendation
Option C now, with Option B's shape kept open as the follow-on — and in either case, stop reporting
successon a container that caught a failure.Weighing the axes:
success: an agent or operator reading the run log today is not merely under-informed, they are actively pointed at the wrong conclusion. Declared = enforced cuts against a log that omits events it recorded, and consumer-side tolerance ("the operator can go read the ERROR log") is exactly the tolerance this principle says not to rely on.Concretely, for Option C: give the
try_catchcontainer step a distinct status (recovered, or whatever this repo's vocabulary prefers) plus a structured record of attempt count, failing node id, and terminal error. That single change converts the run log from misleading to merely summary-level, which is the whole gap on the 防错 axis.Then, if operator demand for a full replay materialises — that is the measured pull B currently lacks — do B on top, with a per-flow cap on retained failed-attempt steps and a check that
run-summary's per-noderunsfold counts retried attempts sensibly rather than inflating them.Whichever is chosen, the
flow-error-handlingclause needs rewriting to match, since it currently asserts a contract the engine has never promised.Source
Extracted from the QA run #7516 (framework a86db17). Checklist item
flow-error-handling, recorded there as one of the two FAILs that are contract decisions rather than defects. Reproduced 2×.