feat(runtime-host): bound shutdown with process fail-stop

[M4n5ter]

English

Summary

This PR establishes the bounded shutdown foundation required before a Runtime Host owns additional Interactive domains:

• the first drain request closes new connection and operation admission and starts one absolute clean-shutdown deadline, even while an already admitted command is still reaching its seal point;
• admitted work keeps the existing normal response window, while operation, composition, residency, control-plane, and owner cleanup share one finite budget rather than restarting a timeout at each phase;
• expiry rejects closed with a typed process_termination_required result, and the dedicated Host process immediately fail-stops instead of continuing ordinary cleanup or force-releasing ownership;
• the OS-held owner lock remains the only writer authority and is reclaimed by process death; a successor still has to acquire that lock before it can recover or serve;
• the production Candidate and the real execution child use the same signal, close, and fail-stop process lifecycle.

Boundary and decisions

The default clean-shutdown grace is 10 seconds. The existing one-second operation and handshake windows remain early transport-drain bounds; they are not separate ownership deadlines. A command that was admitted before shutdown may finish its handler and enqueue its protocol response, but a command that never reaches that boundary can no longer keep the process and owner lock alive forever.

RuntimeHostKernel reports deadline expiry as a typed process-termination requirement instead of calling process.exit() from a reusable in-process class. The dedicated process entrypoint owns the hard exit. This keeps the Kernel testable without weakening the production rule: once fail-stop is required, ordinary endpoint cleanup, registration removal, or owner.close() cannot be used as proof that the old writer is isolated.

This slice does not add a lock-stealing path, a protocol-visible fail-stop state, a supervisor, or arbitrary-descendant containment. Registration and endpoint files remain discovery evidence only. The successor journey is gated by actual OS lock acquisition.

Scope

This PR changes Runtime Host lifecycle and its dedicated process entrypoints only. It does not migrate an Interactive domain, change protocol operation shapes, alter Storage's drain-before-unlock contract, or activate the production Host cutover.

Validation

A real child-process test uses the real local transport and Interactive owner lock, blocks an admitted turn.start before seal, verifies that a contender cannot acquire ownership during the grace period, observes bounded non-clean process exit, and then proves that a different-epoch successor acquires the lock and serves host.status. The existing active-Turn shutdown journey now also proves a normal code-zero exit and preserves its single durable cancelled outcome.

The Runtime Host suite passes 61 tests. Root build, typecheck, lint, and format checks pass.

Tracked by #1167. Architectural context: #853.

简体中文

概要

本 PR 建立 Runtime Host 在继续持有其他 Interactive 领域前所需的 bounded shutdown foundation：

• 第一次 drain request 会关闭新的 connection 与 operation admission，并启动唯一的 absolute clean-shutdown deadline；即使一个已经接纳的 command 仍在走向 seal point，也不会延后计时；
• 已接纳工作继续保留现有的正常回复窗口；operation、composition、residency、control-plane 与 owner cleanup 共用一个有限预算，不会在每个阶段重新获得一份 timeout；
• deadline 到期后，closed 以 typed process_termination_required 结束，专用 Host process 立即 fail-stop，不再继续普通 cleanup 或强制释放 ownership；
• OS-held owner lock 继续作为唯一 writer authority，并只由进程死亡触发回收；successor 仍须真实取得该 lock，之后才能 recovery 或提供服务；
• production Candidate 与真实 execution child 共用同一套 signal、close 与 fail-stop process lifecycle。

边界与决策

默认 clean-shutdown grace 为 10 秒。既有的 1 秒 operation/handshake window 继续作为提前处理 transport drain 的边界，而不是另一份 ownership deadline。shutdown 前已经接纳的 command 可以完成 handler 并把协议回复接纳进写队列；永远到不了该边界的 command 则不能再永久占住进程与 owner lock。

RuntimeHostKernel 以 typed process-termination requirement 报告 deadline 到期，而不是从可复用的进程内 class 直接调用 process.exit()；hard exit 由专用进程入口负责。这样既保持 Kernel 可测试，也不削弱 production 规则：进入 fail-stop 后，普通 endpoint cleanup、registration 删除或 owner.close() 都不能被当作旧 writer 已隔离的证明。

本 slice 不增加偷锁路径、protocol-visible fail-stop 状态、supervisor 或 arbitrary-descendant containment。registration 与 endpoint 文件仍然只提供 discovery evidence；successor journey 由真实 OS lock acquisition 约束。

范围

本 PR 只改变 Runtime Host lifecycle 与专用进程入口。它不迁移 Interactive 领域，不改变 protocol operation shape，不修改 Storage 的 drain-before-unlock 契约，也不启用 production Host cutover。

验证

一个真实 child-process 测试使用真实 local transport 与 Interactive owner lock，让已接纳的 turn.start 在 seal 前永久阻塞，验证 grace 期间 contender 无法取得 ownership，观察有界的 non-clean process exit，并证明不同 Epoch 的 successor 随后取得 lock 且能够提供 host.status。既有 active-Turn shutdown journey 也新增了 code-zero 正常退出断言，并继续保留唯一 durable cancelled outcome。

Runtime Host 全包 61 项通过；根级 build、typecheck、lint 与 format check 通过。

由 #1167 跟踪；架构背景见 #853。

[Astro-Han]

Approving. I traced the shutdown contract end to end and verified it against the code: every shutdown entry point arms the same single deadline (guarded against double-arm), the 1s operation/handshake graces sit inside the 10s budget rather than stacking, every await in #closeResources has an #assertShutdownCanContinue checkpoint after it, and closed settles by the deadline even when an internal await never does. The fail-stop handoff uses process.exit(1), not exitCode, so live handles and parked promises cannot delay death; the lock is released only by death or a completed owner.close(); and the child-process test proves the full journey (contender blocked during grace, bounded exit 1, different-epoch successor acquires and serves). The active-Turn regression still exits 0 with its single durable cancelled outcome. The unbounded-shutdown finding from the #1190 architecture review is closed.

Some P3-level discussion, none of it blocking:

The mid-drain checkpoint path has no test. The uncooperative fixture's blocked turn.start never seals, so #activeCommandOperations stays above zero and #commitRequestedShutdownIfQuiescent returns early forever: #commitShutdown and #closeResources never run. The test covers "deadline fires before commit"; the other half, "deadline fires between two checkpoints inside #closeResources and skips composition.close()/owner.close()", has never executed in a terminating scenario. A fixture with a never-settling query (commit starts normally) or a composition.close() that never resolves would cover it. Worth adding here or in the next slice, since domain coordinators are about to multiply the cleanup phases this path protects.

Mixed signals do not escalate. Both signal listeners share one closing guard, so SIGTERM followed by SIGINT (or the reverse) swallows the second signal and the process waits out the deadline. Only repeating the same signal hits the default disposition for an immediate kill. I reproduced this with a standalone process. Whether an operator's second, different signal should force-kill is a small product call, but the current behavior depends on signal identity in a way nobody will expect.

The enforcement helper is not on the public surface.runRuntimeHostProcessLifecycle is importable only by internal path; server/index.ts exports only the error class. Today's contract is fully delivered because the one production entrypoint uses it, but the enforcement half is opt-in: after a process_termination_required rejection the kernel deliberately keeps the flock, server, and registration held, so any future entrypoint that just sets exitCode will hang forever while holding the owner lock. Exporting the helper before the next slice adds an entrypoint would make the safe path the easy path.

Test timing margin. The 2000ms grace in the uncooperative test serializes two 1000ms read timeouts plus the contender acquisition before asserting the child is still alive. Comfortable today, theoretically flaky on a stalled CI worker.

Contract notes for the next slices, since coordinators with their own drain/close semantics are coming:

• A first drain request seals all admission, including handshakes and queries, not just commands. Coordinators calling requestDrain() should expect queries to start failing with host_draining immediately.
• The deadline starts at the drain request, not at commit. A composition whose commands hold for 9s leaves 1s for composition.close(), residency drains, and owner close combined. New coordinators should budget their cleanup well under the shared 10s.
• Fail-stop is crash-equivalent for recovery: registration may still say ready, the endpoint file may linger, and the only trustworthy signals are pid liveness and lock acquisition. Election already classifies this way; keep it that way.

简体中文

通过。我把 shutdown 契约端到端 trace 了一遍并对照代码验证：所有 shutdown 入口 arm 的是同一个 deadline（有防重复 arm 的 guard）；1 秒的 operation/handshake grace 在 10 秒预算之内而不是叠加；#closeResources 里每个 await 后面都有 #assertShutdownCanContinue() checkpoint；即使内部某个 await 永不 settle，closed 也会按 deadline 收敛。fail-stop handoff 用的是 process.exit(1) 而不是 exitCode，存活的 handle 和悬着的 Promise 都拖不住进程死亡；锁只随进程死亡或完整走完的 owner.close() 释放；child-process 测试证明了完整旅程（grace 期内 contender 拿不到锁、有界 exit 1、不同 epoch 的 successor 取得锁并服务）。active-Turn 回归仍然是 exit 0 加唯一 durable cancelled outcome。#1190 架构评审里的 unbounded-shutdown finding 已关闭。

一些 P3 级讨论，都不阻塞：

mid-drain 的 checkpoint 路径没有测试。 uncooperative fixture 里被卡住的 turn.start 永不 seal，#activeCommandOperations 一直大于零，#commitRequestedShutdownIfQuiescent 永远提前返回：#commitShutdown 和 #closeResources 从未执行。测试覆盖的是"deadline 在 commit 前触发"；另一半"deadline 在 #closeResources 两个 checkpoint 之间触发、跳过 composition.close()/owner.close()"从未在 terminating 场景下跑过。用一个永不 settle 的 query（commit 会正常启动）或一个永不 resolve 的 composition.close() 做 fixture 就能覆盖。值得在本 slice 或下一个 slice 补上，因为 domain coordinator 马上会成倍增这条路径要保护的 cleanup 阶段。

混合信号不会升级。 两个信号 listener 共用一个 closing guard，所以先 SIGTERM 再 SIGINT（或反过来）时第二个信号被吞掉，进程等满 deadline。只有重复同一种信号才走默认处置立即杀死。我用独立进程复现过。操作者发的第二个不同信号是否应该强杀是个小的产品决策，但当前行为取决于信号是否同名，没有人会预期这一点。

enforcement helper 不在公开表面。runRuntimeHostProcessLifecycle 只能经内部文件路径 import，server/index.ts 只导出了 error class。今天的合同完整交付，因为唯一生产入口已接入；但 enforcement 这一半是 opt-in：process_termination_required rejection 之后 kernel 故意继续持有 flock、server 和 registration，任何未来入口若只设 exitCode 会永远 hang 且持有 owner 锁。在下一个 slice 新增入口之前导出这个 helper，能让安全路径成为好走的路径。

测试时间余量。 uncooperative 测试的 2000ms grace 里串行排了两个 1000ms read timeout 加 contender acquisition，然后才断言子进程仍存活。今天够用，CI 机器 stall 时有理论 flake 风险。

给后续 slice 的合同须知，因为自带 drain/close 语义的 coordinator 就要来了：

• 第一次 drain 请求即封堵全部 admission，包括握手和 query，不只是 command。调用 requestDrain() 的 coordinator 要预期 query 立即开始收到 host_draining。
• deadline 从 drain 请求起算，不是从 commit 起算。composition 的 command 挂 9 秒，留给 composition.close()、residency drain 和 owner close 的就只剩 1 秒。新 coordinator 的内部 cleanup 预算应远小于共享的 10 秒。
• fail-stop 对恢复路径等价于 crash：registration 可能仍显示 ready，endpoint 文件可能残留，唯一可信的信号是 pid 存活和锁获取。election 现在就是这么分类的，保持下去。

[M4n5ter]

@Astro-Han Thank you. I agree that all four notes identify real boundaries, but I am keeping this already-approved foundation diff unchanged rather than extending it with speculative API or fixture work.

• The current child journey proves that the absolute deadline starts before shutdown commit, the dedicated process fail-stops, and the OS reclaims the owner lock. A true mid-drain checkpoint test needs a controllably delayed real cleanup phase; it is more valuable alongside the first coordinator/domain slice that introduces one than as an artificial second fixture here.
• The first SIGTERM or SIGINT remains bounded by the same fail-stop deadline. Whether a different second signal means immediate operator escalation is a separate supervisor/operator contract, so I am not adding an untested signal-state machine to this slice.
• runRuntimeHostProcessLifecycle stays internal while Candidate is its only production consumer. It can be exported with the first additional process entrypoint, where bypassing it becomes a concrete risk.
• The 2-second test margin is a theoretical scheduling risk today. If it flakes, the useful fix is to move the liveness/contender assertions earlier, not lengthen the test's grace period.

简体中文

@Astro-Han 感谢审查。我认同这四点都指出了真实边界，但会保持这个已经获批的 foundation diff 不变，不为了它们提前扩展 API 或测试 fixture。

• 当前 child journey 已证明 absolute deadline 在 shutdown commit 前启动、专用进程会 fail-stop、owner lock 最终由 OS 回收。真正的 mid-drain checkpoint 测试需要一个可控延迟的真实 cleanup phase；把它放到首个引入该阶段的 coordinator/domain slice 中，比在这里再造一个人工 fixture 更有价值。
• 第一次 SIGTERM 或 SIGINT 仍受同一 fail-stop deadline 约束。不同种类的第二个 signal 是否表示立即进行 operator escalation，属于独立的 supervisor/operator 契约；本 slice 不增加未经行为测试的 signal state machine。
• 当前 Candidate 是 runRuntimeHostProcessLifecycle 唯一的 production consumer，因此 helper 继续保持内部可见。第一个额外 process entrypoint 出现时，再连同该入口一起导出，届时绕过它才是具体风险。
• 2 秒测试余量目前只是理论上的调度风险。如果后续出现 flake，合理修复是把 liveness/contender 断言前移，而不是继续加长 grace period。