Uh oh!
There was an error while loading. Please reload this page.
test(runtime-host): observe pre-factory drain over handshake, not registration file - #1660
Merged
Merged
Conversation
Astro-Han added a commit
that referenced
this pull request
Jul 30, 2026
requestDrain() flips in-memory #state to 'draining' synchronously, but persisting that to the registration file is the async writeHostRegistration I/O in #closeResources, while #start() had already persisted 'recovering' before entering the factory. The fixed sleep(50) bet that I/O landed in time, which lost on loaded CI runners and read back stale 'recovering' (the #1658 main CI failure). Poll the registration file for 'draining' instead of sleeping a fixed gap. #closeResources writes 'draining' first, and the file stays 'draining' while the factory is suspended, because removeHostRegistration runs only after the blocked compositionStartup await later in #closeResources. A handshake connection does not work here: #closeResources calls server.close() right after the registration write, so on loaded CI the socket stops accepting before connectRuntimeHost finishes its resolveStorageRoot, mkdir, and registration read, returning 'unavailable' instead of 'draining' (the #1660 CI failure). Verified with a 300ms writeHostRegistration delay: the old sleep(50) form fails with the same 'recovering' seen on main, while the polling form passes.
Astro-Hanforce-pushed
the
fix/runtime-host-drain-handshake
branch
from
July 30, 2026 14:30
0583de2 to
8f3ab55Compare… on CI The "drain requested before factory completion begins drain before recovery exactly once" test failed the test job on main after #1658 landed (run 30544963454), even though #1658 only touches packages/headless. The flaky test was introduced in #1359. The flake came from asserting on the registration file mid-flight. requestDrain() flips in-memory #state to 'draining' synchronously, but persisting that to the registration file is the async writeHostRegistration I/O in #closeResources, while #start() had already persisted 'recovering' before entering the factory. The fixed sleep(50) bet that I/O landed in time, which lost on loaded CI runners and read back stale 'recovering'. That assertion tested a non-contract. No production caller reads HostRegistration.state; clients learn draining from the handshake (#admitHandshake returns kind:'draining' from #shutdownRequested), and connectResolvedRuntimeHost reads the registration only for rootId, hostEpoch, and the endpoint. The invariant the test name claims, "begins drain before recovery exactly once", is already proven by the final lifecycle array ['factory-return', 'begin-drain', 'recover', 'close'] with begin-drain counted once. The in-memory draining state is covered elsewhere (candidate.host.state === 'draining'). Drop the sleep, the readHostRegistration call, and the state assertion. The test still verifies, while the factory is suspended, that startup has not settled, that no lifecycle event has fired, and that the host still holds the interactive root owner lock, then checks the full lifecycle ordering after release. A handshake-based replacement was tried first but also raced: #closeResources calls server.close() right after the registration write, so on loaded CI the socket stops accepting before connectRuntimeHost connects, returning 'unavailable' instead of 'draining' (the first push of #1660). Refs #1658 (main CI failure, not a regression from that PR).
Astro-Hanforce-pushed
the
fix/runtime-host-drain-handshake
branch
from
July 30, 2026 14:53
8f3ab55 to
72dac8eCompareUh oh!
There was an error while loading. Please reload this page.
This was referenced Aug 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
drain requested before factory completion begins drain before recovery exactly oncetest inpackages/runtime-hostwas flaky and failed thetestjob onmainafter #1658 landed (run 30544963454), even though #1658 only touchespackages/headlessand the runtime-host code was identical between the PR and main runs. The flaky test was introduced in #1359.The flake came from asserting on the registration file mid-flight.
requestDrain()flips the in-memory#stateto'draining'synchronously, but persisting that to the registration file is the asyncwriteHostRegistrationI/O in#closeResources, and#start()had already persisted'recovering'before entering the composition factory. The test's fixedsleep(50)bet that I/O landed within 50ms, which lost on loaded CI runners and read back stale'recovering'(AssertionError: + 'recovering' - 'draining').That assertion tested a non-contract. No production caller reads
HostRegistration.state; clients learn draining from the handshake (#admitHandshakereturnskind: 'draining'from#shutdownRequested), andconnectResolvedRuntimeHostreads the registration only forrootId,hostEpoch, and the endpoint. The invariant the test name claims, "begins drain before recovery exactly once", is already proven by the final lifecycle array['factory-return', 'begin-drain', 'recover', 'close']withbegin-draincounted once, and the in-memory draining state is covered elsewhere (candidate.host.state === 'draining').Drop the
sleep(50), thereadHostRegistrationcall, and theregistration.state === 'draining'assertion. The test still verifies, while the factory is suspended, that startup has not settled, that no lifecycle event has fired, and that the host still holds the interactive root owner lock, then checks the full lifecycle ordering after release.A handshake-based replacement was tried first but also raced:
#closeResourcescallsserver.close()right after the registration write, so on loaded CI the socket stops accepting beforeconnectRuntimeHostconnects, returningunavailableinstead ofdraining(the first push of this PR). Polling the registration file would fix the timing too, but it keeps testing a field no caller reads, so deleting the assertion is the smaller and more correct fix.Refs #1658 (main CI failure, not a regression from that PR).
Verification
npm --workspace @maka/runtime-host run buildthennode --test dist/__tests__/host-kernel.test.js: 30/30 passnode --test --test-concurrency=4 "dist/__tests__/*.test.js": 364/364 passsleepor I/O left in the assertion path)writeHostRegistrationdelay: the oldsleep(50)form fails with+ 'recovering' - 'draining'; the deleted form has no file read to raceunavailable, matching the first push's CI failurenpm run format:check,npm run lint,npm run typecheck: clean