test(runtime-host): drive liveness-crossing waits from an injected probe cadence - #2450

Merged
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection
Aug 8, 2026
Merged

test(runtime-host): drive liveness-crossing waits from an injected probe cadence#2450
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection

Conversation

@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor

Part of #2389 (Runtime Host workspace; the Pi TUI workspace landed as #2448, desktop e2e and alignment audit follow separately).

What changed

Two tests held work pending for a fixed 2.1s purely to outlive the client connection's hardcoded DEFAULT_LIVENESS_INTERVAL_MS = 2_000:

  • agent-graph-two-client-uds.test.tsFakeAgentGraphAuthority.stop() slept 2.1s (the fake-shutdown case named in the issue) so the pending agent.graph.stop request would cross a liveness probe cycle.
  • host-kernel.test.ts (slow domain work preserves multiplexed requests and retires only explicit deadlines) — the test slept 2.1s while holding an admitted request open before releasing its gate.

Both waits existed to prove the #2392 contract: liveness probes never retire a request that has no explicit deadline. The 2s value is incidental to that contract — what matters is that at least one probe fires while the request is pending.

The probe interval is now injectable: ConnectRuntimeHostInput.livenessIntervalMs (validated by the same requireTimeout as the existing timeout options, default unchanged at 2s), threaded through connectResolvedRuntimeHost into the connection's #scheduleLivenessCheck. Both tests inject a 100ms cadence and derive their waits from it (2 cycles + margin) — the wait is now measured in a unit the test controls instead of guessed against a constant it cannot see.

Retained contracts

  • Probe-crossing semantics are exercised identically: with the 100ms cadence, two probes fire while the request is pending, and the request still completes with its real result.
  • The probe timeout (DEFAULT_LIVENESS_TIMEOUT_MS), handshake/connect deadlines, and the explicit 50ms read_timeout case in the host-kernel test are untouched.
  • The remaining short sleeps in this workspace are all poll intervals inside bounded wait loops (already observable completion) — audited and left as-is.

Production surface

One additive optional field on ConnectRuntimeHostInput; passing nothing preserves today's behavior exactly.

Timing (local, node --test)

TestBeforeAfter
two UDS Clients query and control one Agent graph…~2.5s0.36s
slow domain work preserves multiplexed requests…~2.6s0.38s

Full workspace suite after a clean build: 734/734 pass.

…obe cadence
Part of apache#2389. Two tests held work pending for a fixed 2.1s purely to outlive
the client connection's hardcoded 2s liveness interval:
- the Agent-graph fake authority's slow stop() (the fake-shutdown case named
in the issue), proving agent.graph.stop survives probe cycles;
- host-kernel's slow-domain-work test, holding an admitted request pending
across a probe before releasing it.
The interval is now injectable (ConnectRuntimeHostInput.livenessIntervalMs,
default unchanged at 2s), so both tests measure their probe-crossing waits in
an injected 100ms unit instead of wall-clock guessing against a constant they
cannot see. The waits are derived (2 cycles + margin), not scheduler-load
guesses, and the contract — probes never retire a request that has no
explicit deadline (apache#2392) — is exercised identically.
The remaining short sleeps in this workspace are all poll intervals inside
bounded wait loops, which are already observable completion; they are
untouched.
Focused timing: the two affected tests drop from ~2.5s each to ~0.37s each
(node --test, local). Full workspace suite: 734/734 pass after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The injected cadence matches the existing connectTimeoutMs/handshakeTimeoutMs pattern, default behavior is unchanged (2s), and both tests are genuinely faster (430/450ms). The assertions still bite: if #2392 regresses, the pending request gets retired and the deepEqual fails.

One gap: the tests can't tell "probes fired and the request survived" from "probes never fired". If a future change stops using livenessIntervalMs in the constructor (types still compile), every assertion passes with zero probes in the window. A probe counter on the server side, asserted before release, would make the premise falsifiable. Cheap to add.

Nit: livenessIntervalMs is validated in the constructor, after connect + handshake, so a bad value surfaces as "handshake_failed" instead of a config error; the other timeouts validate up front. Approving.

…dence took
Review follow-up on the injected liveness cadence: the tests could not tell
"probes fired and the request survived" from "probes never fired" — if a
future change stopped threading livenessIntervalMs, every assertion would
still pass with zero probes in the window.
ConnectRuntimeHostInput gains onLivenessProbe, invoked after a probe
round-trips and validates its Host Epoch. Both tests now gate on two observed
probe round-trips while their long-lived request is pending — the host-kernel
test releases its admitted request only after the crossing (bounded, loud
timeout if the cadence stops taking effect), and the Agent-graph fake's stop()
awaits the crossing instead of a derived sleep, making the ordering causal
with no fixed timing at all.
Also validates livenessIntervalMs up front in connectResolvedRuntimeHost
alongside the other connect timeouts, so a bad value is a config error
instead of surfacing as handshake_failed.
Both tests get faster again (~0.29s / ~0.33s); 734/734 pass after a clean
build.
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both points addressed in bad4d15:

  • Falsifiable premise: ConnectRuntimeHostInput.onLivenessProbe fires after a probe round-trips and validates its Host Epoch. The host-kernel test now releases the admitted request only after two observed probe round-trips (bounded — a cadence that stops taking effect times out loudly instead of vacuously passing), and the Agent-graph fake's stop() awaits that same crossing instead of a derived sleep, so the ordering is causal with no fixed timing left at all.
  • Nit: livenessIntervalMs is now validated up front in connectResolvedRuntimeHost alongside connectTimeoutMs/handshakeTimeoutMs, so a bad value surfaces as a config error, not handshake_failed.

Both tests got faster again (~0.29s / ~0.33s); 734/734 after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. Both review points are substantively addressed:

  • the tests now wait for observed, epoch-validated probe round-trips instead of assuming the injected cadence fired;
  • livenessIntervalMs is validated before transport and handshake work, consistently with the existing timeout options.

The resulting tests are faster and their probe-crossing premise is now falsifiable.

P3 — Test observability is exposed through the public client input.

onLivenessProbe is documented as a test hook but is now part of the exported ConnectRuntimeHostInput. Its callback also runs inside the probe success chain, so an observer exception can fail the connection.

This is non-blocking because existing callers do not pass it and default production behavior is unchanged. Longer term, I would prefer observing probes through a package-internal test seam or Host/transport fixture. If the callback remains a public diagnostics hook, its exceptions should be isolated from connection health.

Small test-scoping suggestion: in the Agent Graph test, pass the probe observer only to the final TUI connection that issues agent.graph.stop. That makes it impossible for an earlier slow query on another connection to satisfy the shared counter.

The requested follow-up is effective, and I found no blocking issue. Approved.

简体中文

感谢继续修改。上一轮的两个问题都已经得到实质解决:

  • 测试现在等待实际观察到、并完成 Host Epoch 校验的 probe round-trip,不再假设注入的 cadence 已经触发;
  • livenessIntervalMs 现在与现有 timeout 配置一样,在 transport 和 handshake 之前完成校验。

测试变得更快,而且 probe-crossing 前提现在可以被证伪。

P3 — 测试观测能力进入了公开 client input。

onLivenessProbe 的文档明确说明它是 test hook,但它现在属于公开导出的 ConnectRuntimeHostInput。callback 还运行在 probe success chain 内,因此 observer 抛出的异常可能导致连接失败。

这是非阻塞问题,因为现有调用方不会传入它,默认生产行为也没有变化。长期来看,更适合通过 package-internal 测试 seam 或 Host/transport fixture 观察 probe。如果保留为公开 diagnostics hook,则 observer 异常不应影响连接健康。

还有一个小的测试作用域建议:Agent Graph 测试只给最终执行 agent.graph.stop 的 TUI connection 传入 probe observer。这样前面其他 connection 上的慢查询就不可能提前满足共享 counter。

本轮修改有效,没有发现阻塞问题。Approve。

Review follow-up (non-blocking P3): onLivenessProbe is a diagnostics hook,
so an exception thrown by the observer is now swallowed instead of running
inside the probe success chain where it would fail the connection it is
watching; the input doc says so explicitly.
Also scopes the Agent-graph test's observer to the one connection that
issues agent.graph.stop, so no other connection's slow query could ever
satisfy the shared probe counter.
@Astro-Han
Astro-Han merged commit 1823c9d into apache:mainAug 8, 2026
2 checks passed
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both follow-ups applied in fc78317:

  • Observer isolation: onLivenessProbe exceptions are now swallowed at the call site — a diagnostics hook can no longer fail the connection it is watching — and the input doc states that contract explicitly. On the longer-term point: agreed a package-internal seam would be cleaner; if a second consumer of probe observability ever appears, that's the moment to move it behind a Host/transport fixture rather than the public input.
  • Test scoping: the Agent-graph observer is now wired only onto the final TUI connection that issues agent.graph.stop, so no other connection's slow query can satisfy the shared counter.

Suite green after a clean build; typecheck and biome pass.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@UncertaintyDeterminesYou4ndMe@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

test(runtime-host): drive liveness-crossing waits from an injected probe cadence - #2450

Merged
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection
Aug 8, 2026
Merged

test(runtime-host): drive liveness-crossing waits from an injected probe cadence#2450
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection

Conversation

@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor

Part of #2389 (Runtime Host workspace; the Pi TUI workspace landed as #2448, desktop e2e and alignment audit follow separately).

What changed

Two tests held work pending for a fixed 2.1s purely to outlive the client connection's hardcoded DEFAULT_LIVENESS_INTERVAL_MS = 2_000:

  • agent-graph-two-client-uds.test.tsFakeAgentGraphAuthority.stop() slept 2.1s (the fake-shutdown case named in the issue) so the pending agent.graph.stop request would cross a liveness probe cycle.
  • host-kernel.test.ts (slow domain work preserves multiplexed requests and retires only explicit deadlines) — the test slept 2.1s while holding an admitted request open before releasing its gate.

Both waits existed to prove the #2392 contract: liveness probes never retire a request that has no explicit deadline. The 2s value is incidental to that contract — what matters is that at least one probe fires while the request is pending.

The probe interval is now injectable: ConnectRuntimeHostInput.livenessIntervalMs (validated by the same requireTimeout as the existing timeout options, default unchanged at 2s), threaded through connectResolvedRuntimeHost into the connection's #scheduleLivenessCheck. Both tests inject a 100ms cadence and derive their waits from it (2 cycles + margin) — the wait is now measured in a unit the test controls instead of guessed against a constant it cannot see.

Retained contracts

  • Probe-crossing semantics are exercised identically: with the 100ms cadence, two probes fire while the request is pending, and the request still completes with its real result.
  • The probe timeout (DEFAULT_LIVENESS_TIMEOUT_MS), handshake/connect deadlines, and the explicit 50ms read_timeout case in the host-kernel test are untouched.
  • The remaining short sleeps in this workspace are all poll intervals inside bounded wait loops (already observable completion) — audited and left as-is.

Production surface

One additive optional field on ConnectRuntimeHostInput; passing nothing preserves today's behavior exactly.

Timing (local, node --test)

TestBeforeAfter
two UDS Clients query and control one Agent graph…~2.5s0.36s
slow domain work preserves multiplexed requests…~2.6s0.38s

Full workspace suite after a clean build: 734/734 pass.

…obe cadence
Part of apache#2389. Two tests held work pending for a fixed 2.1s purely to outlive
the client connection's hardcoded 2s liveness interval:
- the Agent-graph fake authority's slow stop() (the fake-shutdown case named
in the issue), proving agent.graph.stop survives probe cycles;
- host-kernel's slow-domain-work test, holding an admitted request pending
across a probe before releasing it.
The interval is now injectable (ConnectRuntimeHostInput.livenessIntervalMs,
default unchanged at 2s), so both tests measure their probe-crossing waits in
an injected 100ms unit instead of wall-clock guessing against a constant they
cannot see. The waits are derived (2 cycles + margin), not scheduler-load
guesses, and the contract — probes never retire a request that has no
explicit deadline (apache#2392) — is exercised identically.
The remaining short sleeps in this workspace are all poll intervals inside
bounded wait loops, which are already observable completion; they are
untouched.
Focused timing: the two affected tests drop from ~2.5s each to ~0.37s each
(node --test, local). Full workspace suite: 734/734 pass after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The injected cadence matches the existing connectTimeoutMs/handshakeTimeoutMs pattern, default behavior is unchanged (2s), and both tests are genuinely faster (430/450ms). The assertions still bite: if #2392 regresses, the pending request gets retired and the deepEqual fails.

One gap: the tests can't tell "probes fired and the request survived" from "probes never fired". If a future change stops using livenessIntervalMs in the constructor (types still compile), every assertion passes with zero probes in the window. A probe counter on the server side, asserted before release, would make the premise falsifiable. Cheap to add.

Nit: livenessIntervalMs is validated in the constructor, after connect + handshake, so a bad value surfaces as "handshake_failed" instead of a config error; the other timeouts validate up front. Approving.

…dence took
Review follow-up on the injected liveness cadence: the tests could not tell
"probes fired and the request survived" from "probes never fired" — if a
future change stopped threading livenessIntervalMs, every assertion would
still pass with zero probes in the window.
ConnectRuntimeHostInput gains onLivenessProbe, invoked after a probe
round-trips and validates its Host Epoch. Both tests now gate on two observed
probe round-trips while their long-lived request is pending — the host-kernel
test releases its admitted request only after the crossing (bounded, loud
timeout if the cadence stops taking effect), and the Agent-graph fake's stop()
awaits the crossing instead of a derived sleep, making the ordering causal
with no fixed timing at all.
Also validates livenessIntervalMs up front in connectResolvedRuntimeHost
alongside the other connect timeouts, so a bad value is a config error
instead of surfacing as handshake_failed.
Both tests get faster again (~0.29s / ~0.33s); 734/734 pass after a clean
build.
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both points addressed in bad4d15:

  • Falsifiable premise: ConnectRuntimeHostInput.onLivenessProbe fires after a probe round-trips and validates its Host Epoch. The host-kernel test now releases the admitted request only after two observed probe round-trips (bounded — a cadence that stops taking effect times out loudly instead of vacuously passing), and the Agent-graph fake's stop() awaits that same crossing instead of a derived sleep, so the ordering is causal with no fixed timing left at all.
  • Nit: livenessIntervalMs is now validated up front in connectResolvedRuntimeHost alongside connectTimeoutMs/handshakeTimeoutMs, so a bad value surfaces as a config error, not handshake_failed.

Both tests got faster again (~0.29s / ~0.33s); 734/734 after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. Both review points are substantively addressed:

  • the tests now wait for observed, epoch-validated probe round-trips instead of assuming the injected cadence fired;
  • livenessIntervalMs is validated before transport and handshake work, consistently with the existing timeout options.

The resulting tests are faster and their probe-crossing premise is now falsifiable.

P3 — Test observability is exposed through the public client input.

onLivenessProbe is documented as a test hook but is now part of the exported ConnectRuntimeHostInput. Its callback also runs inside the probe success chain, so an observer exception can fail the connection.

This is non-blocking because existing callers do not pass it and default production behavior is unchanged. Longer term, I would prefer observing probes through a package-internal test seam or Host/transport fixture. If the callback remains a public diagnostics hook, its exceptions should be isolated from connection health.

Small test-scoping suggestion: in the Agent Graph test, pass the probe observer only to the final TUI connection that issues agent.graph.stop. That makes it impossible for an earlier slow query on another connection to satisfy the shared counter.

The requested follow-up is effective, and I found no blocking issue. Approved.

简体中文

感谢继续修改。上一轮的两个问题都已经得到实质解决:

  • 测试现在等待实际观察到、并完成 Host Epoch 校验的 probe round-trip,不再假设注入的 cadence 已经触发;
  • livenessIntervalMs 现在与现有 timeout 配置一样,在 transport 和 handshake 之前完成校验。

测试变得更快,而且 probe-crossing 前提现在可以被证伪。

P3 — 测试观测能力进入了公开 client input。

onLivenessProbe 的文档明确说明它是 test hook,但它现在属于公开导出的 ConnectRuntimeHostInput。callback 还运行在 probe success chain 内,因此 observer 抛出的异常可能导致连接失败。

这是非阻塞问题,因为现有调用方不会传入它,默认生产行为也没有变化。长期来看,更适合通过 package-internal 测试 seam 或 Host/transport fixture 观察 probe。如果保留为公开 diagnostics hook,则 observer 异常不应影响连接健康。

还有一个小的测试作用域建议:Agent Graph 测试只给最终执行 agent.graph.stop 的 TUI connection 传入 probe observer。这样前面其他 connection 上的慢查询就不可能提前满足共享 counter。

本轮修改有效,没有发现阻塞问题。Approve。

Review follow-up (non-blocking P3): onLivenessProbe is a diagnostics hook,
so an exception thrown by the observer is now swallowed instead of running
inside the probe success chain where it would fail the connection it is
watching; the input doc says so explicitly.
Also scopes the Agent-graph test's observer to the one connection that
issues agent.graph.stop, so no other connection's slow query could ever
satisfy the shared probe counter.
@Astro-Han
Astro-Han merged commit 1823c9d into apache:mainAug 8, 2026
2 checks passed
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both follow-ups applied in fc78317:

  • Observer isolation: onLivenessProbe exceptions are now swallowed at the call site — a diagnostics hook can no longer fail the connection it is watching — and the input doc states that contract explicitly. On the longer-term point: agreed a package-internal seam would be cleaner; if a second consumer of probe observability ever appears, that's the moment to move it behind a Host/transport fixture rather than the public input.
  • Test scoping: the Agent-graph observer is now wired only onto the final TUI connection that issues agent.graph.stop, so no other connection's slow query can satisfy the shared counter.

Suite green after a clean build; typecheck and biome pass.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@UncertaintyDeterminesYou4ndMe@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

test(runtime-host): drive liveness-crossing waits from an injected probe cadence - #2450

Merged
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection
Aug 8, 2026
Merged

test(runtime-host): drive liveness-crossing waits from an injected probe cadence#2450
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection

Conversation

@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor

Part of #2389 (Runtime Host workspace; the Pi TUI workspace landed as #2448, desktop e2e and alignment audit follow separately).

What changed

Two tests held work pending for a fixed 2.1s purely to outlive the client connection's hardcoded DEFAULT_LIVENESS_INTERVAL_MS = 2_000:

  • agent-graph-two-client-uds.test.tsFakeAgentGraphAuthority.stop() slept 2.1s (the fake-shutdown case named in the issue) so the pending agent.graph.stop request would cross a liveness probe cycle.
  • host-kernel.test.ts (slow domain work preserves multiplexed requests and retires only explicit deadlines) — the test slept 2.1s while holding an admitted request open before releasing its gate.

Both waits existed to prove the #2392 contract: liveness probes never retire a request that has no explicit deadline. The 2s value is incidental to that contract — what matters is that at least one probe fires while the request is pending.

The probe interval is now injectable: ConnectRuntimeHostInput.livenessIntervalMs (validated by the same requireTimeout as the existing timeout options, default unchanged at 2s), threaded through connectResolvedRuntimeHost into the connection's #scheduleLivenessCheck. Both tests inject a 100ms cadence and derive their waits from it (2 cycles + margin) — the wait is now measured in a unit the test controls instead of guessed against a constant it cannot see.

Retained contracts

  • Probe-crossing semantics are exercised identically: with the 100ms cadence, two probes fire while the request is pending, and the request still completes with its real result.
  • The probe timeout (DEFAULT_LIVENESS_TIMEOUT_MS), handshake/connect deadlines, and the explicit 50ms read_timeout case in the host-kernel test are untouched.
  • The remaining short sleeps in this workspace are all poll intervals inside bounded wait loops (already observable completion) — audited and left as-is.

Production surface

One additive optional field on ConnectRuntimeHostInput; passing nothing preserves today's behavior exactly.

Timing (local, node --test)

TestBeforeAfter
two UDS Clients query and control one Agent graph…~2.5s0.36s
slow domain work preserves multiplexed requests…~2.6s0.38s

Full workspace suite after a clean build: 734/734 pass.

…obe cadence
Part of apache#2389. Two tests held work pending for a fixed 2.1s purely to outlive
the client connection's hardcoded 2s liveness interval:
- the Agent-graph fake authority's slow stop() (the fake-shutdown case named
in the issue), proving agent.graph.stop survives probe cycles;
- host-kernel's slow-domain-work test, holding an admitted request pending
across a probe before releasing it.
The interval is now injectable (ConnectRuntimeHostInput.livenessIntervalMs,
default unchanged at 2s), so both tests measure their probe-crossing waits in
an injected 100ms unit instead of wall-clock guessing against a constant they
cannot see. The waits are derived (2 cycles + margin), not scheduler-load
guesses, and the contract — probes never retire a request that has no
explicit deadline (apache#2392) — is exercised identically.
The remaining short sleeps in this workspace are all poll intervals inside
bounded wait loops, which are already observable completion; they are
untouched.
Focused timing: the two affected tests drop from ~2.5s each to ~0.37s each
(node --test, local). Full workspace suite: 734/734 pass after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The injected cadence matches the existing connectTimeoutMs/handshakeTimeoutMs pattern, default behavior is unchanged (2s), and both tests are genuinely faster (430/450ms). The assertions still bite: if #2392 regresses, the pending request gets retired and the deepEqual fails.

One gap: the tests can't tell "probes fired and the request survived" from "probes never fired". If a future change stops using livenessIntervalMs in the constructor (types still compile), every assertion passes with zero probes in the window. A probe counter on the server side, asserted before release, would make the premise falsifiable. Cheap to add.

Nit: livenessIntervalMs is validated in the constructor, after connect + handshake, so a bad value surfaces as "handshake_failed" instead of a config error; the other timeouts validate up front. Approving.

…dence took
Review follow-up on the injected liveness cadence: the tests could not tell
"probes fired and the request survived" from "probes never fired" — if a
future change stopped threading livenessIntervalMs, every assertion would
still pass with zero probes in the window.
ConnectRuntimeHostInput gains onLivenessProbe, invoked after a probe
round-trips and validates its Host Epoch. Both tests now gate on two observed
probe round-trips while their long-lived request is pending — the host-kernel
test releases its admitted request only after the crossing (bounded, loud
timeout if the cadence stops taking effect), and the Agent-graph fake's stop()
awaits the crossing instead of a derived sleep, making the ordering causal
with no fixed timing at all.
Also validates livenessIntervalMs up front in connectResolvedRuntimeHost
alongside the other connect timeouts, so a bad value is a config error
instead of surfacing as handshake_failed.
Both tests get faster again (~0.29s / ~0.33s); 734/734 pass after a clean
build.
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both points addressed in bad4d15:

  • Falsifiable premise: ConnectRuntimeHostInput.onLivenessProbe fires after a probe round-trips and validates its Host Epoch. The host-kernel test now releases the admitted request only after two observed probe round-trips (bounded — a cadence that stops taking effect times out loudly instead of vacuously passing), and the Agent-graph fake's stop() awaits that same crossing instead of a derived sleep, so the ordering is causal with no fixed timing left at all.
  • Nit: livenessIntervalMs is now validated up front in connectResolvedRuntimeHost alongside connectTimeoutMs/handshakeTimeoutMs, so a bad value surfaces as a config error, not handshake_failed.

Both tests got faster again (~0.29s / ~0.33s); 734/734 after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. Both review points are substantively addressed:

  • the tests now wait for observed, epoch-validated probe round-trips instead of assuming the injected cadence fired;
  • livenessIntervalMs is validated before transport and handshake work, consistently with the existing timeout options.

The resulting tests are faster and their probe-crossing premise is now falsifiable.

P3 — Test observability is exposed through the public client input.

onLivenessProbe is documented as a test hook but is now part of the exported ConnectRuntimeHostInput. Its callback also runs inside the probe success chain, so an observer exception can fail the connection.

This is non-blocking because existing callers do not pass it and default production behavior is unchanged. Longer term, I would prefer observing probes through a package-internal test seam or Host/transport fixture. If the callback remains a public diagnostics hook, its exceptions should be isolated from connection health.

Small test-scoping suggestion: in the Agent Graph test, pass the probe observer only to the final TUI connection that issues agent.graph.stop. That makes it impossible for an earlier slow query on another connection to satisfy the shared counter.

The requested follow-up is effective, and I found no blocking issue. Approved.

简体中文

感谢继续修改。上一轮的两个问题都已经得到实质解决:

  • 测试现在等待实际观察到、并完成 Host Epoch 校验的 probe round-trip,不再假设注入的 cadence 已经触发;
  • livenessIntervalMs 现在与现有 timeout 配置一样,在 transport 和 handshake 之前完成校验。

测试变得更快,而且 probe-crossing 前提现在可以被证伪。

P3 — 测试观测能力进入了公开 client input。

onLivenessProbe 的文档明确说明它是 test hook,但它现在属于公开导出的 ConnectRuntimeHostInput。callback 还运行在 probe success chain 内,因此 observer 抛出的异常可能导致连接失败。

这是非阻塞问题,因为现有调用方不会传入它,默认生产行为也没有变化。长期来看,更适合通过 package-internal 测试 seam 或 Host/transport fixture 观察 probe。如果保留为公开 diagnostics hook,则 observer 异常不应影响连接健康。

还有一个小的测试作用域建议:Agent Graph 测试只给最终执行 agent.graph.stop 的 TUI connection 传入 probe observer。这样前面其他 connection 上的慢查询就不可能提前满足共享 counter。

本轮修改有效,没有发现阻塞问题。Approve。

Review follow-up (non-blocking P3): onLivenessProbe is a diagnostics hook,
so an exception thrown by the observer is now swallowed instead of running
inside the probe success chain where it would fail the connection it is
watching; the input doc says so explicitly.
Also scopes the Agent-graph test's observer to the one connection that
issues agent.graph.stop, so no other connection's slow query could ever
satisfy the shared probe counter.
@Astro-Han
Astro-Han merged commit 1823c9d into apache:mainAug 8, 2026
2 checks passed
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both follow-ups applied in fc78317:

  • Observer isolation: onLivenessProbe exceptions are now swallowed at the call site — a diagnostics hook can no longer fail the connection it is watching — and the input doc states that contract explicitly. On the longer-term point: agreed a package-internal seam would be cleaner; if a second consumer of probe observability ever appears, that's the moment to move it behind a Host/transport fixture rather than the public input.
  • Test scoping: the Agent-graph observer is now wired only onto the final TUI connection that issues agent.graph.stop, so no other connection's slow query can satisfy the shared counter.

Suite green after a clean build; typecheck and biome pass.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@UncertaintyDeterminesYou4ndMe@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

test(runtime-host): drive liveness-crossing waits from an injected probe cadence - #2450

Merged
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection
Aug 8, 2026
Merged

test(runtime-host): drive liveness-crossing waits from an injected probe cadence#2450
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection

Conversation

@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor

Part of #2389 (Runtime Host workspace; the Pi TUI workspace landed as #2448, desktop e2e and alignment audit follow separately).

What changed

Two tests held work pending for a fixed 2.1s purely to outlive the client connection's hardcoded DEFAULT_LIVENESS_INTERVAL_MS = 2_000:

  • agent-graph-two-client-uds.test.tsFakeAgentGraphAuthority.stop() slept 2.1s (the fake-shutdown case named in the issue) so the pending agent.graph.stop request would cross a liveness probe cycle.
  • host-kernel.test.ts (slow domain work preserves multiplexed requests and retires only explicit deadlines) — the test slept 2.1s while holding an admitted request open before releasing its gate.

Both waits existed to prove the #2392 contract: liveness probes never retire a request that has no explicit deadline. The 2s value is incidental to that contract — what matters is that at least one probe fires while the request is pending.

The probe interval is now injectable: ConnectRuntimeHostInput.livenessIntervalMs (validated by the same requireTimeout as the existing timeout options, default unchanged at 2s), threaded through connectResolvedRuntimeHost into the connection's #scheduleLivenessCheck. Both tests inject a 100ms cadence and derive their waits from it (2 cycles + margin) — the wait is now measured in a unit the test controls instead of guessed against a constant it cannot see.

Retained contracts

  • Probe-crossing semantics are exercised identically: with the 100ms cadence, two probes fire while the request is pending, and the request still completes with its real result.
  • The probe timeout (DEFAULT_LIVENESS_TIMEOUT_MS), handshake/connect deadlines, and the explicit 50ms read_timeout case in the host-kernel test are untouched.
  • The remaining short sleeps in this workspace are all poll intervals inside bounded wait loops (already observable completion) — audited and left as-is.

Production surface

One additive optional field on ConnectRuntimeHostInput; passing nothing preserves today's behavior exactly.

Timing (local, node --test)

TestBeforeAfter
two UDS Clients query and control one Agent graph…~2.5s0.36s
slow domain work preserves multiplexed requests…~2.6s0.38s

Full workspace suite after a clean build: 734/734 pass.

…obe cadence
Part of apache#2389. Two tests held work pending for a fixed 2.1s purely to outlive
the client connection's hardcoded 2s liveness interval:
- the Agent-graph fake authority's slow stop() (the fake-shutdown case named
in the issue), proving agent.graph.stop survives probe cycles;
- host-kernel's slow-domain-work test, holding an admitted request pending
across a probe before releasing it.
The interval is now injectable (ConnectRuntimeHostInput.livenessIntervalMs,
default unchanged at 2s), so both tests measure their probe-crossing waits in
an injected 100ms unit instead of wall-clock guessing against a constant they
cannot see. The waits are derived (2 cycles + margin), not scheduler-load
guesses, and the contract — probes never retire a request that has no
explicit deadline (apache#2392) — is exercised identically.
The remaining short sleeps in this workspace are all poll intervals inside
bounded wait loops, which are already observable completion; they are
untouched.
Focused timing: the two affected tests drop from ~2.5s each to ~0.37s each
(node --test, local). Full workspace suite: 734/734 pass after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The injected cadence matches the existing connectTimeoutMs/handshakeTimeoutMs pattern, default behavior is unchanged (2s), and both tests are genuinely faster (430/450ms). The assertions still bite: if #2392 regresses, the pending request gets retired and the deepEqual fails.

One gap: the tests can't tell "probes fired and the request survived" from "probes never fired". If a future change stops using livenessIntervalMs in the constructor (types still compile), every assertion passes with zero probes in the window. A probe counter on the server side, asserted before release, would make the premise falsifiable. Cheap to add.

Nit: livenessIntervalMs is validated in the constructor, after connect + handshake, so a bad value surfaces as "handshake_failed" instead of a config error; the other timeouts validate up front. Approving.

…dence took
Review follow-up on the injected liveness cadence: the tests could not tell
"probes fired and the request survived" from "probes never fired" — if a
future change stopped threading livenessIntervalMs, every assertion would
still pass with zero probes in the window.
ConnectRuntimeHostInput gains onLivenessProbe, invoked after a probe
round-trips and validates its Host Epoch. Both tests now gate on two observed
probe round-trips while their long-lived request is pending — the host-kernel
test releases its admitted request only after the crossing (bounded, loud
timeout if the cadence stops taking effect), and the Agent-graph fake's stop()
awaits the crossing instead of a derived sleep, making the ordering causal
with no fixed timing at all.
Also validates livenessIntervalMs up front in connectResolvedRuntimeHost
alongside the other connect timeouts, so a bad value is a config error
instead of surfacing as handshake_failed.
Both tests get faster again (~0.29s / ~0.33s); 734/734 pass after a clean
build.
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both points addressed in bad4d15:

  • Falsifiable premise: ConnectRuntimeHostInput.onLivenessProbe fires after a probe round-trips and validates its Host Epoch. The host-kernel test now releases the admitted request only after two observed probe round-trips (bounded — a cadence that stops taking effect times out loudly instead of vacuously passing), and the Agent-graph fake's stop() awaits that same crossing instead of a derived sleep, so the ordering is causal with no fixed timing left at all.
  • Nit: livenessIntervalMs is now validated up front in connectResolvedRuntimeHost alongside connectTimeoutMs/handshakeTimeoutMs, so a bad value surfaces as a config error, not handshake_failed.

Both tests got faster again (~0.29s / ~0.33s); 734/734 after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. Both review points are substantively addressed:

  • the tests now wait for observed, epoch-validated probe round-trips instead of assuming the injected cadence fired;
  • livenessIntervalMs is validated before transport and handshake work, consistently with the existing timeout options.

The resulting tests are faster and their probe-crossing premise is now falsifiable.

P3 — Test observability is exposed through the public client input.

onLivenessProbe is documented as a test hook but is now part of the exported ConnectRuntimeHostInput. Its callback also runs inside the probe success chain, so an observer exception can fail the connection.

This is non-blocking because existing callers do not pass it and default production behavior is unchanged. Longer term, I would prefer observing probes through a package-internal test seam or Host/transport fixture. If the callback remains a public diagnostics hook, its exceptions should be isolated from connection health.

Small test-scoping suggestion: in the Agent Graph test, pass the probe observer only to the final TUI connection that issues agent.graph.stop. That makes it impossible for an earlier slow query on another connection to satisfy the shared counter.

The requested follow-up is effective, and I found no blocking issue. Approved.

简体中文

感谢继续修改。上一轮的两个问题都已经得到实质解决:

  • 测试现在等待实际观察到、并完成 Host Epoch 校验的 probe round-trip,不再假设注入的 cadence 已经触发;
  • livenessIntervalMs 现在与现有 timeout 配置一样,在 transport 和 handshake 之前完成校验。

测试变得更快,而且 probe-crossing 前提现在可以被证伪。

P3 — 测试观测能力进入了公开 client input。

onLivenessProbe 的文档明确说明它是 test hook,但它现在属于公开导出的 ConnectRuntimeHostInput。callback 还运行在 probe success chain 内,因此 observer 抛出的异常可能导致连接失败。

这是非阻塞问题,因为现有调用方不会传入它,默认生产行为也没有变化。长期来看,更适合通过 package-internal 测试 seam 或 Host/transport fixture 观察 probe。如果保留为公开 diagnostics hook,则 observer 异常不应影响连接健康。

还有一个小的测试作用域建议:Agent Graph 测试只给最终执行 agent.graph.stop 的 TUI connection 传入 probe observer。这样前面其他 connection 上的慢查询就不可能提前满足共享 counter。

本轮修改有效,没有发现阻塞问题。Approve。

Review follow-up (non-blocking P3): onLivenessProbe is a diagnostics hook,
so an exception thrown by the observer is now swallowed instead of running
inside the probe success chain where it would fail the connection it is
watching; the input doc says so explicitly.
Also scopes the Agent-graph test's observer to the one connection that
issues agent.graph.stop, so no other connection's slow query could ever
satisfy the shared probe counter.
@Astro-Han
Astro-Han merged commit 1823c9d into apache:mainAug 8, 2026
2 checks passed
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both follow-ups applied in fc78317:

  • Observer isolation: onLivenessProbe exceptions are now swallowed at the call site — a diagnostics hook can no longer fail the connection it is watching — and the input doc states that contract explicitly. On the longer-term point: agreed a package-internal seam would be cleaner; if a second consumer of probe observability ever appears, that's the moment to move it behind a Host/transport fixture rather than the public input.
  • Test scoping: the Agent-graph observer is now wired only onto the final TUI connection that issues agent.graph.stop, so no other connection's slow query can satisfy the shared counter.

Suite green after a clean build; typecheck and biome pass.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@UncertaintyDeterminesYou4ndMe@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

test(runtime-host): drive liveness-crossing waits from an injected probe cadence - #2450

Merged
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection
Aug 8, 2026
Merged

test(runtime-host): drive liveness-crossing waits from an injected probe cadence#2450
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection

Conversation

@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor

Part of #2389 (Runtime Host workspace; the Pi TUI workspace landed as #2448, desktop e2e and alignment audit follow separately).

What changed

Two tests held work pending for a fixed 2.1s purely to outlive the client connection's hardcoded DEFAULT_LIVENESS_INTERVAL_MS = 2_000:

  • agent-graph-two-client-uds.test.tsFakeAgentGraphAuthority.stop() slept 2.1s (the fake-shutdown case named in the issue) so the pending agent.graph.stop request would cross a liveness probe cycle.
  • host-kernel.test.ts (slow domain work preserves multiplexed requests and retires only explicit deadlines) — the test slept 2.1s while holding an admitted request open before releasing its gate.

Both waits existed to prove the #2392 contract: liveness probes never retire a request that has no explicit deadline. The 2s value is incidental to that contract — what matters is that at least one probe fires while the request is pending.

The probe interval is now injectable: ConnectRuntimeHostInput.livenessIntervalMs (validated by the same requireTimeout as the existing timeout options, default unchanged at 2s), threaded through connectResolvedRuntimeHost into the connection's #scheduleLivenessCheck. Both tests inject a 100ms cadence and derive their waits from it (2 cycles + margin) — the wait is now measured in a unit the test controls instead of guessed against a constant it cannot see.

Retained contracts

  • Probe-crossing semantics are exercised identically: with the 100ms cadence, two probes fire while the request is pending, and the request still completes with its real result.
  • The probe timeout (DEFAULT_LIVENESS_TIMEOUT_MS), handshake/connect deadlines, and the explicit 50ms read_timeout case in the host-kernel test are untouched.
  • The remaining short sleeps in this workspace are all poll intervals inside bounded wait loops (already observable completion) — audited and left as-is.

Production surface

One additive optional field on ConnectRuntimeHostInput; passing nothing preserves today's behavior exactly.

Timing (local, node --test)

TestBeforeAfter
two UDS Clients query and control one Agent graph…~2.5s0.36s
slow domain work preserves multiplexed requests…~2.6s0.38s

Full workspace suite after a clean build: 734/734 pass.

…obe cadence
Part of apache#2389. Two tests held work pending for a fixed 2.1s purely to outlive
the client connection's hardcoded 2s liveness interval:
- the Agent-graph fake authority's slow stop() (the fake-shutdown case named
in the issue), proving agent.graph.stop survives probe cycles;
- host-kernel's slow-domain-work test, holding an admitted request pending
across a probe before releasing it.
The interval is now injectable (ConnectRuntimeHostInput.livenessIntervalMs,
default unchanged at 2s), so both tests measure their probe-crossing waits in
an injected 100ms unit instead of wall-clock guessing against a constant they
cannot see. The waits are derived (2 cycles + margin), not scheduler-load
guesses, and the contract — probes never retire a request that has no
explicit deadline (apache#2392) — is exercised identically.
The remaining short sleeps in this workspace are all poll intervals inside
bounded wait loops, which are already observable completion; they are
untouched.
Focused timing: the two affected tests drop from ~2.5s each to ~0.37s each
(node --test, local). Full workspace suite: 734/734 pass after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The injected cadence matches the existing connectTimeoutMs/handshakeTimeoutMs pattern, default behavior is unchanged (2s), and both tests are genuinely faster (430/450ms). The assertions still bite: if #2392 regresses, the pending request gets retired and the deepEqual fails.

One gap: the tests can't tell "probes fired and the request survived" from "probes never fired". If a future change stops using livenessIntervalMs in the constructor (types still compile), every assertion passes with zero probes in the window. A probe counter on the server side, asserted before release, would make the premise falsifiable. Cheap to add.

Nit: livenessIntervalMs is validated in the constructor, after connect + handshake, so a bad value surfaces as "handshake_failed" instead of a config error; the other timeouts validate up front. Approving.

…dence took
Review follow-up on the injected liveness cadence: the tests could not tell
"probes fired and the request survived" from "probes never fired" — if a
future change stopped threading livenessIntervalMs, every assertion would
still pass with zero probes in the window.
ConnectRuntimeHostInput gains onLivenessProbe, invoked after a probe
round-trips and validates its Host Epoch. Both tests now gate on two observed
probe round-trips while their long-lived request is pending — the host-kernel
test releases its admitted request only after the crossing (bounded, loud
timeout if the cadence stops taking effect), and the Agent-graph fake's stop()
awaits the crossing instead of a derived sleep, making the ordering causal
with no fixed timing at all.
Also validates livenessIntervalMs up front in connectResolvedRuntimeHost
alongside the other connect timeouts, so a bad value is a config error
instead of surfacing as handshake_failed.
Both tests get faster again (~0.29s / ~0.33s); 734/734 pass after a clean
build.
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both points addressed in bad4d15:

  • Falsifiable premise: ConnectRuntimeHostInput.onLivenessProbe fires after a probe round-trips and validates its Host Epoch. The host-kernel test now releases the admitted request only after two observed probe round-trips (bounded — a cadence that stops taking effect times out loudly instead of vacuously passing), and the Agent-graph fake's stop() awaits that same crossing instead of a derived sleep, so the ordering is causal with no fixed timing left at all.
  • Nit: livenessIntervalMs is now validated up front in connectResolvedRuntimeHost alongside connectTimeoutMs/handshakeTimeoutMs, so a bad value surfaces as a config error, not handshake_failed.

Both tests got faster again (~0.29s / ~0.33s); 734/734 after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. Both review points are substantively addressed:

  • the tests now wait for observed, epoch-validated probe round-trips instead of assuming the injected cadence fired;
  • livenessIntervalMs is validated before transport and handshake work, consistently with the existing timeout options.

The resulting tests are faster and their probe-crossing premise is now falsifiable.

P3 — Test observability is exposed through the public client input.

onLivenessProbe is documented as a test hook but is now part of the exported ConnectRuntimeHostInput. Its callback also runs inside the probe success chain, so an observer exception can fail the connection.

This is non-blocking because existing callers do not pass it and default production behavior is unchanged. Longer term, I would prefer observing probes through a package-internal test seam or Host/transport fixture. If the callback remains a public diagnostics hook, its exceptions should be isolated from connection health.

Small test-scoping suggestion: in the Agent Graph test, pass the probe observer only to the final TUI connection that issues agent.graph.stop. That makes it impossible for an earlier slow query on another connection to satisfy the shared counter.

The requested follow-up is effective, and I found no blocking issue. Approved.

简体中文

感谢继续修改。上一轮的两个问题都已经得到实质解决:

  • 测试现在等待实际观察到、并完成 Host Epoch 校验的 probe round-trip,不再假设注入的 cadence 已经触发;
  • livenessIntervalMs 现在与现有 timeout 配置一样,在 transport 和 handshake 之前完成校验。

测试变得更快,而且 probe-crossing 前提现在可以被证伪。

P3 — 测试观测能力进入了公开 client input。

onLivenessProbe 的文档明确说明它是 test hook,但它现在属于公开导出的 ConnectRuntimeHostInput。callback 还运行在 probe success chain 内,因此 observer 抛出的异常可能导致连接失败。

这是非阻塞问题,因为现有调用方不会传入它,默认生产行为也没有变化。长期来看,更适合通过 package-internal 测试 seam 或 Host/transport fixture 观察 probe。如果保留为公开 diagnostics hook,则 observer 异常不应影响连接健康。

还有一个小的测试作用域建议:Agent Graph 测试只给最终执行 agent.graph.stop 的 TUI connection 传入 probe observer。这样前面其他 connection 上的慢查询就不可能提前满足共享 counter。

本轮修改有效,没有发现阻塞问题。Approve。

Review follow-up (non-blocking P3): onLivenessProbe is a diagnostics hook,
so an exception thrown by the observer is now swallowed instead of running
inside the probe success chain where it would fail the connection it is
watching; the input doc says so explicitly.
Also scopes the Agent-graph test's observer to the one connection that
issues agent.graph.stop, so no other connection's slow query could ever
satisfy the shared probe counter.
@Astro-Han
Astro-Han merged commit 1823c9d into apache:mainAug 8, 2026
2 checks passed
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both follow-ups applied in fc78317:

  • Observer isolation: onLivenessProbe exceptions are now swallowed at the call site — a diagnostics hook can no longer fail the connection it is watching — and the input doc states that contract explicitly. On the longer-term point: agreed a package-internal seam would be cleaner; if a second consumer of probe observability ever appears, that's the moment to move it behind a Host/transport fixture rather than the public input.
  • Test scoping: the Agent-graph observer is now wired only onto the final TUI connection that issues agent.graph.stop, so no other connection's slow query can satisfy the shared counter.

Suite green after a clean build; typecheck and biome pass.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@UncertaintyDeterminesYou4ndMe@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

test(runtime-host): drive liveness-crossing waits from an injected probe cadence - #2450

Merged
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection
Aug 8, 2026
Merged

test(runtime-host): drive liveness-crossing waits from an injected probe cadence#2450
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection

Conversation

@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor

Part of #2389 (Runtime Host workspace; the Pi TUI workspace landed as #2448, desktop e2e and alignment audit follow separately).

What changed

Two tests held work pending for a fixed 2.1s purely to outlive the client connection's hardcoded DEFAULT_LIVENESS_INTERVAL_MS = 2_000:

  • agent-graph-two-client-uds.test.tsFakeAgentGraphAuthority.stop() slept 2.1s (the fake-shutdown case named in the issue) so the pending agent.graph.stop request would cross a liveness probe cycle.
  • host-kernel.test.ts (slow domain work preserves multiplexed requests and retires only explicit deadlines) — the test slept 2.1s while holding an admitted request open before releasing its gate.

Both waits existed to prove the #2392 contract: liveness probes never retire a request that has no explicit deadline. The 2s value is incidental to that contract — what matters is that at least one probe fires while the request is pending.

The probe interval is now injectable: ConnectRuntimeHostInput.livenessIntervalMs (validated by the same requireTimeout as the existing timeout options, default unchanged at 2s), threaded through connectResolvedRuntimeHost into the connection's #scheduleLivenessCheck. Both tests inject a 100ms cadence and derive their waits from it (2 cycles + margin) — the wait is now measured in a unit the test controls instead of guessed against a constant it cannot see.

Retained contracts

  • Probe-crossing semantics are exercised identically: with the 100ms cadence, two probes fire while the request is pending, and the request still completes with its real result.
  • The probe timeout (DEFAULT_LIVENESS_TIMEOUT_MS), handshake/connect deadlines, and the explicit 50ms read_timeout case in the host-kernel test are untouched.
  • The remaining short sleeps in this workspace are all poll intervals inside bounded wait loops (already observable completion) — audited and left as-is.

Production surface

One additive optional field on ConnectRuntimeHostInput; passing nothing preserves today's behavior exactly.

Timing (local, node --test)

TestBeforeAfter
two UDS Clients query and control one Agent graph…~2.5s0.36s
slow domain work preserves multiplexed requests…~2.6s0.38s

Full workspace suite after a clean build: 734/734 pass.

…obe cadence
Part of apache#2389. Two tests held work pending for a fixed 2.1s purely to outlive
the client connection's hardcoded 2s liveness interval:
- the Agent-graph fake authority's slow stop() (the fake-shutdown case named
in the issue), proving agent.graph.stop survives probe cycles;
- host-kernel's slow-domain-work test, holding an admitted request pending
across a probe before releasing it.
The interval is now injectable (ConnectRuntimeHostInput.livenessIntervalMs,
default unchanged at 2s), so both tests measure their probe-crossing waits in
an injected 100ms unit instead of wall-clock guessing against a constant they
cannot see. The waits are derived (2 cycles + margin), not scheduler-load
guesses, and the contract — probes never retire a request that has no
explicit deadline (apache#2392) — is exercised identically.
The remaining short sleeps in this workspace are all poll intervals inside
bounded wait loops, which are already observable completion; they are
untouched.
Focused timing: the two affected tests drop from ~2.5s each to ~0.37s each
(node --test, local). Full workspace suite: 734/734 pass after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The injected cadence matches the existing connectTimeoutMs/handshakeTimeoutMs pattern, default behavior is unchanged (2s), and both tests are genuinely faster (430/450ms). The assertions still bite: if #2392 regresses, the pending request gets retired and the deepEqual fails.

One gap: the tests can't tell "probes fired and the request survived" from "probes never fired". If a future change stops using livenessIntervalMs in the constructor (types still compile), every assertion passes with zero probes in the window. A probe counter on the server side, asserted before release, would make the premise falsifiable. Cheap to add.

Nit: livenessIntervalMs is validated in the constructor, after connect + handshake, so a bad value surfaces as "handshake_failed" instead of a config error; the other timeouts validate up front. Approving.

…dence took
Review follow-up on the injected liveness cadence: the tests could not tell
"probes fired and the request survived" from "probes never fired" — if a
future change stopped threading livenessIntervalMs, every assertion would
still pass with zero probes in the window.
ConnectRuntimeHostInput gains onLivenessProbe, invoked after a probe
round-trips and validates its Host Epoch. Both tests now gate on two observed
probe round-trips while their long-lived request is pending — the host-kernel
test releases its admitted request only after the crossing (bounded, loud
timeout if the cadence stops taking effect), and the Agent-graph fake's stop()
awaits the crossing instead of a derived sleep, making the ordering causal
with no fixed timing at all.
Also validates livenessIntervalMs up front in connectResolvedRuntimeHost
alongside the other connect timeouts, so a bad value is a config error
instead of surfacing as handshake_failed.
Both tests get faster again (~0.29s / ~0.33s); 734/734 pass after a clean
build.
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both points addressed in bad4d15:

  • Falsifiable premise: ConnectRuntimeHostInput.onLivenessProbe fires after a probe round-trips and validates its Host Epoch. The host-kernel test now releases the admitted request only after two observed probe round-trips (bounded — a cadence that stops taking effect times out loudly instead of vacuously passing), and the Agent-graph fake's stop() awaits that same crossing instead of a derived sleep, so the ordering is causal with no fixed timing left at all.
  • Nit: livenessIntervalMs is now validated up front in connectResolvedRuntimeHost alongside connectTimeoutMs/handshakeTimeoutMs, so a bad value surfaces as a config error, not handshake_failed.

Both tests got faster again (~0.29s / ~0.33s); 734/734 after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. Both review points are substantively addressed:

  • the tests now wait for observed, epoch-validated probe round-trips instead of assuming the injected cadence fired;
  • livenessIntervalMs is validated before transport and handshake work, consistently with the existing timeout options.

The resulting tests are faster and their probe-crossing premise is now falsifiable.

P3 — Test observability is exposed through the public client input.

onLivenessProbe is documented as a test hook but is now part of the exported ConnectRuntimeHostInput. Its callback also runs inside the probe success chain, so an observer exception can fail the connection.

This is non-blocking because existing callers do not pass it and default production behavior is unchanged. Longer term, I would prefer observing probes through a package-internal test seam or Host/transport fixture. If the callback remains a public diagnostics hook, its exceptions should be isolated from connection health.

Small test-scoping suggestion: in the Agent Graph test, pass the probe observer only to the final TUI connection that issues agent.graph.stop. That makes it impossible for an earlier slow query on another connection to satisfy the shared counter.

The requested follow-up is effective, and I found no blocking issue. Approved.

简体中文

感谢继续修改。上一轮的两个问题都已经得到实质解决:

  • 测试现在等待实际观察到、并完成 Host Epoch 校验的 probe round-trip,不再假设注入的 cadence 已经触发;
  • livenessIntervalMs 现在与现有 timeout 配置一样,在 transport 和 handshake 之前完成校验。

测试变得更快,而且 probe-crossing 前提现在可以被证伪。

P3 — 测试观测能力进入了公开 client input。

onLivenessProbe 的文档明确说明它是 test hook,但它现在属于公开导出的 ConnectRuntimeHostInput。callback 还运行在 probe success chain 内,因此 observer 抛出的异常可能导致连接失败。

这是非阻塞问题,因为现有调用方不会传入它,默认生产行为也没有变化。长期来看,更适合通过 package-internal 测试 seam 或 Host/transport fixture 观察 probe。如果保留为公开 diagnostics hook,则 observer 异常不应影响连接健康。

还有一个小的测试作用域建议:Agent Graph 测试只给最终执行 agent.graph.stop 的 TUI connection 传入 probe observer。这样前面其他 connection 上的慢查询就不可能提前满足共享 counter。

本轮修改有效,没有发现阻塞问题。Approve。

Review follow-up (non-blocking P3): onLivenessProbe is a diagnostics hook,
so an exception thrown by the observer is now swallowed instead of running
inside the probe success chain where it would fail the connection it is
watching; the input doc says so explicitly.
Also scopes the Agent-graph test's observer to the one connection that
issues agent.graph.stop, so no other connection's slow query could ever
satisfy the shared probe counter.
@Astro-Han
Astro-Han merged commit 1823c9d into apache:mainAug 8, 2026
2 checks passed
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both follow-ups applied in fc78317:

  • Observer isolation: onLivenessProbe exceptions are now swallowed at the call site — a diagnostics hook can no longer fail the connection it is watching — and the input doc states that contract explicitly. On the longer-term point: agreed a package-internal seam would be cleaner; if a second consumer of probe observability ever appears, that's the moment to move it behind a Host/transport fixture rather than the public input.
  • Test scoping: the Agent-graph observer is now wired only onto the final TUI connection that issues agent.graph.stop, so no other connection's slow query can satisfy the shared counter.

Suite green after a clean build; typecheck and biome pass.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@UncertaintyDeterminesYou4ndMe@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

test(runtime-host): drive liveness-crossing waits from an injected probe cadence - #2450

Merged
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection
Aug 8, 2026
Merged

test(runtime-host): drive liveness-crossing waits from an injected probe cadence#2450
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection

Conversation

@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor

Part of #2389 (Runtime Host workspace; the Pi TUI workspace landed as #2448, desktop e2e and alignment audit follow separately).

What changed

Two tests held work pending for a fixed 2.1s purely to outlive the client connection's hardcoded DEFAULT_LIVENESS_INTERVAL_MS = 2_000:

  • agent-graph-two-client-uds.test.tsFakeAgentGraphAuthority.stop() slept 2.1s (the fake-shutdown case named in the issue) so the pending agent.graph.stop request would cross a liveness probe cycle.
  • host-kernel.test.ts (slow domain work preserves multiplexed requests and retires only explicit deadlines) — the test slept 2.1s while holding an admitted request open before releasing its gate.

Both waits existed to prove the #2392 contract: liveness probes never retire a request that has no explicit deadline. The 2s value is incidental to that contract — what matters is that at least one probe fires while the request is pending.

The probe interval is now injectable: ConnectRuntimeHostInput.livenessIntervalMs (validated by the same requireTimeout as the existing timeout options, default unchanged at 2s), threaded through connectResolvedRuntimeHost into the connection's #scheduleLivenessCheck. Both tests inject a 100ms cadence and derive their waits from it (2 cycles + margin) — the wait is now measured in a unit the test controls instead of guessed against a constant it cannot see.

Retained contracts

  • Probe-crossing semantics are exercised identically: with the 100ms cadence, two probes fire while the request is pending, and the request still completes with its real result.
  • The probe timeout (DEFAULT_LIVENESS_TIMEOUT_MS), handshake/connect deadlines, and the explicit 50ms read_timeout case in the host-kernel test are untouched.
  • The remaining short sleeps in this workspace are all poll intervals inside bounded wait loops (already observable completion) — audited and left as-is.

Production surface

One additive optional field on ConnectRuntimeHostInput; passing nothing preserves today's behavior exactly.

Timing (local, node --test)

TestBeforeAfter
two UDS Clients query and control one Agent graph…~2.5s0.36s
slow domain work preserves multiplexed requests…~2.6s0.38s

Full workspace suite after a clean build: 734/734 pass.

…obe cadence
Part of apache#2389. Two tests held work pending for a fixed 2.1s purely to outlive
the client connection's hardcoded 2s liveness interval:
- the Agent-graph fake authority's slow stop() (the fake-shutdown case named
in the issue), proving agent.graph.stop survives probe cycles;
- host-kernel's slow-domain-work test, holding an admitted request pending
across a probe before releasing it.
The interval is now injectable (ConnectRuntimeHostInput.livenessIntervalMs,
default unchanged at 2s), so both tests measure their probe-crossing waits in
an injected 100ms unit instead of wall-clock guessing against a constant they
cannot see. The waits are derived (2 cycles + margin), not scheduler-load
guesses, and the contract — probes never retire a request that has no
explicit deadline (apache#2392) — is exercised identically.
The remaining short sleeps in this workspace are all poll intervals inside
bounded wait loops, which are already observable completion; they are
untouched.
Focused timing: the two affected tests drop from ~2.5s each to ~0.37s each
(node --test, local). Full workspace suite: 734/734 pass after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The injected cadence matches the existing connectTimeoutMs/handshakeTimeoutMs pattern, default behavior is unchanged (2s), and both tests are genuinely faster (430/450ms). The assertions still bite: if #2392 regresses, the pending request gets retired and the deepEqual fails.

One gap: the tests can't tell "probes fired and the request survived" from "probes never fired". If a future change stops using livenessIntervalMs in the constructor (types still compile), every assertion passes with zero probes in the window. A probe counter on the server side, asserted before release, would make the premise falsifiable. Cheap to add.

Nit: livenessIntervalMs is validated in the constructor, after connect + handshake, so a bad value surfaces as "handshake_failed" instead of a config error; the other timeouts validate up front. Approving.

…dence took
Review follow-up on the injected liveness cadence: the tests could not tell
"probes fired and the request survived" from "probes never fired" — if a
future change stopped threading livenessIntervalMs, every assertion would
still pass with zero probes in the window.
ConnectRuntimeHostInput gains onLivenessProbe, invoked after a probe
round-trips and validates its Host Epoch. Both tests now gate on two observed
probe round-trips while their long-lived request is pending — the host-kernel
test releases its admitted request only after the crossing (bounded, loud
timeout if the cadence stops taking effect), and the Agent-graph fake's stop()
awaits the crossing instead of a derived sleep, making the ordering causal
with no fixed timing at all.
Also validates livenessIntervalMs up front in connectResolvedRuntimeHost
alongside the other connect timeouts, so a bad value is a config error
instead of surfacing as handshake_failed.
Both tests get faster again (~0.29s / ~0.33s); 734/734 pass after a clean
build.
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both points addressed in bad4d15:

  • Falsifiable premise: ConnectRuntimeHostInput.onLivenessProbe fires after a probe round-trips and validates its Host Epoch. The host-kernel test now releases the admitted request only after two observed probe round-trips (bounded — a cadence that stops taking effect times out loudly instead of vacuously passing), and the Agent-graph fake's stop() awaits that same crossing instead of a derived sleep, so the ordering is causal with no fixed timing left at all.
  • Nit: livenessIntervalMs is now validated up front in connectResolvedRuntimeHost alongside connectTimeoutMs/handshakeTimeoutMs, so a bad value surfaces as a config error, not handshake_failed.

Both tests got faster again (~0.29s / ~0.33s); 734/734 after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. Both review points are substantively addressed:

  • the tests now wait for observed, epoch-validated probe round-trips instead of assuming the injected cadence fired;
  • livenessIntervalMs is validated before transport and handshake work, consistently with the existing timeout options.

The resulting tests are faster and their probe-crossing premise is now falsifiable.

P3 — Test observability is exposed through the public client input.

onLivenessProbe is documented as a test hook but is now part of the exported ConnectRuntimeHostInput. Its callback also runs inside the probe success chain, so an observer exception can fail the connection.

This is non-blocking because existing callers do not pass it and default production behavior is unchanged. Longer term, I would prefer observing probes through a package-internal test seam or Host/transport fixture. If the callback remains a public diagnostics hook, its exceptions should be isolated from connection health.

Small test-scoping suggestion: in the Agent Graph test, pass the probe observer only to the final TUI connection that issues agent.graph.stop. That makes it impossible for an earlier slow query on another connection to satisfy the shared counter.

The requested follow-up is effective, and I found no blocking issue. Approved.

简体中文

感谢继续修改。上一轮的两个问题都已经得到实质解决:

  • 测试现在等待实际观察到、并完成 Host Epoch 校验的 probe round-trip,不再假设注入的 cadence 已经触发;
  • livenessIntervalMs 现在与现有 timeout 配置一样,在 transport 和 handshake 之前完成校验。

测试变得更快,而且 probe-crossing 前提现在可以被证伪。

P3 — 测试观测能力进入了公开 client input。

onLivenessProbe 的文档明确说明它是 test hook,但它现在属于公开导出的 ConnectRuntimeHostInput。callback 还运行在 probe success chain 内,因此 observer 抛出的异常可能导致连接失败。

这是非阻塞问题,因为现有调用方不会传入它,默认生产行为也没有变化。长期来看,更适合通过 package-internal 测试 seam 或 Host/transport fixture 观察 probe。如果保留为公开 diagnostics hook,则 observer 异常不应影响连接健康。

还有一个小的测试作用域建议:Agent Graph 测试只给最终执行 agent.graph.stop 的 TUI connection 传入 probe observer。这样前面其他 connection 上的慢查询就不可能提前满足共享 counter。

本轮修改有效,没有发现阻塞问题。Approve。

Review follow-up (non-blocking P3): onLivenessProbe is a diagnostics hook,
so an exception thrown by the observer is now swallowed instead of running
inside the probe success chain where it would fail the connection it is
watching; the input doc says so explicitly.
Also scopes the Agent-graph test's observer to the one connection that
issues agent.graph.stop, so no other connection's slow query could ever
satisfy the shared probe counter.
@Astro-Han
Astro-Han merged commit 1823c9d into apache:mainAug 8, 2026
2 checks passed
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both follow-ups applied in fc78317:

  • Observer isolation: onLivenessProbe exceptions are now swallowed at the call site — a diagnostics hook can no longer fail the connection it is watching — and the input doc states that contract explicitly. On the longer-term point: agreed a package-internal seam would be cleaner; if a second consumer of probe observability ever appears, that's the moment to move it behind a Host/transport fixture rather than the public input.
  • Test scoping: the Agent-graph observer is now wired only onto the final TUI connection that issues agent.graph.stop, so no other connection's slow query can satisfy the shared counter.

Suite green after a clean build; typecheck and biome pass.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@UncertaintyDeterminesYou4ndMe@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

test(runtime-host): drive liveness-crossing waits from an injected probe cadence - #2450

Merged
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection
Aug 8, 2026
Merged

test(runtime-host): drive liveness-crossing waits from an injected probe cadence#2450
Astro-Han merged 3 commits into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:perf/2389-runtime-host-liveness-injection

Conversation

@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor

Part of #2389 (Runtime Host workspace; the Pi TUI workspace landed as #2448, desktop e2e and alignment audit follow separately).

What changed

Two tests held work pending for a fixed 2.1s purely to outlive the client connection's hardcoded DEFAULT_LIVENESS_INTERVAL_MS = 2_000:

  • agent-graph-two-client-uds.test.tsFakeAgentGraphAuthority.stop() slept 2.1s (the fake-shutdown case named in the issue) so the pending agent.graph.stop request would cross a liveness probe cycle.
  • host-kernel.test.ts (slow domain work preserves multiplexed requests and retires only explicit deadlines) — the test slept 2.1s while holding an admitted request open before releasing its gate.

Both waits existed to prove the #2392 contract: liveness probes never retire a request that has no explicit deadline. The 2s value is incidental to that contract — what matters is that at least one probe fires while the request is pending.

The probe interval is now injectable: ConnectRuntimeHostInput.livenessIntervalMs (validated by the same requireTimeout as the existing timeout options, default unchanged at 2s), threaded through connectResolvedRuntimeHost into the connection's #scheduleLivenessCheck. Both tests inject a 100ms cadence and derive their waits from it (2 cycles + margin) — the wait is now measured in a unit the test controls instead of guessed against a constant it cannot see.

Retained contracts

  • Probe-crossing semantics are exercised identically: with the 100ms cadence, two probes fire while the request is pending, and the request still completes with its real result.
  • The probe timeout (DEFAULT_LIVENESS_TIMEOUT_MS), handshake/connect deadlines, and the explicit 50ms read_timeout case in the host-kernel test are untouched.
  • The remaining short sleeps in this workspace are all poll intervals inside bounded wait loops (already observable completion) — audited and left as-is.

Production surface

One additive optional field on ConnectRuntimeHostInput; passing nothing preserves today's behavior exactly.

Timing (local, node --test)

TestBeforeAfter
two UDS Clients query and control one Agent graph…~2.5s0.36s
slow domain work preserves multiplexed requests…~2.6s0.38s

Full workspace suite after a clean build: 734/734 pass.

…obe cadence
Part of apache#2389. Two tests held work pending for a fixed 2.1s purely to outlive
the client connection's hardcoded 2s liveness interval:
- the Agent-graph fake authority's slow stop() (the fake-shutdown case named
in the issue), proving agent.graph.stop survives probe cycles;
- host-kernel's slow-domain-work test, holding an admitted request pending
across a probe before releasing it.
The interval is now injectable (ConnectRuntimeHostInput.livenessIntervalMs,
default unchanged at 2s), so both tests measure their probe-crossing waits in
an injected 100ms unit instead of wall-clock guessing against a constant they
cannot see. The waits are derived (2 cycles + margin), not scheduler-load
guesses, and the contract — probes never retire a request that has no
explicit deadline (apache#2392) — is exercised identically.
The remaining short sleeps in this workspace are all poll intervals inside
bounded wait loops, which are already observable completion; they are
untouched.
Focused timing: the two affected tests drop from ~2.5s each to ~0.37s each
(node --test, local). Full workspace suite: 734/734 pass after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The injected cadence matches the existing connectTimeoutMs/handshakeTimeoutMs pattern, default behavior is unchanged (2s), and both tests are genuinely faster (430/450ms). The assertions still bite: if #2392 regresses, the pending request gets retired and the deepEqual fails.

One gap: the tests can't tell "probes fired and the request survived" from "probes never fired". If a future change stops using livenessIntervalMs in the constructor (types still compile), every assertion passes with zero probes in the window. A probe counter on the server side, asserted before release, would make the premise falsifiable. Cheap to add.

Nit: livenessIntervalMs is validated in the constructor, after connect + handshake, so a bad value surfaces as "handshake_failed" instead of a config error; the other timeouts validate up front. Approving.

…dence took
Review follow-up on the injected liveness cadence: the tests could not tell
"probes fired and the request survived" from "probes never fired" — if a
future change stopped threading livenessIntervalMs, every assertion would
still pass with zero probes in the window.
ConnectRuntimeHostInput gains onLivenessProbe, invoked after a probe
round-trips and validates its Host Epoch. Both tests now gate on two observed
probe round-trips while their long-lived request is pending — the host-kernel
test releases its admitted request only after the crossing (bounded, loud
timeout if the cadence stops taking effect), and the Agent-graph fake's stop()
awaits the crossing instead of a derived sleep, making the ordering causal
with no fixed timing at all.
Also validates livenessIntervalMs up front in connectResolvedRuntimeHost
alongside the other connect timeouts, so a bad value is a config error
instead of surfacing as handshake_failed.
Both tests get faster again (~0.29s / ~0.33s); 734/734 pass after a clean
build.
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both points addressed in bad4d15:

  • Falsifiable premise: ConnectRuntimeHostInput.onLivenessProbe fires after a probe round-trips and validates its Host Epoch. The host-kernel test now releases the admitted request only after two observed probe round-trips (bounded — a cadence that stops taking effect times out loudly instead of vacuously passing), and the Agent-graph fake's stop() awaits that same crossing instead of a derived sleep, so the ordering is causal with no fixed timing left at all.
  • Nit: livenessIntervalMs is now validated up front in connectResolvedRuntimeHost alongside connectTimeoutMs/handshakeTimeoutMs, so a bad value surfaces as a config error, not handshake_failed.

Both tests got faster again (~0.29s / ~0.33s); 734/734 after a clean build.

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up. Both review points are substantively addressed:

  • the tests now wait for observed, epoch-validated probe round-trips instead of assuming the injected cadence fired;
  • livenessIntervalMs is validated before transport and handshake work, consistently with the existing timeout options.

The resulting tests are faster and their probe-crossing premise is now falsifiable.

P3 — Test observability is exposed through the public client input.

onLivenessProbe is documented as a test hook but is now part of the exported ConnectRuntimeHostInput. Its callback also runs inside the probe success chain, so an observer exception can fail the connection.

This is non-blocking because existing callers do not pass it and default production behavior is unchanged. Longer term, I would prefer observing probes through a package-internal test seam or Host/transport fixture. If the callback remains a public diagnostics hook, its exceptions should be isolated from connection health.

Small test-scoping suggestion: in the Agent Graph test, pass the probe observer only to the final TUI connection that issues agent.graph.stop. That makes it impossible for an earlier slow query on another connection to satisfy the shared counter.

The requested follow-up is effective, and I found no blocking issue. Approved.

简体中文

感谢继续修改。上一轮的两个问题都已经得到实质解决:

  • 测试现在等待实际观察到、并完成 Host Epoch 校验的 probe round-trip,不再假设注入的 cadence 已经触发;
  • livenessIntervalMs 现在与现有 timeout 配置一样,在 transport 和 handshake 之前完成校验。

测试变得更快,而且 probe-crossing 前提现在可以被证伪。

P3 — 测试观测能力进入了公开 client input。

onLivenessProbe 的文档明确说明它是 test hook,但它现在属于公开导出的 ConnectRuntimeHostInput。callback 还运行在 probe success chain 内,因此 observer 抛出的异常可能导致连接失败。

这是非阻塞问题,因为现有调用方不会传入它,默认生产行为也没有变化。长期来看,更适合通过 package-internal 测试 seam 或 Host/transport fixture 观察 probe。如果保留为公开 diagnostics hook,则 observer 异常不应影响连接健康。

还有一个小的测试作用域建议:Agent Graph 测试只给最终执行 agent.graph.stop 的 TUI connection 传入 probe observer。这样前面其他 connection 上的慢查询就不可能提前满足共享 counter。

本轮修改有效,没有发现阻塞问题。Approve。

Review follow-up (non-blocking P3): onLivenessProbe is a diagnostics hook,
so an exception thrown by the observer is now swallowed instead of running
inside the probe success chain where it would fail the connection it is
watching; the input doc says so explicitly.
Also scopes the Agent-graph test's observer to the one connection that
issues agent.graph.stop, so no other connection's slow query could ever
satisfy the shared probe counter.
@Astro-Han
Astro-Han merged commit 1823c9d into apache:mainAug 8, 2026
2 checks passed
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
ContributorAuthor

Both follow-ups applied in fc78317:

  • Observer isolation: onLivenessProbe exceptions are now swallowed at the call site — a diagnostics hook can no longer fail the connection it is watching — and the input doc states that contract explicitly. On the longer-term point: agreed a package-internal seam would be cleaner; if a second consumer of probe observability ever appears, that's the moment to move it behind a Host/transport fixture rather than the public input.
  • Test scoping: the Agent-graph observer is now wired only onto the final TUI connection that issues agent.graph.stop, so no other connection's slow query can satisfy the shared counter.

Suite green after a clean build; typecheck and biome pass.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@UncertaintyDeterminesYou4ndMe@Astro-Han