Skip to content

fix(runtime-host): give owned-candidate connect a CI-safe election window - #3221

Merged
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake
Aug 20, 2026
Merged

fix(runtime-host): give owned-candidate connect a CI-safe election window#3221
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake

Conversation

@1625567290

Copy link
Copy Markdown
Contributor

Summary

owned Host exits promptly after its first connection closes flakes under the full Runtime Host suite. CI run 32101082229 finished 962/963 tests and failed this one at 2.19s with failed instead of connected. The 2s election window is enough on an idle machine and too short under suite load. The promptness claim is Host exit after the first connection closes, not the connect itself.

  • Widen that test's election window to 8s and keep settle(500) as the prompt-exit check.
  • Put the concrete failed:<reason> on the assertion so a later miss is diagnosable.
  • A late candidate report after the election ends no longer blocks return; the missed election stays startup_timeout.

Fixes#3190

Verification

  • node --test packages/runtime-host/dist/__tests__/owned-candidate.test.js — 7/7

Checklist

  • Tests cover the change and fail without it
  • Focused lint/typecheck and the affected suites pass locally
  • Full workspace lint/format/typecheck

Does this PR entail a change in behavior?

  • Yes — owned connect under a missed election returns without waiting on a late candidate report, and the flaking test uses an 8s election window

@coderabbitai

coderabbitaiBot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b6dc0fd2-ecac-44d4-a27f-7360a9c69d26

📥 Commits

Reviewing files that changed from the base of the PR and between 95e0072 and 8ca108e.

📒 Files selected for processing (1)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/tests/owned-candidate.test.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.


📝 Walkthrough

What problem this solves

This PR reduces CI timing flakes in the Runtime Host owned-candidate connection test.

  • It increases the election window from 2 seconds to 8 seconds.
  • It keeps settle(500) as the prompt-exit check after the first connection closes.
  • It preserves and asserts the concrete failed:<reason> result.
  • It returns startup_timeout when a candidate reports after a missed election.
  • It classifies missing or failed launch.spawned results as host_unresponsive.
  • It releases late spawned hosts instead of terminating them with SIGKILL.

Source of truth

The PR extends the existing owned-candidate connection, election, and launch-cleanup paths. It does not create a parallel connection path or add a public API.

Scope and simplification

The changes are the smallest coherent solution for the reported timing failure and late-startup behavior.

Asynchronous releaseOwnedLaunch cleanup is necessary because a late candidate must not block the startup_timeout result. It also avoids terminating a host that another client may adopt.

The diagnostic formatter, polling helper, and regression tests preserve failure details and cover the timing cases. No deletion or simplification is apparent without weakening diagnostics or regression coverage.

Validation and risks

Focused verification passed: 7/7 tests in owned-candidate.test.js.

The tests cover the longer election window, concrete failure reporting, unresolved candidates, and late candidate adoption. Full-suite validation is not reported, so required checks remain unverified.

Complexity delta

  • Authorities: No new source of truth.
  • States: Clarifies existing host_unresponsive, existing_host, and startup_timeout results.
  • Branches: Adds handling for rejected or missing launch.spawned results and asynchronous release cleanup.
  • Configuration: Increases the test election deadline from 2 seconds to 8 seconds.
  • Public surface: No exported or public entities change.
  • Test maintenance: Adds regression coverage, diagnostics, and a polling helper.

Total maintenance complexity stays justified. The added branches and tests directly cover the reported failure modes.

Review-relevant risks

The connection behavior changes when launch.spawned fails, returns no host, or reports after a missed election. The cleanup behavior also changes from termination to release. These changes can affect Runtime Host error classification, host ownership, and user-visible connection outcomes. Material changes in these areas require independent human review under repository policy.

No security, licensing, release, or governance effect was identified in the current diff.

Required checks remain unverified beyond the reported focused test result. The person performing the merge reviews the final diff. A maintainer makes the final determination.

Walkthrough

The runtime-host client now classifies owned host startup failures and distinguishes launch failures from existing-host outcomes. Tests cover startup timeouts, late candidates, election timing, cleanup, and connection diagnostics.

Changes

Owned runtime-host connection handling

Layer / File(s)Summary
Classify and clean up startup failures
packages/runtime-host/src/client/connect-or-spawn.ts
Owned host startup failures and missing hosts produce host_unresponsive. Connected results produce existing_host only when no launch occurs. Failure cleanup releases owned launches and handles rejected spawn promises.
Cover timeout and late-candidate behavior
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Tests cover unresolved candidates, late candidate adoption, candidate cleanup, pending promises, and the extended election deadline.
Add connection diagnostics and polling
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Test helpers format connection results and poll for delayed candidate startup.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk:⚪ Minimal · up to 8ca10

This localized change widens the owned-candidate election window and makes missed-election failures diagnosable; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check nameStatusExplanationResolution
Out of Scope Changes check⚠️ WarningThe connect-or-spawn.ts changes alter production candidate-handling behavior beyond the linked issue's test-flake investigation scope.Separate the production behavior change into a linked issue or explain its direct necessity for issue #3190.
Ai Use Disclosure⚠️ WarningThe PR description selects neither disclosure statement, and all five PR commits lack a valid Generated-by trailer.Add exactly one required declaration. If generative tooling made a substantive contribution, name the tool and scope; if it authored material content, add a trailer that survives squash/amend. See “Human ownership and AI attribution” in...
✅ Passed checks (3 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly describes the main change: extending the owned-candidate election window for CI reliability.
Description check✅ PassedThe description covers the problem, fix, issue reference, verification, tests, and behavior change, but omits the required AI use section.
Linked Issues check✅ PassedThe PR preserves the failure reason and increases the election deadline, addressing the timing-flake objectives in issue #3190.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Stabilize owned-candidate election handling under CI load

🐞 Bug fix🧪 Tests🕐 10-20 Minutes

Grey Divider

AI Description

• Preserve startup timeouts when candidate spawn failures arrive after the election deadline.
• Extend the disconnect test election window to eight seconds for loaded CI.
• Add regression coverage and concrete diagnostics for owned connection failures.
Diagram

sequenceDiagram
actor Caller
participant Owned as Owned Connect
participant Election as Host Election
participant Candidate as Owned Candidate
Caller->>Owned: request connection
Owned->>Election: connect or spawn
Election->>Candidate: launch candidate
Note over Election: deadline expires
Election-->>Owned: startup_timeout
Candidate--x Owned: late spawn failure
Owned-->>Caller: preserve timeout
Loading
High-Level Assessment

The focused wrapper change is appropriate: it treats a rejected spawn report as an unavailable owned host while preserving an already-determined election failure. Returning immediately without observing the candidate promise could reduce latency further, but risks losing ownership and cleanup of a process that eventually spawns; introducing launch cancellation would be disproportionate for this fix.

Files changed (2) +46 / -4

Bug fix (1) +6 / -2
connect-or-spawn.tsPreserve owned election failures after spawn rejection+6/-2

Preserve owned election failures after spawn rejection

• Handles rejected owned-candidate spawn promises without falling through to the generic exception path. Existing election failures are returned unchanged, while connected results without a usable launched host are classified as 'host_unresponsive' or 'existing_host'.

packages/runtime-host/src/client/connect-or-spawn.ts

Tests (1) +40 / -2
owned-candidate.test.tsCover late candidate reports and stabilize the disconnect lifecycle test+40/-2

Cover late candidate reports and stabilize the disconnect lifecycle test

• Adds a regression test proving that a late rejected candidate report does not replace 'startup_timeout'. Expands the disconnect test election window from two to eight seconds and reports concrete failure reasons while retaining the 500ms host-exit assertion.

packages/runtime-host/src/tests/owned-candidate.test.ts

@qodo-code-review

qodo-code-reviewBot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0)📘 Rule violations (0)📜 Skill insights (0)

Grey Divider


Action required

1. Late report still blocks return✓ Resolved🐞 Bug≡ Correctness
Description
After the election produces startup_timeout, connectOwnedRuntimeHostWithDependencies() still
awaits launch.spawned, so a late or never-settling candidate report delays or permanently blocks
the result beyond the election deadline. This violates the stated behavior that a missed election
returns without waiting for the late report.
Code

packages/runtime-host/src/client/connect-or-spawn.ts[130]

+ const host = await launch?.spawned.catch(() => undefined);
Relevance

●●● Strong

Accepted precedent supports fixing blocking async cleanup; this is a deterministic missed-deadline
correctness bug matching PR intent.

PR-#2674

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The election waits for launch.spawned only through the deadline and then returns
startup_timeout, but the outer owned wrapper immediately awaits the original unbounded promise
again. The added test delays rejection for 20 ms after a 1 ms deadline yet checks only the returned
reason, so it passes even though the function waits for that late report; replacing the test promise
with a never-settling promise reproducibly prevents completion.

packages/runtime-host/src/client/connect-or-spawn.ts[249-260]
packages/runtime-host/src/client/connect-or-spawn.ts[293-297]
packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
packages/runtime-host/src/tests/owned-candidate.test.ts[50-74]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`connectOwnedRuntimeHostWithDependencies()` unconditionally awaits the owned candidate's `spawned` promise after the bounded election completes. Consequently, a non-connected election result cannot return until that late promise settles, and a never-settling report hangs indefinitely.
## Issue Context
The election already bounds candidate waiting with `settleBeforeDeadline()` and returns `startup_timeout`. Preserve that result immediately when it is non-connected; attach non-blocking cleanup handlers for a candidate that reports later rather than awaiting it. Extend the regression test to prove completion occurs near the election deadline with a delayed or never-settling report.
## Fix Focus Areas
- packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
- packages/runtime-host/src/__tests__/owned-candidate.test.ts[50-74]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⚖️ Balanced: This changes runtime connection/election behavior and error handling in an asynchronous host lifecycle path, so it carries genuine correctness risk, but the logic is localized enough for a careful single-pass review.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 92e25a18-7c46-472b-b5f2-0caa3a90d1eb

📥 Commits

Reviewing files that changed from the base of the PR and between 21929e3 and f1f11fb.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 2 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated
Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning at the election deadline instead of awaiting a never-settling spawn is the right goal, but the new fire-and-forget cleanup transfers an unsafe kill decision to the timed-out client. A late candidate can become the shared State Root host and accept another client before this cleanup fires; settle(1_000) then kills that live host. The owner of the process must decide whether it is idle, rather than a client that already lost the election. The regression test also needs to cover a candidate that resolves late, not only one that rejects.

AI-assisted review: Codex coordinated two independent reviewer passes and an OpenCode Go DeepSeek V4 Flash high-effort adversarial pass. I verified the exact head f944c30984deeeff827db3871f3cf122c4c249de, the launcher kill behavior, the State Root connection path, and current CI. No local tests were run in this review.

中文审查

不再等待永不结束的 spawn、按 election deadline 返回,这个目标正确;但新的后台清理把危险的 kill 决策交给了已经超时的客户端。迟到 candidate 可能已经成为同一 State Root 的共享 Host,并接入另一个客户端,随后仍会被 settle(1_000) 杀死。进程是否空闲应由 Host 自身判断,而不是由已经输掉选举的客户端决定。回归测试也需要覆盖 candidate 迟到但成功 resolve 的路径,而不只是 reject。

本次为 AI 辅助审查:Codex 协调两轮独立 reviewer 审查及 OpenCode Go DeepSeek V4 Flash high 对抗性复核,并核验了精确 head、launcher kill 行为、State Root 连接路径和当前 CI。本轮未运行本地测试。

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated
Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/runtime-host/src/__tests__/owned-candidate.test.ts (1)

301-317: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consolidate the duplicate connection-result formatters.

ownedConnectFailure and adoptedConnectFailure have identical branches. Use one helper with a union of the two result types. This removes duplicate maintenance when result kinds change.

Disposition: optional.

As per path instructions, “Flag concrete cases where code can be deleted or simplified.”

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e4a9d78f-8cc3-422b-a298-18cdecac0b9d

📥 Commits

Reviewing files that changed from the base of the PR and between f944c30 and 95e0072.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts
…ndow
The promptness claim is Host exit after the first connection closes,
not the connect itself. A two-second election misses spawn plus
handshake under a full suite and only reports `failed`. Widen the
window, keep the concrete reason on the assertion, and map a
deadline throw to startup_timeout.
Fixesapache#3190
Election already bounds candidate startup. Awaiting launch.spawned
after a non-connected result hangs connectOwned when the candidate
never reports. Return that result immediately and settle a late
host in the background.
Fixesapache#3190
A timed-out owned connect no longer settles a late candidate. That
path sent SIGKILL after 1s and could terminate a host another client
had already adopted. Release the process and let idle-grace /
initial-connection policy exit it when unused.
Fixesapache#3190
…tling
The adopt regression only checked that the late pid stayed alive. Wrap
the delivered candidate so a settle() cleanup cannot pass that check
and kill the adopted host after the assertion.
Fixesapache#3190
@1625567290
1625567290force-pushed the fix/host-owned-candidate-connection-flake branch from 95e0072 to 8ca108eCompareAugust 18, 2026 17:05

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head resolves the earlier ownership race correctly. A missed election now returns at its own deadline, preserves the specific failure reason, and releases a late owned candidate to the Runtime Host lifecycle instead of killing a process another client may have adopted. The implementation extends the existing OwnedCandidateAttempt seam and does not create another election authority.

I found no reproducible P0–P3 issue on this head. I also checked two adversarial candidates and rejected them: root ownership prevents the pid-mismatch candidate from simultaneously being the adopted authority, and the test's deliberately unresolved spawn promise occupies the first election until its deadline, so the real candidate cannot make that call enter the connected branch.

The current windows_recovery failure is in the unchanged Local IPC trust fixture (insecure_endpoint_directory), while the affected Runtime Host checks pass. This approval is for the code; the PR is not merge-ready until required CI is green.

AI-assisted review disclosure: Codex performed the final review using two independent reviewer passes and OpenCode Go DeepSeek V4 Flash (high effort) as an adversarial advisory pass. I verified the election state machine, root ownership, tests, exact head 8ca108e3e, and the failing CI log. No local tests were run.

中文评论

当前 head 已正确解决此前的 ownership race:missed election 会在自己的 deadline 返回并保留具体失败原因;迟到的 owned candidate 会交还 Runtime Host lifecycle,而不是杀掉可能已被其他 client 接管的进程。实现直接扩展既有 OwnedCandidateAttempt seam,没有新增 election authority。

当前 head 未发现可复现的 P0–P3。我也复核并排除了两个对抗性候选:root ownership 保证 pid mismatch 的 candidate 不可能同时成为被接管的权威;测试中故意不 resolve 的 spawn promise 会占用第一次 election 直到 deadline,因此真实 candidate 不会让该调用进入 connected 分支。

当前 windows_recovery 失败来自未改动的 Local IPC trust fixture(insecure_endpoint_directory),受影响的 Runtime Host checks 已通过。本次 approve 针对代码;必需 CI 全绿前不能合并。

AI 辅助审查说明:Codex 使用两轮独立 reviewer 审查,并以 OpenCode Go DeepSeek V4 Flash(high effort)进行对抗性辅助审查;我已核对 election 状态机、root ownership、测试、精确 head 8ca108e3e 和失败 CI 日志。本轮未运行本地测试。

@Astro-Han

Copy link
Copy Markdown
Contributor

@1625567290 Hi, Could we state that if there is any ai tools used in the PR?

@Astro-Han
Astro-Han merged commit 803edc1 into apache:mainAug 20, 2026
18 of 19 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(runtime-host): investigate owned candidate connection flake

2 participants

@1625567290@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
fix(runtime-host): give owned-candidate connect a CI-safe election window by 1625567290 · Pull Request #3221 · apache/maka · GitHub
Skip to content

fix(runtime-host): give owned-candidate connect a CI-safe election window - #3221

Merged
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake
Aug 20, 2026
Merged

fix(runtime-host): give owned-candidate connect a CI-safe election window#3221
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake

Conversation

@1625567290

Copy link
Copy Markdown
Contributor

Summary

owned Host exits promptly after its first connection closes flakes under the full Runtime Host suite. CI run 32101082229 finished 962/963 tests and failed this one at 2.19s with failed instead of connected. The 2s election window is enough on an idle machine and too short under suite load. The promptness claim is Host exit after the first connection closes, not the connect itself.

  • Widen that test's election window to 8s and keep settle(500) as the prompt-exit check.
  • Put the concrete failed:<reason> on the assertion so a later miss is diagnosable.
  • A late candidate report after the election ends no longer blocks return; the missed election stays startup_timeout.

Fixes#3190

Verification

  • node --test packages/runtime-host/dist/__tests__/owned-candidate.test.js — 7/7

Checklist

  • Tests cover the change and fail without it
  • Focused lint/typecheck and the affected suites pass locally
  • Full workspace lint/format/typecheck

Does this PR entail a change in behavior?

  • Yes — owned connect under a missed election returns without waiting on a late candidate report, and the flaking test uses an 8s election window

@coderabbitai

coderabbitaiBot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b6dc0fd2-ecac-44d4-a27f-7360a9c69d26

📥 Commits

Reviewing files that changed from the base of the PR and between 95e0072 and 8ca108e.

📒 Files selected for processing (1)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/tests/owned-candidate.test.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.


📝 Walkthrough

What problem this solves

This PR reduces CI timing flakes in the Runtime Host owned-candidate connection test.

  • It increases the election window from 2 seconds to 8 seconds.
  • It keeps settle(500) as the prompt-exit check after the first connection closes.
  • It preserves and asserts the concrete failed:<reason> result.
  • It returns startup_timeout when a candidate reports after a missed election.
  • It classifies missing or failed launch.spawned results as host_unresponsive.
  • It releases late spawned hosts instead of terminating them with SIGKILL.

Source of truth

The PR extends the existing owned-candidate connection, election, and launch-cleanup paths. It does not create a parallel connection path or add a public API.

Scope and simplification

The changes are the smallest coherent solution for the reported timing failure and late-startup behavior.

Asynchronous releaseOwnedLaunch cleanup is necessary because a late candidate must not block the startup_timeout result. It also avoids terminating a host that another client may adopt.

The diagnostic formatter, polling helper, and regression tests preserve failure details and cover the timing cases. No deletion or simplification is apparent without weakening diagnostics or regression coverage.

Validation and risks

Focused verification passed: 7/7 tests in owned-candidate.test.js.

The tests cover the longer election window, concrete failure reporting, unresolved candidates, and late candidate adoption. Full-suite validation is not reported, so required checks remain unverified.

Complexity delta

  • Authorities: No new source of truth.
  • States: Clarifies existing host_unresponsive, existing_host, and startup_timeout results.
  • Branches: Adds handling for rejected or missing launch.spawned results and asynchronous release cleanup.
  • Configuration: Increases the test election deadline from 2 seconds to 8 seconds.
  • Public surface: No exported or public entities change.
  • Test maintenance: Adds regression coverage, diagnostics, and a polling helper.

Total maintenance complexity stays justified. The added branches and tests directly cover the reported failure modes.

Review-relevant risks

The connection behavior changes when launch.spawned fails, returns no host, or reports after a missed election. The cleanup behavior also changes from termination to release. These changes can affect Runtime Host error classification, host ownership, and user-visible connection outcomes. Material changes in these areas require independent human review under repository policy.

No security, licensing, release, or governance effect was identified in the current diff.

Required checks remain unverified beyond the reported focused test result. The person performing the merge reviews the final diff. A maintainer makes the final determination.

Walkthrough

The runtime-host client now classifies owned host startup failures and distinguishes launch failures from existing-host outcomes. Tests cover startup timeouts, late candidates, election timing, cleanup, and connection diagnostics.

Changes

Owned runtime-host connection handling

Layer / File(s)Summary
Classify and clean up startup failures
packages/runtime-host/src/client/connect-or-spawn.ts
Owned host startup failures and missing hosts produce host_unresponsive. Connected results produce existing_host only when no launch occurs. Failure cleanup releases owned launches and handles rejected spawn promises.
Cover timeout and late-candidate behavior
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Tests cover unresolved candidates, late candidate adoption, candidate cleanup, pending promises, and the extended election deadline.
Add connection diagnostics and polling
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Test helpers format connection results and poll for delayed candidate startup.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk:⚪ Minimal · up to 8ca10

This localized change widens the owned-candidate election window and makes missed-election failures diagnosable; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check nameStatusExplanationResolution
Out of Scope Changes check⚠️ WarningThe connect-or-spawn.ts changes alter production candidate-handling behavior beyond the linked issue's test-flake investigation scope.Separate the production behavior change into a linked issue or explain its direct necessity for issue #3190.
Ai Use Disclosure⚠️ WarningThe PR description selects neither disclosure statement, and all five PR commits lack a valid Generated-by trailer.Add exactly one required declaration. If generative tooling made a substantive contribution, name the tool and scope; if it authored material content, add a trailer that survives squash/amend. See “Human ownership and AI attribution” in...
✅ Passed checks (3 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly describes the main change: extending the owned-candidate election window for CI reliability.
Description check✅ PassedThe description covers the problem, fix, issue reference, verification, tests, and behavior change, but omits the required AI use section.
Linked Issues check✅ PassedThe PR preserves the failure reason and increases the election deadline, addressing the timing-flake objectives in issue #3190.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Stabilize owned-candidate election handling under CI load

🐞 Bug fix🧪 Tests🕐 10-20 Minutes

Grey Divider

AI Description

• Preserve startup timeouts when candidate spawn failures arrive after the election deadline.
• Extend the disconnect test election window to eight seconds for loaded CI.
• Add regression coverage and concrete diagnostics for owned connection failures.
Diagram

sequenceDiagram
actor Caller
participant Owned as Owned Connect
participant Election as Host Election
participant Candidate as Owned Candidate
Caller->>Owned: request connection
Owned->>Election: connect or spawn
Election->>Candidate: launch candidate
Note over Election: deadline expires
Election-->>Owned: startup_timeout
Candidate--x Owned: late spawn failure
Owned-->>Caller: preserve timeout
Loading
High-Level Assessment

The focused wrapper change is appropriate: it treats a rejected spawn report as an unavailable owned host while preserving an already-determined election failure. Returning immediately without observing the candidate promise could reduce latency further, but risks losing ownership and cleanup of a process that eventually spawns; introducing launch cancellation would be disproportionate for this fix.

Files changed (2) +46 / -4

Bug fix (1) +6 / -2
connect-or-spawn.tsPreserve owned election failures after spawn rejection+6/-2

Preserve owned election failures after spawn rejection

• Handles rejected owned-candidate spawn promises without falling through to the generic exception path. Existing election failures are returned unchanged, while connected results without a usable launched host are classified as 'host_unresponsive' or 'existing_host'.

packages/runtime-host/src/client/connect-or-spawn.ts

Tests (1) +40 / -2
owned-candidate.test.tsCover late candidate reports and stabilize the disconnect lifecycle test+40/-2

Cover late candidate reports and stabilize the disconnect lifecycle test

• Adds a regression test proving that a late rejected candidate report does not replace 'startup_timeout'. Expands the disconnect test election window from two to eight seconds and reports concrete failure reasons while retaining the 500ms host-exit assertion.

packages/runtime-host/src/tests/owned-candidate.test.ts

@qodo-code-review

qodo-code-reviewBot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0)📘 Rule violations (0)📜 Skill insights (0)

Grey Divider


Action required

1. Late report still blocks return✓ Resolved🐞 Bug≡ Correctness
Description
After the election produces startup_timeout, connectOwnedRuntimeHostWithDependencies() still
awaits launch.spawned, so a late or never-settling candidate report delays or permanently blocks
the result beyond the election deadline. This violates the stated behavior that a missed election
returns without waiting for the late report.
Code

packages/runtime-host/src/client/connect-or-spawn.ts[130]

+ const host = await launch?.spawned.catch(() => undefined);
Relevance

●●● Strong

Accepted precedent supports fixing blocking async cleanup; this is a deterministic missed-deadline
correctness bug matching PR intent.

PR-#2674

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The election waits for launch.spawned only through the deadline and then returns
startup_timeout, but the outer owned wrapper immediately awaits the original unbounded promise
again. The added test delays rejection for 20 ms after a 1 ms deadline yet checks only the returned
reason, so it passes even though the function waits for that late report; replacing the test promise
with a never-settling promise reproducibly prevents completion.

packages/runtime-host/src/client/connect-or-spawn.ts[249-260]
packages/runtime-host/src/client/connect-or-spawn.ts[293-297]
packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
packages/runtime-host/src/tests/owned-candidate.test.ts[50-74]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`connectOwnedRuntimeHostWithDependencies()` unconditionally awaits the owned candidate's `spawned` promise after the bounded election completes. Consequently, a non-connected election result cannot return until that late promise settles, and a never-settling report hangs indefinitely.
## Issue Context
The election already bounds candidate waiting with `settleBeforeDeadline()` and returns `startup_timeout`. Preserve that result immediately when it is non-connected; attach non-blocking cleanup handlers for a candidate that reports later rather than awaiting it. Extend the regression test to prove completion occurs near the election deadline with a delayed or never-settling report.
## Fix Focus Areas
- packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
- packages/runtime-host/src/__tests__/owned-candidate.test.ts[50-74]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⚖️ Balanced: This changes runtime connection/election behavior and error handling in an asynchronous host lifecycle path, so it carries genuine correctness risk, but the logic is localized enough for a careful single-pass review.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 92e25a18-7c46-472b-b5f2-0caa3a90d1eb

📥 Commits

Reviewing files that changed from the base of the PR and between 21929e3 and f1f11fb.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 2 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated
Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning at the election deadline instead of awaiting a never-settling spawn is the right goal, but the new fire-and-forget cleanup transfers an unsafe kill decision to the timed-out client. A late candidate can become the shared State Root host and accept another client before this cleanup fires; settle(1_000) then kills that live host. The owner of the process must decide whether it is idle, rather than a client that already lost the election. The regression test also needs to cover a candidate that resolves late, not only one that rejects.

AI-assisted review: Codex coordinated two independent reviewer passes and an OpenCode Go DeepSeek V4 Flash high-effort adversarial pass. I verified the exact head f944c30984deeeff827db3871f3cf122c4c249de, the launcher kill behavior, the State Root connection path, and current CI. No local tests were run in this review.

中文审查

不再等待永不结束的 spawn、按 election deadline 返回,这个目标正确;但新的后台清理把危险的 kill 决策交给了已经超时的客户端。迟到 candidate 可能已经成为同一 State Root 的共享 Host,并接入另一个客户端,随后仍会被 settle(1_000) 杀死。进程是否空闲应由 Host 自身判断,而不是由已经输掉选举的客户端决定。回归测试也需要覆盖 candidate 迟到但成功 resolve 的路径,而不只是 reject。

本次为 AI 辅助审查:Codex 协调两轮独立 reviewer 审查及 OpenCode Go DeepSeek V4 Flash high 对抗性复核,并核验了精确 head、launcher kill 行为、State Root 连接路径和当前 CI。本轮未运行本地测试。

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated
Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/runtime-host/src/__tests__/owned-candidate.test.ts (1)

301-317: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consolidate the duplicate connection-result formatters.

ownedConnectFailure and adoptedConnectFailure have identical branches. Use one helper with a union of the two result types. This removes duplicate maintenance when result kinds change.

Disposition: optional.

As per path instructions, “Flag concrete cases where code can be deleted or simplified.”

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e4a9d78f-8cc3-422b-a298-18cdecac0b9d

📥 Commits

Reviewing files that changed from the base of the PR and between f944c30 and 95e0072.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts
…ndow
The promptness claim is Host exit after the first connection closes,
not the connect itself. A two-second election misses spawn plus
handshake under a full suite and only reports `failed`. Widen the
window, keep the concrete reason on the assertion, and map a
deadline throw to startup_timeout.
Fixesapache#3190
Election already bounds candidate startup. Awaiting launch.spawned
after a non-connected result hangs connectOwned when the candidate
never reports. Return that result immediately and settle a late
host in the background.
Fixesapache#3190
A timed-out owned connect no longer settles a late candidate. That
path sent SIGKILL after 1s and could terminate a host another client
had already adopted. Release the process and let idle-grace /
initial-connection policy exit it when unused.
Fixesapache#3190
…tling
The adopt regression only checked that the late pid stayed alive. Wrap
the delivered candidate so a settle() cleanup cannot pass that check
and kill the adopted host after the assertion.
Fixesapache#3190
@1625567290
1625567290force-pushed the fix/host-owned-candidate-connection-flake branch from 95e0072 to 8ca108eCompareAugust 18, 2026 17:05

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head resolves the earlier ownership race correctly. A missed election now returns at its own deadline, preserves the specific failure reason, and releases a late owned candidate to the Runtime Host lifecycle instead of killing a process another client may have adopted. The implementation extends the existing OwnedCandidateAttempt seam and does not create another election authority.

I found no reproducible P0–P3 issue on this head. I also checked two adversarial candidates and rejected them: root ownership prevents the pid-mismatch candidate from simultaneously being the adopted authority, and the test's deliberately unresolved spawn promise occupies the first election until its deadline, so the real candidate cannot make that call enter the connected branch.

The current windows_recovery failure is in the unchanged Local IPC trust fixture (insecure_endpoint_directory), while the affected Runtime Host checks pass. This approval is for the code; the PR is not merge-ready until required CI is green.

AI-assisted review disclosure: Codex performed the final review using two independent reviewer passes and OpenCode Go DeepSeek V4 Flash (high effort) as an adversarial advisory pass. I verified the election state machine, root ownership, tests, exact head 8ca108e3e, and the failing CI log. No local tests were run.

中文评论

当前 head 已正确解决此前的 ownership race:missed election 会在自己的 deadline 返回并保留具体失败原因;迟到的 owned candidate 会交还 Runtime Host lifecycle,而不是杀掉可能已被其他 client 接管的进程。实现直接扩展既有 OwnedCandidateAttempt seam,没有新增 election authority。

当前 head 未发现可复现的 P0–P3。我也复核并排除了两个对抗性候选:root ownership 保证 pid mismatch 的 candidate 不可能同时成为被接管的权威;测试中故意不 resolve 的 spawn promise 会占用第一次 election 直到 deadline,因此真实 candidate 不会让该调用进入 connected 分支。

当前 windows_recovery 失败来自未改动的 Local IPC trust fixture(insecure_endpoint_directory),受影响的 Runtime Host checks 已通过。本次 approve 针对代码;必需 CI 全绿前不能合并。

AI 辅助审查说明:Codex 使用两轮独立 reviewer 审查,并以 OpenCode Go DeepSeek V4 Flash(high effort)进行对抗性辅助审查;我已核对 election 状态机、root ownership、测试、精确 head 8ca108e3e 和失败 CI 日志。本轮未运行本地测试。

@Astro-Han

Copy link
Copy Markdown
Contributor

@1625567290 Hi, Could we state that if there is any ai tools used in the PR?

@Astro-Han
Astro-Han merged commit 803edc1 into apache:mainAug 20, 2026
18 of 19 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(runtime-host): investigate owned candidate connection flake

2 participants

@1625567290@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fix(runtime-host): give owned-candidate connect a CI-safe election window by 1625567290 · Pull Request #3221 · apache/maka · GitHub
Skip to content

fix(runtime-host): give owned-candidate connect a CI-safe election window - #3221

Merged
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake
Aug 20, 2026
Merged

fix(runtime-host): give owned-candidate connect a CI-safe election window#3221
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake

Conversation

@1625567290

Copy link
Copy Markdown
Contributor

Summary

owned Host exits promptly after its first connection closes flakes under the full Runtime Host suite. CI run 32101082229 finished 962/963 tests and failed this one at 2.19s with failed instead of connected. The 2s election window is enough on an idle machine and too short under suite load. The promptness claim is Host exit after the first connection closes, not the connect itself.

  • Widen that test's election window to 8s and keep settle(500) as the prompt-exit check.
  • Put the concrete failed:<reason> on the assertion so a later miss is diagnosable.
  • A late candidate report after the election ends no longer blocks return; the missed election stays startup_timeout.

Fixes#3190

Verification

  • node --test packages/runtime-host/dist/__tests__/owned-candidate.test.js — 7/7

Checklist

  • Tests cover the change and fail without it
  • Focused lint/typecheck and the affected suites pass locally
  • Full workspace lint/format/typecheck

Does this PR entail a change in behavior?

  • Yes — owned connect under a missed election returns without waiting on a late candidate report, and the flaking test uses an 8s election window

@coderabbitai

coderabbitaiBot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b6dc0fd2-ecac-44d4-a27f-7360a9c69d26

📥 Commits

Reviewing files that changed from the base of the PR and between 95e0072 and 8ca108e.

📒 Files selected for processing (1)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/tests/owned-candidate.test.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.


📝 Walkthrough

What problem this solves

This PR reduces CI timing flakes in the Runtime Host owned-candidate connection test.

  • It increases the election window from 2 seconds to 8 seconds.
  • It keeps settle(500) as the prompt-exit check after the first connection closes.
  • It preserves and asserts the concrete failed:<reason> result.
  • It returns startup_timeout when a candidate reports after a missed election.
  • It classifies missing or failed launch.spawned results as host_unresponsive.
  • It releases late spawned hosts instead of terminating them with SIGKILL.

Source of truth

The PR extends the existing owned-candidate connection, election, and launch-cleanup paths. It does not create a parallel connection path or add a public API.

Scope and simplification

The changes are the smallest coherent solution for the reported timing failure and late-startup behavior.

Asynchronous releaseOwnedLaunch cleanup is necessary because a late candidate must not block the startup_timeout result. It also avoids terminating a host that another client may adopt.

The diagnostic formatter, polling helper, and regression tests preserve failure details and cover the timing cases. No deletion or simplification is apparent without weakening diagnostics or regression coverage.

Validation and risks

Focused verification passed: 7/7 tests in owned-candidate.test.js.

The tests cover the longer election window, concrete failure reporting, unresolved candidates, and late candidate adoption. Full-suite validation is not reported, so required checks remain unverified.

Complexity delta

  • Authorities: No new source of truth.
  • States: Clarifies existing host_unresponsive, existing_host, and startup_timeout results.
  • Branches: Adds handling for rejected or missing launch.spawned results and asynchronous release cleanup.
  • Configuration: Increases the test election deadline from 2 seconds to 8 seconds.
  • Public surface: No exported or public entities change.
  • Test maintenance: Adds regression coverage, diagnostics, and a polling helper.

Total maintenance complexity stays justified. The added branches and tests directly cover the reported failure modes.

Review-relevant risks

The connection behavior changes when launch.spawned fails, returns no host, or reports after a missed election. The cleanup behavior also changes from termination to release. These changes can affect Runtime Host error classification, host ownership, and user-visible connection outcomes. Material changes in these areas require independent human review under repository policy.

No security, licensing, release, or governance effect was identified in the current diff.

Required checks remain unverified beyond the reported focused test result. The person performing the merge reviews the final diff. A maintainer makes the final determination.

Walkthrough

The runtime-host client now classifies owned host startup failures and distinguishes launch failures from existing-host outcomes. Tests cover startup timeouts, late candidates, election timing, cleanup, and connection diagnostics.

Changes

Owned runtime-host connection handling

Layer / File(s)Summary
Classify and clean up startup failures
packages/runtime-host/src/client/connect-or-spawn.ts
Owned host startup failures and missing hosts produce host_unresponsive. Connected results produce existing_host only when no launch occurs. Failure cleanup releases owned launches and handles rejected spawn promises.
Cover timeout and late-candidate behavior
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Tests cover unresolved candidates, late candidate adoption, candidate cleanup, pending promises, and the extended election deadline.
Add connection diagnostics and polling
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Test helpers format connection results and poll for delayed candidate startup.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk:⚪ Minimal · up to 8ca10

This localized change widens the owned-candidate election window and makes missed-election failures diagnosable; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check nameStatusExplanationResolution
Out of Scope Changes check⚠️ WarningThe connect-or-spawn.ts changes alter production candidate-handling behavior beyond the linked issue's test-flake investigation scope.Separate the production behavior change into a linked issue or explain its direct necessity for issue #3190.
Ai Use Disclosure⚠️ WarningThe PR description selects neither disclosure statement, and all five PR commits lack a valid Generated-by trailer.Add exactly one required declaration. If generative tooling made a substantive contribution, name the tool and scope; if it authored material content, add a trailer that survives squash/amend. See “Human ownership and AI attribution” in...
✅ Passed checks (3 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly describes the main change: extending the owned-candidate election window for CI reliability.
Description check✅ PassedThe description covers the problem, fix, issue reference, verification, tests, and behavior change, but omits the required AI use section.
Linked Issues check✅ PassedThe PR preserves the failure reason and increases the election deadline, addressing the timing-flake objectives in issue #3190.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Stabilize owned-candidate election handling under CI load

🐞 Bug fix🧪 Tests🕐 10-20 Minutes

Grey Divider

AI Description

• Preserve startup timeouts when candidate spawn failures arrive after the election deadline.
• Extend the disconnect test election window to eight seconds for loaded CI.
• Add regression coverage and concrete diagnostics for owned connection failures.
Diagram

sequenceDiagram
actor Caller
participant Owned as Owned Connect
participant Election as Host Election
participant Candidate as Owned Candidate
Caller->>Owned: request connection
Owned->>Election: connect or spawn
Election->>Candidate: launch candidate
Note over Election: deadline expires
Election-->>Owned: startup_timeout
Candidate--x Owned: late spawn failure
Owned-->>Caller: preserve timeout
Loading
High-Level Assessment

The focused wrapper change is appropriate: it treats a rejected spawn report as an unavailable owned host while preserving an already-determined election failure. Returning immediately without observing the candidate promise could reduce latency further, but risks losing ownership and cleanup of a process that eventually spawns; introducing launch cancellation would be disproportionate for this fix.

Files changed (2) +46 / -4

Bug fix (1) +6 / -2
connect-or-spawn.tsPreserve owned election failures after spawn rejection+6/-2

Preserve owned election failures after spawn rejection

• Handles rejected owned-candidate spawn promises without falling through to the generic exception path. Existing election failures are returned unchanged, while connected results without a usable launched host are classified as 'host_unresponsive' or 'existing_host'.

packages/runtime-host/src/client/connect-or-spawn.ts

Tests (1) +40 / -2
owned-candidate.test.tsCover late candidate reports and stabilize the disconnect lifecycle test+40/-2

Cover late candidate reports and stabilize the disconnect lifecycle test

• Adds a regression test proving that a late rejected candidate report does not replace 'startup_timeout'. Expands the disconnect test election window from two to eight seconds and reports concrete failure reasons while retaining the 500ms host-exit assertion.

packages/runtime-host/src/tests/owned-candidate.test.ts

@qodo-code-review

qodo-code-reviewBot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0)📘 Rule violations (0)📜 Skill insights (0)

Grey Divider


Action required

1. Late report still blocks return✓ Resolved🐞 Bug≡ Correctness
Description
After the election produces startup_timeout, connectOwnedRuntimeHostWithDependencies() still
awaits launch.spawned, so a late or never-settling candidate report delays or permanently blocks
the result beyond the election deadline. This violates the stated behavior that a missed election
returns without waiting for the late report.
Code

packages/runtime-host/src/client/connect-or-spawn.ts[130]

+ const host = await launch?.spawned.catch(() => undefined);
Relevance

●●● Strong

Accepted precedent supports fixing blocking async cleanup; this is a deterministic missed-deadline
correctness bug matching PR intent.

PR-#2674

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The election waits for launch.spawned only through the deadline and then returns
startup_timeout, but the outer owned wrapper immediately awaits the original unbounded promise
again. The added test delays rejection for 20 ms after a 1 ms deadline yet checks only the returned
reason, so it passes even though the function waits for that late report; replacing the test promise
with a never-settling promise reproducibly prevents completion.

packages/runtime-host/src/client/connect-or-spawn.ts[249-260]
packages/runtime-host/src/client/connect-or-spawn.ts[293-297]
packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
packages/runtime-host/src/tests/owned-candidate.test.ts[50-74]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`connectOwnedRuntimeHostWithDependencies()` unconditionally awaits the owned candidate's `spawned` promise after the bounded election completes. Consequently, a non-connected election result cannot return until that late promise settles, and a never-settling report hangs indefinitely.
## Issue Context
The election already bounds candidate waiting with `settleBeforeDeadline()` and returns `startup_timeout`. Preserve that result immediately when it is non-connected; attach non-blocking cleanup handlers for a candidate that reports later rather than awaiting it. Extend the regression test to prove completion occurs near the election deadline with a delayed or never-settling report.
## Fix Focus Areas
- packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
- packages/runtime-host/src/__tests__/owned-candidate.test.ts[50-74]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⚖️ Balanced: This changes runtime connection/election behavior and error handling in an asynchronous host lifecycle path, so it carries genuine correctness risk, but the logic is localized enough for a careful single-pass review.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 92e25a18-7c46-472b-b5f2-0caa3a90d1eb

📥 Commits

Reviewing files that changed from the base of the PR and between 21929e3 and f1f11fb.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 2 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated
Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning at the election deadline instead of awaiting a never-settling spawn is the right goal, but the new fire-and-forget cleanup transfers an unsafe kill decision to the timed-out client. A late candidate can become the shared State Root host and accept another client before this cleanup fires; settle(1_000) then kills that live host. The owner of the process must decide whether it is idle, rather than a client that already lost the election. The regression test also needs to cover a candidate that resolves late, not only one that rejects.

AI-assisted review: Codex coordinated two independent reviewer passes and an OpenCode Go DeepSeek V4 Flash high-effort adversarial pass. I verified the exact head f944c30984deeeff827db3871f3cf122c4c249de, the launcher kill behavior, the State Root connection path, and current CI. No local tests were run in this review.

中文审查

不再等待永不结束的 spawn、按 election deadline 返回,这个目标正确;但新的后台清理把危险的 kill 决策交给了已经超时的客户端。迟到 candidate 可能已经成为同一 State Root 的共享 Host,并接入另一个客户端,随后仍会被 settle(1_000) 杀死。进程是否空闲应由 Host 自身判断,而不是由已经输掉选举的客户端决定。回归测试也需要覆盖 candidate 迟到但成功 resolve 的路径,而不只是 reject。

本次为 AI 辅助审查:Codex 协调两轮独立 reviewer 审查及 OpenCode Go DeepSeek V4 Flash high 对抗性复核,并核验了精确 head、launcher kill 行为、State Root 连接路径和当前 CI。本轮未运行本地测试。

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated
Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/runtime-host/src/__tests__/owned-candidate.test.ts (1)

301-317: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consolidate the duplicate connection-result formatters.

ownedConnectFailure and adoptedConnectFailure have identical branches. Use one helper with a union of the two result types. This removes duplicate maintenance when result kinds change.

Disposition: optional.

As per path instructions, “Flag concrete cases where code can be deleted or simplified.”

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e4a9d78f-8cc3-422b-a298-18cdecac0b9d

📥 Commits

Reviewing files that changed from the base of the PR and between f944c30 and 95e0072.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts
…ndow
The promptness claim is Host exit after the first connection closes,
not the connect itself. A two-second election misses spawn plus
handshake under a full suite and only reports `failed`. Widen the
window, keep the concrete reason on the assertion, and map a
deadline throw to startup_timeout.
Fixesapache#3190
Election already bounds candidate startup. Awaiting launch.spawned
after a non-connected result hangs connectOwned when the candidate
never reports. Return that result immediately and settle a late
host in the background.
Fixesapache#3190
A timed-out owned connect no longer settles a late candidate. That
path sent SIGKILL after 1s and could terminate a host another client
had already adopted. Release the process and let idle-grace /
initial-connection policy exit it when unused.
Fixesapache#3190
…tling
The adopt regression only checked that the late pid stayed alive. Wrap
the delivered candidate so a settle() cleanup cannot pass that check
and kill the adopted host after the assertion.
Fixesapache#3190
@1625567290
1625567290force-pushed the fix/host-owned-candidate-connection-flake branch from 95e0072 to 8ca108eCompareAugust 18, 2026 17:05

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head resolves the earlier ownership race correctly. A missed election now returns at its own deadline, preserves the specific failure reason, and releases a late owned candidate to the Runtime Host lifecycle instead of killing a process another client may have adopted. The implementation extends the existing OwnedCandidateAttempt seam and does not create another election authority.

I found no reproducible P0–P3 issue on this head. I also checked two adversarial candidates and rejected them: root ownership prevents the pid-mismatch candidate from simultaneously being the adopted authority, and the test's deliberately unresolved spawn promise occupies the first election until its deadline, so the real candidate cannot make that call enter the connected branch.

The current windows_recovery failure is in the unchanged Local IPC trust fixture (insecure_endpoint_directory), while the affected Runtime Host checks pass. This approval is for the code; the PR is not merge-ready until required CI is green.

AI-assisted review disclosure: Codex performed the final review using two independent reviewer passes and OpenCode Go DeepSeek V4 Flash (high effort) as an adversarial advisory pass. I verified the election state machine, root ownership, tests, exact head 8ca108e3e, and the failing CI log. No local tests were run.

中文评论

当前 head 已正确解决此前的 ownership race:missed election 会在自己的 deadline 返回并保留具体失败原因;迟到的 owned candidate 会交还 Runtime Host lifecycle,而不是杀掉可能已被其他 client 接管的进程。实现直接扩展既有 OwnedCandidateAttempt seam,没有新增 election authority。

当前 head 未发现可复现的 P0–P3。我也复核并排除了两个对抗性候选:root ownership 保证 pid mismatch 的 candidate 不可能同时成为被接管的权威;测试中故意不 resolve 的 spawn promise 会占用第一次 election 直到 deadline,因此真实 candidate 不会让该调用进入 connected 分支。

当前 windows_recovery 失败来自未改动的 Local IPC trust fixture(insecure_endpoint_directory),受影响的 Runtime Host checks 已通过。本次 approve 针对代码;必需 CI 全绿前不能合并。

AI 辅助审查说明:Codex 使用两轮独立 reviewer 审查,并以 OpenCode Go DeepSeek V4 Flash(high effort)进行对抗性辅助审查;我已核对 election 状态机、root ownership、测试、精确 head 8ca108e3e 和失败 CI 日志。本轮未运行本地测试。

@Astro-Han

Copy link
Copy Markdown
Contributor

@1625567290 Hi, Could we state that if there is any ai tools used in the PR?

@Astro-Han
Astro-Han merged commit 803edc1 into apache:mainAug 20, 2026
18 of 19 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(runtime-host): investigate owned candidate connection flake

2 participants

@1625567290@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fix(runtime-host): give owned-candidate connect a CI-safe election window by 1625567290 · Pull Request #3221 · apache/maka · GitHub
Skip to content

fix(runtime-host): give owned-candidate connect a CI-safe election window - #3221

Merged
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake
Aug 20, 2026
Merged

fix(runtime-host): give owned-candidate connect a CI-safe election window#3221
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake

Conversation

@1625567290

Copy link
Copy Markdown
Contributor

Summary

owned Host exits promptly after its first connection closes flakes under the full Runtime Host suite. CI run 32101082229 finished 962/963 tests and failed this one at 2.19s with failed instead of connected. The 2s election window is enough on an idle machine and too short under suite load. The promptness claim is Host exit after the first connection closes, not the connect itself.

  • Widen that test's election window to 8s and keep settle(500) as the prompt-exit check.
  • Put the concrete failed:<reason> on the assertion so a later miss is diagnosable.
  • A late candidate report after the election ends no longer blocks return; the missed election stays startup_timeout.

Fixes#3190

Verification

  • node --test packages/runtime-host/dist/__tests__/owned-candidate.test.js — 7/7

Checklist

  • Tests cover the change and fail without it
  • Focused lint/typecheck and the affected suites pass locally
  • Full workspace lint/format/typecheck

Does this PR entail a change in behavior?

  • Yes — owned connect under a missed election returns without waiting on a late candidate report, and the flaking test uses an 8s election window

@coderabbitai

coderabbitaiBot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b6dc0fd2-ecac-44d4-a27f-7360a9c69d26

📥 Commits

Reviewing files that changed from the base of the PR and between 95e0072 and 8ca108e.

📒 Files selected for processing (1)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/tests/owned-candidate.test.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.


📝 Walkthrough

What problem this solves

This PR reduces CI timing flakes in the Runtime Host owned-candidate connection test.

  • It increases the election window from 2 seconds to 8 seconds.
  • It keeps settle(500) as the prompt-exit check after the first connection closes.
  • It preserves and asserts the concrete failed:<reason> result.
  • It returns startup_timeout when a candidate reports after a missed election.
  • It classifies missing or failed launch.spawned results as host_unresponsive.
  • It releases late spawned hosts instead of terminating them with SIGKILL.

Source of truth

The PR extends the existing owned-candidate connection, election, and launch-cleanup paths. It does not create a parallel connection path or add a public API.

Scope and simplification

The changes are the smallest coherent solution for the reported timing failure and late-startup behavior.

Asynchronous releaseOwnedLaunch cleanup is necessary because a late candidate must not block the startup_timeout result. It also avoids terminating a host that another client may adopt.

The diagnostic formatter, polling helper, and regression tests preserve failure details and cover the timing cases. No deletion or simplification is apparent without weakening diagnostics or regression coverage.

Validation and risks

Focused verification passed: 7/7 tests in owned-candidate.test.js.

The tests cover the longer election window, concrete failure reporting, unresolved candidates, and late candidate adoption. Full-suite validation is not reported, so required checks remain unverified.

Complexity delta

  • Authorities: No new source of truth.
  • States: Clarifies existing host_unresponsive, existing_host, and startup_timeout results.
  • Branches: Adds handling for rejected or missing launch.spawned results and asynchronous release cleanup.
  • Configuration: Increases the test election deadline from 2 seconds to 8 seconds.
  • Public surface: No exported or public entities change.
  • Test maintenance: Adds regression coverage, diagnostics, and a polling helper.

Total maintenance complexity stays justified. The added branches and tests directly cover the reported failure modes.

Review-relevant risks

The connection behavior changes when launch.spawned fails, returns no host, or reports after a missed election. The cleanup behavior also changes from termination to release. These changes can affect Runtime Host error classification, host ownership, and user-visible connection outcomes. Material changes in these areas require independent human review under repository policy.

No security, licensing, release, or governance effect was identified in the current diff.

Required checks remain unverified beyond the reported focused test result. The person performing the merge reviews the final diff. A maintainer makes the final determination.

Walkthrough

The runtime-host client now classifies owned host startup failures and distinguishes launch failures from existing-host outcomes. Tests cover startup timeouts, late candidates, election timing, cleanup, and connection diagnostics.

Changes

Owned runtime-host connection handling

Layer / File(s)Summary
Classify and clean up startup failures
packages/runtime-host/src/client/connect-or-spawn.ts
Owned host startup failures and missing hosts produce host_unresponsive. Connected results produce existing_host only when no launch occurs. Failure cleanup releases owned launches and handles rejected spawn promises.
Cover timeout and late-candidate behavior
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Tests cover unresolved candidates, late candidate adoption, candidate cleanup, pending promises, and the extended election deadline.
Add connection diagnostics and polling
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Test helpers format connection results and poll for delayed candidate startup.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk:⚪ Minimal · up to 8ca10

This localized change widens the owned-candidate election window and makes missed-election failures diagnosable; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check nameStatusExplanationResolution
Out of Scope Changes check⚠️ WarningThe connect-or-spawn.ts changes alter production candidate-handling behavior beyond the linked issue's test-flake investigation scope.Separate the production behavior change into a linked issue or explain its direct necessity for issue #3190.
Ai Use Disclosure⚠️ WarningThe PR description selects neither disclosure statement, and all five PR commits lack a valid Generated-by trailer.Add exactly one required declaration. If generative tooling made a substantive contribution, name the tool and scope; if it authored material content, add a trailer that survives squash/amend. See “Human ownership and AI attribution” in...
✅ Passed checks (3 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly describes the main change: extending the owned-candidate election window for CI reliability.
Description check✅ PassedThe description covers the problem, fix, issue reference, verification, tests, and behavior change, but omits the required AI use section.
Linked Issues check✅ PassedThe PR preserves the failure reason and increases the election deadline, addressing the timing-flake objectives in issue #3190.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Stabilize owned-candidate election handling under CI load

🐞 Bug fix🧪 Tests🕐 10-20 Minutes

Grey Divider

AI Description

• Preserve startup timeouts when candidate spawn failures arrive after the election deadline.
• Extend the disconnect test election window to eight seconds for loaded CI.
• Add regression coverage and concrete diagnostics for owned connection failures.
Diagram

sequenceDiagram
actor Caller
participant Owned as Owned Connect
participant Election as Host Election
participant Candidate as Owned Candidate
Caller->>Owned: request connection
Owned->>Election: connect or spawn
Election->>Candidate: launch candidate
Note over Election: deadline expires
Election-->>Owned: startup_timeout
Candidate--x Owned: late spawn failure
Owned-->>Caller: preserve timeout
Loading
High-Level Assessment

The focused wrapper change is appropriate: it treats a rejected spawn report as an unavailable owned host while preserving an already-determined election failure. Returning immediately without observing the candidate promise could reduce latency further, but risks losing ownership and cleanup of a process that eventually spawns; introducing launch cancellation would be disproportionate for this fix.

Files changed (2) +46 / -4

Bug fix (1) +6 / -2
connect-or-spawn.tsPreserve owned election failures after spawn rejection+6/-2

Preserve owned election failures after spawn rejection

• Handles rejected owned-candidate spawn promises without falling through to the generic exception path. Existing election failures are returned unchanged, while connected results without a usable launched host are classified as 'host_unresponsive' or 'existing_host'.

packages/runtime-host/src/client/connect-or-spawn.ts

Tests (1) +40 / -2
owned-candidate.test.tsCover late candidate reports and stabilize the disconnect lifecycle test+40/-2

Cover late candidate reports and stabilize the disconnect lifecycle test

• Adds a regression test proving that a late rejected candidate report does not replace 'startup_timeout'. Expands the disconnect test election window from two to eight seconds and reports concrete failure reasons while retaining the 500ms host-exit assertion.

packages/runtime-host/src/tests/owned-candidate.test.ts

@qodo-code-review

qodo-code-reviewBot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0)📘 Rule violations (0)📜 Skill insights (0)

Grey Divider


Action required

1. Late report still blocks return✓ Resolved🐞 Bug≡ Correctness
Description
After the election produces startup_timeout, connectOwnedRuntimeHostWithDependencies() still
awaits launch.spawned, so a late or never-settling candidate report delays or permanently blocks
the result beyond the election deadline. This violates the stated behavior that a missed election
returns without waiting for the late report.
Code

packages/runtime-host/src/client/connect-or-spawn.ts[130]

+ const host = await launch?.spawned.catch(() => undefined);
Relevance

●●● Strong

Accepted precedent supports fixing blocking async cleanup; this is a deterministic missed-deadline
correctness bug matching PR intent.

PR-#2674

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The election waits for launch.spawned only through the deadline and then returns
startup_timeout, but the outer owned wrapper immediately awaits the original unbounded promise
again. The added test delays rejection for 20 ms after a 1 ms deadline yet checks only the returned
reason, so it passes even though the function waits for that late report; replacing the test promise
with a never-settling promise reproducibly prevents completion.

packages/runtime-host/src/client/connect-or-spawn.ts[249-260]
packages/runtime-host/src/client/connect-or-spawn.ts[293-297]
packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
packages/runtime-host/src/tests/owned-candidate.test.ts[50-74]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`connectOwnedRuntimeHostWithDependencies()` unconditionally awaits the owned candidate's `spawned` promise after the bounded election completes. Consequently, a non-connected election result cannot return until that late promise settles, and a never-settling report hangs indefinitely.
## Issue Context
The election already bounds candidate waiting with `settleBeforeDeadline()` and returns `startup_timeout`. Preserve that result immediately when it is non-connected; attach non-blocking cleanup handlers for a candidate that reports later rather than awaiting it. Extend the regression test to prove completion occurs near the election deadline with a delayed or never-settling report.
## Fix Focus Areas
- packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
- packages/runtime-host/src/__tests__/owned-candidate.test.ts[50-74]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⚖️ Balanced: This changes runtime connection/election behavior and error handling in an asynchronous host lifecycle path, so it carries genuine correctness risk, but the logic is localized enough for a careful single-pass review.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 92e25a18-7c46-472b-b5f2-0caa3a90d1eb

📥 Commits

Reviewing files that changed from the base of the PR and between 21929e3 and f1f11fb.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 2 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated
Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning at the election deadline instead of awaiting a never-settling spawn is the right goal, but the new fire-and-forget cleanup transfers an unsafe kill decision to the timed-out client. A late candidate can become the shared State Root host and accept another client before this cleanup fires; settle(1_000) then kills that live host. The owner of the process must decide whether it is idle, rather than a client that already lost the election. The regression test also needs to cover a candidate that resolves late, not only one that rejects.

AI-assisted review: Codex coordinated two independent reviewer passes and an OpenCode Go DeepSeek V4 Flash high-effort adversarial pass. I verified the exact head f944c30984deeeff827db3871f3cf122c4c249de, the launcher kill behavior, the State Root connection path, and current CI. No local tests were run in this review.

中文审查

不再等待永不结束的 spawn、按 election deadline 返回,这个目标正确;但新的后台清理把危险的 kill 决策交给了已经超时的客户端。迟到 candidate 可能已经成为同一 State Root 的共享 Host,并接入另一个客户端,随后仍会被 settle(1_000) 杀死。进程是否空闲应由 Host 自身判断,而不是由已经输掉选举的客户端决定。回归测试也需要覆盖 candidate 迟到但成功 resolve 的路径,而不只是 reject。

本次为 AI 辅助审查:Codex 协调两轮独立 reviewer 审查及 OpenCode Go DeepSeek V4 Flash high 对抗性复核,并核验了精确 head、launcher kill 行为、State Root 连接路径和当前 CI。本轮未运行本地测试。

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated
Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/runtime-host/src/__tests__/owned-candidate.test.ts (1)

301-317: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consolidate the duplicate connection-result formatters.

ownedConnectFailure and adoptedConnectFailure have identical branches. Use one helper with a union of the two result types. This removes duplicate maintenance when result kinds change.

Disposition: optional.

As per path instructions, “Flag concrete cases where code can be deleted or simplified.”

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e4a9d78f-8cc3-422b-a298-18cdecac0b9d

📥 Commits

Reviewing files that changed from the base of the PR and between f944c30 and 95e0072.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts
…ndow
The promptness claim is Host exit after the first connection closes,
not the connect itself. A two-second election misses spawn plus
handshake under a full suite and only reports `failed`. Widen the
window, keep the concrete reason on the assertion, and map a
deadline throw to startup_timeout.
Fixesapache#3190
Election already bounds candidate startup. Awaiting launch.spawned
after a non-connected result hangs connectOwned when the candidate
never reports. Return that result immediately and settle a late
host in the background.
Fixesapache#3190
A timed-out owned connect no longer settles a late candidate. That
path sent SIGKILL after 1s and could terminate a host another client
had already adopted. Release the process and let idle-grace /
initial-connection policy exit it when unused.
Fixesapache#3190
…tling
The adopt regression only checked that the late pid stayed alive. Wrap
the delivered candidate so a settle() cleanup cannot pass that check
and kill the adopted host after the assertion.
Fixesapache#3190
@1625567290
1625567290force-pushed the fix/host-owned-candidate-connection-flake branch from 95e0072 to 8ca108eCompareAugust 18, 2026 17:05

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head resolves the earlier ownership race correctly. A missed election now returns at its own deadline, preserves the specific failure reason, and releases a late owned candidate to the Runtime Host lifecycle instead of killing a process another client may have adopted. The implementation extends the existing OwnedCandidateAttempt seam and does not create another election authority.

I found no reproducible P0–P3 issue on this head. I also checked two adversarial candidates and rejected them: root ownership prevents the pid-mismatch candidate from simultaneously being the adopted authority, and the test's deliberately unresolved spawn promise occupies the first election until its deadline, so the real candidate cannot make that call enter the connected branch.

The current windows_recovery failure is in the unchanged Local IPC trust fixture (insecure_endpoint_directory), while the affected Runtime Host checks pass. This approval is for the code; the PR is not merge-ready until required CI is green.

AI-assisted review disclosure: Codex performed the final review using two independent reviewer passes and OpenCode Go DeepSeek V4 Flash (high effort) as an adversarial advisory pass. I verified the election state machine, root ownership, tests, exact head 8ca108e3e, and the failing CI log. No local tests were run.

中文评论

当前 head 已正确解决此前的 ownership race:missed election 会在自己的 deadline 返回并保留具体失败原因;迟到的 owned candidate 会交还 Runtime Host lifecycle,而不是杀掉可能已被其他 client 接管的进程。实现直接扩展既有 OwnedCandidateAttempt seam,没有新增 election authority。

当前 head 未发现可复现的 P0–P3。我也复核并排除了两个对抗性候选:root ownership 保证 pid mismatch 的 candidate 不可能同时成为被接管的权威;测试中故意不 resolve 的 spawn promise 会占用第一次 election 直到 deadline,因此真实 candidate 不会让该调用进入 connected 分支。

当前 windows_recovery 失败来自未改动的 Local IPC trust fixture(insecure_endpoint_directory),受影响的 Runtime Host checks 已通过。本次 approve 针对代码;必需 CI 全绿前不能合并。

AI 辅助审查说明:Codex 使用两轮独立 reviewer 审查,并以 OpenCode Go DeepSeek V4 Flash(high effort)进行对抗性辅助审查;我已核对 election 状态机、root ownership、测试、精确 head 8ca108e3e 和失败 CI 日志。本轮未运行本地测试。

@Astro-Han

Copy link
Copy Markdown
Contributor

@1625567290 Hi, Could we state that if there is any ai tools used in the PR?

@Astro-Han
Astro-Han merged commit 803edc1 into apache:mainAug 20, 2026
18 of 19 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(runtime-host): investigate owned candidate connection flake

2 participants

@1625567290@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' fix(runtime-host): give owned-candidate connect a CI-safe election window by 1625567290 · Pull Request #3221 · apache/maka · GitHub
Skip to content

fix(runtime-host): give owned-candidate connect a CI-safe election window - #3221

Merged
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake
Aug 20, 2026
Merged

fix(runtime-host): give owned-candidate connect a CI-safe election window#3221
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake

Conversation

@1625567290

Copy link
Copy Markdown
Contributor

Summary

owned Host exits promptly after its first connection closes flakes under the full Runtime Host suite. CI run 32101082229 finished 962/963 tests and failed this one at 2.19s with failed instead of connected. The 2s election window is enough on an idle machine and too short under suite load. The promptness claim is Host exit after the first connection closes, not the connect itself.

  • Widen that test's election window to 8s and keep settle(500) as the prompt-exit check.
  • Put the concrete failed:<reason> on the assertion so a later miss is diagnosable.
  • A late candidate report after the election ends no longer blocks return; the missed election stays startup_timeout.

Fixes#3190

Verification

  • node --test packages/runtime-host/dist/__tests__/owned-candidate.test.js — 7/7

Checklist

  • Tests cover the change and fail without it
  • Focused lint/typecheck and the affected suites pass locally
  • Full workspace lint/format/typecheck

Does this PR entail a change in behavior?

  • Yes — owned connect under a missed election returns without waiting on a late candidate report, and the flaking test uses an 8s election window

@coderabbitai

coderabbitaiBot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b6dc0fd2-ecac-44d4-a27f-7360a9c69d26

📥 Commits

Reviewing files that changed from the base of the PR and between 95e0072 and 8ca108e.

📒 Files selected for processing (1)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/tests/owned-candidate.test.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.


📝 Walkthrough

What problem this solves

This PR reduces CI timing flakes in the Runtime Host owned-candidate connection test.

  • It increases the election window from 2 seconds to 8 seconds.
  • It keeps settle(500) as the prompt-exit check after the first connection closes.
  • It preserves and asserts the concrete failed:<reason> result.
  • It returns startup_timeout when a candidate reports after a missed election.
  • It classifies missing or failed launch.spawned results as host_unresponsive.
  • It releases late spawned hosts instead of terminating them with SIGKILL.

Source of truth

The PR extends the existing owned-candidate connection, election, and launch-cleanup paths. It does not create a parallel connection path or add a public API.

Scope and simplification

The changes are the smallest coherent solution for the reported timing failure and late-startup behavior.

Asynchronous releaseOwnedLaunch cleanup is necessary because a late candidate must not block the startup_timeout result. It also avoids terminating a host that another client may adopt.

The diagnostic formatter, polling helper, and regression tests preserve failure details and cover the timing cases. No deletion or simplification is apparent without weakening diagnostics or regression coverage.

Validation and risks

Focused verification passed: 7/7 tests in owned-candidate.test.js.

The tests cover the longer election window, concrete failure reporting, unresolved candidates, and late candidate adoption. Full-suite validation is not reported, so required checks remain unverified.

Complexity delta

  • Authorities: No new source of truth.
  • States: Clarifies existing host_unresponsive, existing_host, and startup_timeout results.
  • Branches: Adds handling for rejected or missing launch.spawned results and asynchronous release cleanup.
  • Configuration: Increases the test election deadline from 2 seconds to 8 seconds.
  • Public surface: No exported or public entities change.
  • Test maintenance: Adds regression coverage, diagnostics, and a polling helper.

Total maintenance complexity stays justified. The added branches and tests directly cover the reported failure modes.

Review-relevant risks

The connection behavior changes when launch.spawned fails, returns no host, or reports after a missed election. The cleanup behavior also changes from termination to release. These changes can affect Runtime Host error classification, host ownership, and user-visible connection outcomes. Material changes in these areas require independent human review under repository policy.

No security, licensing, release, or governance effect was identified in the current diff.

Required checks remain unverified beyond the reported focused test result. The person performing the merge reviews the final diff. A maintainer makes the final determination.

Walkthrough

The runtime-host client now classifies owned host startup failures and distinguishes launch failures from existing-host outcomes. Tests cover startup timeouts, late candidates, election timing, cleanup, and connection diagnostics.

Changes

Owned runtime-host connection handling

Layer / File(s)Summary
Classify and clean up startup failures
packages/runtime-host/src/client/connect-or-spawn.ts
Owned host startup failures and missing hosts produce host_unresponsive. Connected results produce existing_host only when no launch occurs. Failure cleanup releases owned launches and handles rejected spawn promises.
Cover timeout and late-candidate behavior
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Tests cover unresolved candidates, late candidate adoption, candidate cleanup, pending promises, and the extended election deadline.
Add connection diagnostics and polling
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Test helpers format connection results and poll for delayed candidate startup.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk:⚪ Minimal · up to 8ca10

This localized change widens the owned-candidate election window and makes missed-election failures diagnosable; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check nameStatusExplanationResolution
Out of Scope Changes check⚠️ WarningThe connect-or-spawn.ts changes alter production candidate-handling behavior beyond the linked issue's test-flake investigation scope.Separate the production behavior change into a linked issue or explain its direct necessity for issue #3190.
Ai Use Disclosure⚠️ WarningThe PR description selects neither disclosure statement, and all five PR commits lack a valid Generated-by trailer.Add exactly one required declaration. If generative tooling made a substantive contribution, name the tool and scope; if it authored material content, add a trailer that survives squash/amend. See “Human ownership and AI attribution” in...
✅ Passed checks (3 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly describes the main change: extending the owned-candidate election window for CI reliability.
Description check✅ PassedThe description covers the problem, fix, issue reference, verification, tests, and behavior change, but omits the required AI use section.
Linked Issues check✅ PassedThe PR preserves the failure reason and increases the election deadline, addressing the timing-flake objectives in issue #3190.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Stabilize owned-candidate election handling under CI load

🐞 Bug fix🧪 Tests🕐 10-20 Minutes

Grey Divider

AI Description

• Preserve startup timeouts when candidate spawn failures arrive after the election deadline.
• Extend the disconnect test election window to eight seconds for loaded CI.
• Add regression coverage and concrete diagnostics for owned connection failures.
Diagram

sequenceDiagram
actor Caller
participant Owned as Owned Connect
participant Election as Host Election
participant Candidate as Owned Candidate
Caller->>Owned: request connection
Owned->>Election: connect or spawn
Election->>Candidate: launch candidate
Note over Election: deadline expires
Election-->>Owned: startup_timeout
Candidate--x Owned: late spawn failure
Owned-->>Caller: preserve timeout
Loading
High-Level Assessment

The focused wrapper change is appropriate: it treats a rejected spawn report as an unavailable owned host while preserving an already-determined election failure. Returning immediately without observing the candidate promise could reduce latency further, but risks losing ownership and cleanup of a process that eventually spawns; introducing launch cancellation would be disproportionate for this fix.

Files changed (2) +46 / -4

Bug fix (1) +6 / -2
connect-or-spawn.tsPreserve owned election failures after spawn rejection+6/-2

Preserve owned election failures after spawn rejection

• Handles rejected owned-candidate spawn promises without falling through to the generic exception path. Existing election failures are returned unchanged, while connected results without a usable launched host are classified as 'host_unresponsive' or 'existing_host'.

packages/runtime-host/src/client/connect-or-spawn.ts

Tests (1) +40 / -2
owned-candidate.test.tsCover late candidate reports and stabilize the disconnect lifecycle test+40/-2

Cover late candidate reports and stabilize the disconnect lifecycle test

• Adds a regression test proving that a late rejected candidate report does not replace 'startup_timeout'. Expands the disconnect test election window from two to eight seconds and reports concrete failure reasons while retaining the 500ms host-exit assertion.

packages/runtime-host/src/tests/owned-candidate.test.ts

@qodo-code-review

qodo-code-reviewBot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0)📘 Rule violations (0)📜 Skill insights (0)

Grey Divider


Action required

1. Late report still blocks return✓ Resolved🐞 Bug≡ Correctness
Description
After the election produces startup_timeout, connectOwnedRuntimeHostWithDependencies() still
awaits launch.spawned, so a late or never-settling candidate report delays or permanently blocks
the result beyond the election deadline. This violates the stated behavior that a missed election
returns without waiting for the late report.
Code

packages/runtime-host/src/client/connect-or-spawn.ts[130]

+ const host = await launch?.spawned.catch(() => undefined);
Relevance

●●● Strong

Accepted precedent supports fixing blocking async cleanup; this is a deterministic missed-deadline
correctness bug matching PR intent.

PR-#2674

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The election waits for launch.spawned only through the deadline and then returns
startup_timeout, but the outer owned wrapper immediately awaits the original unbounded promise
again. The added test delays rejection for 20 ms after a 1 ms deadline yet checks only the returned
reason, so it passes even though the function waits for that late report; replacing the test promise
with a never-settling promise reproducibly prevents completion.

packages/runtime-host/src/client/connect-or-spawn.ts[249-260]
packages/runtime-host/src/client/connect-or-spawn.ts[293-297]
packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
packages/runtime-host/src/tests/owned-candidate.test.ts[50-74]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`connectOwnedRuntimeHostWithDependencies()` unconditionally awaits the owned candidate's `spawned` promise after the bounded election completes. Consequently, a non-connected election result cannot return until that late promise settles, and a never-settling report hangs indefinitely.
## Issue Context
The election already bounds candidate waiting with `settleBeforeDeadline()` and returns `startup_timeout`. Preserve that result immediately when it is non-connected; attach non-blocking cleanup handlers for a candidate that reports later rather than awaiting it. Extend the regression test to prove completion occurs near the election deadline with a delayed or never-settling report.
## Fix Focus Areas
- packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
- packages/runtime-host/src/__tests__/owned-candidate.test.ts[50-74]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⚖️ Balanced: This changes runtime connection/election behavior and error handling in an asynchronous host lifecycle path, so it carries genuine correctness risk, but the logic is localized enough for a careful single-pass review.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 92e25a18-7c46-472b-b5f2-0caa3a90d1eb

📥 Commits

Reviewing files that changed from the base of the PR and between 21929e3 and f1f11fb.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 2 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated
Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning at the election deadline instead of awaiting a never-settling spawn is the right goal, but the new fire-and-forget cleanup transfers an unsafe kill decision to the timed-out client. A late candidate can become the shared State Root host and accept another client before this cleanup fires; settle(1_000) then kills that live host. The owner of the process must decide whether it is idle, rather than a client that already lost the election. The regression test also needs to cover a candidate that resolves late, not only one that rejects.

AI-assisted review: Codex coordinated two independent reviewer passes and an OpenCode Go DeepSeek V4 Flash high-effort adversarial pass. I verified the exact head f944c30984deeeff827db3871f3cf122c4c249de, the launcher kill behavior, the State Root connection path, and current CI. No local tests were run in this review.

中文审查

不再等待永不结束的 spawn、按 election deadline 返回,这个目标正确;但新的后台清理把危险的 kill 决策交给了已经超时的客户端。迟到 candidate 可能已经成为同一 State Root 的共享 Host,并接入另一个客户端,随后仍会被 settle(1_000) 杀死。进程是否空闲应由 Host 自身判断,而不是由已经输掉选举的客户端决定。回归测试也需要覆盖 candidate 迟到但成功 resolve 的路径,而不只是 reject。

本次为 AI 辅助审查:Codex 协调两轮独立 reviewer 审查及 OpenCode Go DeepSeek V4 Flash high 对抗性复核,并核验了精确 head、launcher kill 行为、State Root 连接路径和当前 CI。本轮未运行本地测试。

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated
Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/runtime-host/src/__tests__/owned-candidate.test.ts (1)

301-317: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consolidate the duplicate connection-result formatters.

ownedConnectFailure and adoptedConnectFailure have identical branches. Use one helper with a union of the two result types. This removes duplicate maintenance when result kinds change.

Disposition: optional.

As per path instructions, “Flag concrete cases where code can be deleted or simplified.”

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e4a9d78f-8cc3-422b-a298-18cdecac0b9d

📥 Commits

Reviewing files that changed from the base of the PR and between f944c30 and 95e0072.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts
…ndow
The promptness claim is Host exit after the first connection closes,
not the connect itself. A two-second election misses spawn plus
handshake under a full suite and only reports `failed`. Widen the
window, keep the concrete reason on the assertion, and map a
deadline throw to startup_timeout.
Fixesapache#3190
Election already bounds candidate startup. Awaiting launch.spawned
after a non-connected result hangs connectOwned when the candidate
never reports. Return that result immediately and settle a late
host in the background.
Fixesapache#3190
A timed-out owned connect no longer settles a late candidate. That
path sent SIGKILL after 1s and could terminate a host another client
had already adopted. Release the process and let idle-grace /
initial-connection policy exit it when unused.
Fixesapache#3190
…tling
The adopt regression only checked that the late pid stayed alive. Wrap
the delivered candidate so a settle() cleanup cannot pass that check
and kill the adopted host after the assertion.
Fixesapache#3190
@1625567290
1625567290force-pushed the fix/host-owned-candidate-connection-flake branch from 95e0072 to 8ca108eCompareAugust 18, 2026 17:05

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head resolves the earlier ownership race correctly. A missed election now returns at its own deadline, preserves the specific failure reason, and releases a late owned candidate to the Runtime Host lifecycle instead of killing a process another client may have adopted. The implementation extends the existing OwnedCandidateAttempt seam and does not create another election authority.

I found no reproducible P0–P3 issue on this head. I also checked two adversarial candidates and rejected them: root ownership prevents the pid-mismatch candidate from simultaneously being the adopted authority, and the test's deliberately unresolved spawn promise occupies the first election until its deadline, so the real candidate cannot make that call enter the connected branch.

The current windows_recovery failure is in the unchanged Local IPC trust fixture (insecure_endpoint_directory), while the affected Runtime Host checks pass. This approval is for the code; the PR is not merge-ready until required CI is green.

AI-assisted review disclosure: Codex performed the final review using two independent reviewer passes and OpenCode Go DeepSeek V4 Flash (high effort) as an adversarial advisory pass. I verified the election state machine, root ownership, tests, exact head 8ca108e3e, and the failing CI log. No local tests were run.

中文评论

当前 head 已正确解决此前的 ownership race:missed election 会在自己的 deadline 返回并保留具体失败原因;迟到的 owned candidate 会交还 Runtime Host lifecycle,而不是杀掉可能已被其他 client 接管的进程。实现直接扩展既有 OwnedCandidateAttempt seam,没有新增 election authority。

当前 head 未发现可复现的 P0–P3。我也复核并排除了两个对抗性候选:root ownership 保证 pid mismatch 的 candidate 不可能同时成为被接管的权威;测试中故意不 resolve 的 spawn promise 会占用第一次 election 直到 deadline,因此真实 candidate 不会让该调用进入 connected 分支。

当前 windows_recovery 失败来自未改动的 Local IPC trust fixture(insecure_endpoint_directory),受影响的 Runtime Host checks 已通过。本次 approve 针对代码;必需 CI 全绿前不能合并。

AI 辅助审查说明:Codex 使用两轮独立 reviewer 审查,并以 OpenCode Go DeepSeek V4 Flash(high effort)进行对抗性辅助审查;我已核对 election 状态机、root ownership、测试、精确 head 8ca108e3e 和失败 CI 日志。本轮未运行本地测试。

@Astro-Han

Copy link
Copy Markdown
Contributor

@1625567290 Hi, Could we state that if there is any ai tools used in the PR?

@Astro-Han
Astro-Han merged commit 803edc1 into apache:mainAug 20, 2026
18 of 19 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(runtime-host): investigate owned candidate connection flake

2 participants

@1625567290@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fix(runtime-host): give owned-candidate connect a CI-safe election window by 1625567290 · Pull Request #3221 · apache/maka · GitHub
Skip to content

fix(runtime-host): give owned-candidate connect a CI-safe election window - #3221

Merged
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake
Aug 20, 2026
Merged

fix(runtime-host): give owned-candidate connect a CI-safe election window#3221
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake

Conversation

@1625567290

Copy link
Copy Markdown
Contributor

Summary

owned Host exits promptly after its first connection closes flakes under the full Runtime Host suite. CI run 32101082229 finished 962/963 tests and failed this one at 2.19s with failed instead of connected. The 2s election window is enough on an idle machine and too short under suite load. The promptness claim is Host exit after the first connection closes, not the connect itself.

  • Widen that test's election window to 8s and keep settle(500) as the prompt-exit check.
  • Put the concrete failed:<reason> on the assertion so a later miss is diagnosable.
  • A late candidate report after the election ends no longer blocks return; the missed election stays startup_timeout.

Fixes#3190

Verification

  • node --test packages/runtime-host/dist/__tests__/owned-candidate.test.js — 7/7

Checklist

  • Tests cover the change and fail without it
  • Focused lint/typecheck and the affected suites pass locally
  • Full workspace lint/format/typecheck

Does this PR entail a change in behavior?

  • Yes — owned connect under a missed election returns without waiting on a late candidate report, and the flaking test uses an 8s election window

@coderabbitai

coderabbitaiBot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b6dc0fd2-ecac-44d4-a27f-7360a9c69d26

📥 Commits

Reviewing files that changed from the base of the PR and between 95e0072 and 8ca108e.

📒 Files selected for processing (1)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/tests/owned-candidate.test.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.


📝 Walkthrough

What problem this solves

This PR reduces CI timing flakes in the Runtime Host owned-candidate connection test.

  • It increases the election window from 2 seconds to 8 seconds.
  • It keeps settle(500) as the prompt-exit check after the first connection closes.
  • It preserves and asserts the concrete failed:<reason> result.
  • It returns startup_timeout when a candidate reports after a missed election.
  • It classifies missing or failed launch.spawned results as host_unresponsive.
  • It releases late spawned hosts instead of terminating them with SIGKILL.

Source of truth

The PR extends the existing owned-candidate connection, election, and launch-cleanup paths. It does not create a parallel connection path or add a public API.

Scope and simplification

The changes are the smallest coherent solution for the reported timing failure and late-startup behavior.

Asynchronous releaseOwnedLaunch cleanup is necessary because a late candidate must not block the startup_timeout result. It also avoids terminating a host that another client may adopt.

The diagnostic formatter, polling helper, and regression tests preserve failure details and cover the timing cases. No deletion or simplification is apparent without weakening diagnostics or regression coverage.

Validation and risks

Focused verification passed: 7/7 tests in owned-candidate.test.js.

The tests cover the longer election window, concrete failure reporting, unresolved candidates, and late candidate adoption. Full-suite validation is not reported, so required checks remain unverified.

Complexity delta

  • Authorities: No new source of truth.
  • States: Clarifies existing host_unresponsive, existing_host, and startup_timeout results.
  • Branches: Adds handling for rejected or missing launch.spawned results and asynchronous release cleanup.
  • Configuration: Increases the test election deadline from 2 seconds to 8 seconds.
  • Public surface: No exported or public entities change.
  • Test maintenance: Adds regression coverage, diagnostics, and a polling helper.

Total maintenance complexity stays justified. The added branches and tests directly cover the reported failure modes.

Review-relevant risks

The connection behavior changes when launch.spawned fails, returns no host, or reports after a missed election. The cleanup behavior also changes from termination to release. These changes can affect Runtime Host error classification, host ownership, and user-visible connection outcomes. Material changes in these areas require independent human review under repository policy.

No security, licensing, release, or governance effect was identified in the current diff.

Required checks remain unverified beyond the reported focused test result. The person performing the merge reviews the final diff. A maintainer makes the final determination.

Walkthrough

The runtime-host client now classifies owned host startup failures and distinguishes launch failures from existing-host outcomes. Tests cover startup timeouts, late candidates, election timing, cleanup, and connection diagnostics.

Changes

Owned runtime-host connection handling

Layer / File(s)Summary
Classify and clean up startup failures
packages/runtime-host/src/client/connect-or-spawn.ts
Owned host startup failures and missing hosts produce host_unresponsive. Connected results produce existing_host only when no launch occurs. Failure cleanup releases owned launches and handles rejected spawn promises.
Cover timeout and late-candidate behavior
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Tests cover unresolved candidates, late candidate adoption, candidate cleanup, pending promises, and the extended election deadline.
Add connection diagnostics and polling
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Test helpers format connection results and poll for delayed candidate startup.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk:⚪ Minimal · up to 8ca10

This localized change widens the owned-candidate election window and makes missed-election failures diagnosable; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check nameStatusExplanationResolution
Out of Scope Changes check⚠️ WarningThe connect-or-spawn.ts changes alter production candidate-handling behavior beyond the linked issue's test-flake investigation scope.Separate the production behavior change into a linked issue or explain its direct necessity for issue #3190.
Ai Use Disclosure⚠️ WarningThe PR description selects neither disclosure statement, and all five PR commits lack a valid Generated-by trailer.Add exactly one required declaration. If generative tooling made a substantive contribution, name the tool and scope; if it authored material content, add a trailer that survives squash/amend. See “Human ownership and AI attribution” in...
✅ Passed checks (3 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly describes the main change: extending the owned-candidate election window for CI reliability.
Description check✅ PassedThe description covers the problem, fix, issue reference, verification, tests, and behavior change, but omits the required AI use section.
Linked Issues check✅ PassedThe PR preserves the failure reason and increases the election deadline, addressing the timing-flake objectives in issue #3190.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Stabilize owned-candidate election handling under CI load

🐞 Bug fix🧪 Tests🕐 10-20 Minutes

Grey Divider

AI Description

• Preserve startup timeouts when candidate spawn failures arrive after the election deadline.
• Extend the disconnect test election window to eight seconds for loaded CI.
• Add regression coverage and concrete diagnostics for owned connection failures.
Diagram

sequenceDiagram
actor Caller
participant Owned as Owned Connect
participant Election as Host Election
participant Candidate as Owned Candidate
Caller->>Owned: request connection
Owned->>Election: connect or spawn
Election->>Candidate: launch candidate
Note over Election: deadline expires
Election-->>Owned: startup_timeout
Candidate--x Owned: late spawn failure
Owned-->>Caller: preserve timeout
Loading
High-Level Assessment

The focused wrapper change is appropriate: it treats a rejected spawn report as an unavailable owned host while preserving an already-determined election failure. Returning immediately without observing the candidate promise could reduce latency further, but risks losing ownership and cleanup of a process that eventually spawns; introducing launch cancellation would be disproportionate for this fix.

Files changed (2) +46 / -4

Bug fix (1) +6 / -2
connect-or-spawn.tsPreserve owned election failures after spawn rejection+6/-2

Preserve owned election failures after spawn rejection

• Handles rejected owned-candidate spawn promises without falling through to the generic exception path. Existing election failures are returned unchanged, while connected results without a usable launched host are classified as 'host_unresponsive' or 'existing_host'.

packages/runtime-host/src/client/connect-or-spawn.ts

Tests (1) +40 / -2
owned-candidate.test.tsCover late candidate reports and stabilize the disconnect lifecycle test+40/-2

Cover late candidate reports and stabilize the disconnect lifecycle test

• Adds a regression test proving that a late rejected candidate report does not replace 'startup_timeout'. Expands the disconnect test election window from two to eight seconds and reports concrete failure reasons while retaining the 500ms host-exit assertion.

packages/runtime-host/src/tests/owned-candidate.test.ts

@qodo-code-review

qodo-code-reviewBot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0)📘 Rule violations (0)📜 Skill insights (0)

Grey Divider


Action required

1. Late report still blocks return✓ Resolved🐞 Bug≡ Correctness
Description
After the election produces startup_timeout, connectOwnedRuntimeHostWithDependencies() still
awaits launch.spawned, so a late or never-settling candidate report delays or permanently blocks
the result beyond the election deadline. This violates the stated behavior that a missed election
returns without waiting for the late report.
Code

packages/runtime-host/src/client/connect-or-spawn.ts[130]

+ const host = await launch?.spawned.catch(() => undefined);
Relevance

●●● Strong

Accepted precedent supports fixing blocking async cleanup; this is a deterministic missed-deadline
correctness bug matching PR intent.

PR-#2674

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The election waits for launch.spawned only through the deadline and then returns
startup_timeout, but the outer owned wrapper immediately awaits the original unbounded promise
again. The added test delays rejection for 20 ms after a 1 ms deadline yet checks only the returned
reason, so it passes even though the function waits for that late report; replacing the test promise
with a never-settling promise reproducibly prevents completion.

packages/runtime-host/src/client/connect-or-spawn.ts[249-260]
packages/runtime-host/src/client/connect-or-spawn.ts[293-297]
packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
packages/runtime-host/src/tests/owned-candidate.test.ts[50-74]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`connectOwnedRuntimeHostWithDependencies()` unconditionally awaits the owned candidate's `spawned` promise after the bounded election completes. Consequently, a non-connected election result cannot return until that late promise settles, and a never-settling report hangs indefinitely.
## Issue Context
The election already bounds candidate waiting with `settleBeforeDeadline()` and returns `startup_timeout`. Preserve that result immediately when it is non-connected; attach non-blocking cleanup handlers for a candidate that reports later rather than awaiting it. Extend the regression test to prove completion occurs near the election deadline with a delayed or never-settling report.
## Fix Focus Areas
- packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
- packages/runtime-host/src/__tests__/owned-candidate.test.ts[50-74]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⚖️ Balanced: This changes runtime connection/election behavior and error handling in an asynchronous host lifecycle path, so it carries genuine correctness risk, but the logic is localized enough for a careful single-pass review.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 92e25a18-7c46-472b-b5f2-0caa3a90d1eb

📥 Commits

Reviewing files that changed from the base of the PR and between 21929e3 and f1f11fb.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 2 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated
Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning at the election deadline instead of awaiting a never-settling spawn is the right goal, but the new fire-and-forget cleanup transfers an unsafe kill decision to the timed-out client. A late candidate can become the shared State Root host and accept another client before this cleanup fires; settle(1_000) then kills that live host. The owner of the process must decide whether it is idle, rather than a client that already lost the election. The regression test also needs to cover a candidate that resolves late, not only one that rejects.

AI-assisted review: Codex coordinated two independent reviewer passes and an OpenCode Go DeepSeek V4 Flash high-effort adversarial pass. I verified the exact head f944c30984deeeff827db3871f3cf122c4c249de, the launcher kill behavior, the State Root connection path, and current CI. No local tests were run in this review.

中文审查

不再等待永不结束的 spawn、按 election deadline 返回,这个目标正确;但新的后台清理把危险的 kill 决策交给了已经超时的客户端。迟到 candidate 可能已经成为同一 State Root 的共享 Host,并接入另一个客户端,随后仍会被 settle(1_000) 杀死。进程是否空闲应由 Host 自身判断,而不是由已经输掉选举的客户端决定。回归测试也需要覆盖 candidate 迟到但成功 resolve 的路径,而不只是 reject。

本次为 AI 辅助审查:Codex 协调两轮独立 reviewer 审查及 OpenCode Go DeepSeek V4 Flash high 对抗性复核,并核验了精确 head、launcher kill 行为、State Root 连接路径和当前 CI。本轮未运行本地测试。

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated
Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/runtime-host/src/__tests__/owned-candidate.test.ts (1)

301-317: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consolidate the duplicate connection-result formatters.

ownedConnectFailure and adoptedConnectFailure have identical branches. Use one helper with a union of the two result types. This removes duplicate maintenance when result kinds change.

Disposition: optional.

As per path instructions, “Flag concrete cases where code can be deleted or simplified.”

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e4a9d78f-8cc3-422b-a298-18cdecac0b9d

📥 Commits

Reviewing files that changed from the base of the PR and between f944c30 and 95e0072.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts
…ndow
The promptness claim is Host exit after the first connection closes,
not the connect itself. A two-second election misses spawn plus
handshake under a full suite and only reports `failed`. Widen the
window, keep the concrete reason on the assertion, and map a
deadline throw to startup_timeout.
Fixesapache#3190
Election already bounds candidate startup. Awaiting launch.spawned
after a non-connected result hangs connectOwned when the candidate
never reports. Return that result immediately and settle a late
host in the background.
Fixesapache#3190
A timed-out owned connect no longer settles a late candidate. That
path sent SIGKILL after 1s and could terminate a host another client
had already adopted. Release the process and let idle-grace /
initial-connection policy exit it when unused.
Fixesapache#3190
…tling
The adopt regression only checked that the late pid stayed alive. Wrap
the delivered candidate so a settle() cleanup cannot pass that check
and kill the adopted host after the assertion.
Fixesapache#3190
@1625567290
1625567290force-pushed the fix/host-owned-candidate-connection-flake branch from 95e0072 to 8ca108eCompareAugust 18, 2026 17:05

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head resolves the earlier ownership race correctly. A missed election now returns at its own deadline, preserves the specific failure reason, and releases a late owned candidate to the Runtime Host lifecycle instead of killing a process another client may have adopted. The implementation extends the existing OwnedCandidateAttempt seam and does not create another election authority.

I found no reproducible P0–P3 issue on this head. I also checked two adversarial candidates and rejected them: root ownership prevents the pid-mismatch candidate from simultaneously being the adopted authority, and the test's deliberately unresolved spawn promise occupies the first election until its deadline, so the real candidate cannot make that call enter the connected branch.

The current windows_recovery failure is in the unchanged Local IPC trust fixture (insecure_endpoint_directory), while the affected Runtime Host checks pass. This approval is for the code; the PR is not merge-ready until required CI is green.

AI-assisted review disclosure: Codex performed the final review using two independent reviewer passes and OpenCode Go DeepSeek V4 Flash (high effort) as an adversarial advisory pass. I verified the election state machine, root ownership, tests, exact head 8ca108e3e, and the failing CI log. No local tests were run.

中文评论

当前 head 已正确解决此前的 ownership race:missed election 会在自己的 deadline 返回并保留具体失败原因;迟到的 owned candidate 会交还 Runtime Host lifecycle,而不是杀掉可能已被其他 client 接管的进程。实现直接扩展既有 OwnedCandidateAttempt seam,没有新增 election authority。

当前 head 未发现可复现的 P0–P3。我也复核并排除了两个对抗性候选:root ownership 保证 pid mismatch 的 candidate 不可能同时成为被接管的权威;测试中故意不 resolve 的 spawn promise 会占用第一次 election 直到 deadline,因此真实 candidate 不会让该调用进入 connected 分支。

当前 windows_recovery 失败来自未改动的 Local IPC trust fixture(insecure_endpoint_directory),受影响的 Runtime Host checks 已通过。本次 approve 针对代码;必需 CI 全绿前不能合并。

AI 辅助审查说明:Codex 使用两轮独立 reviewer 审查,并以 OpenCode Go DeepSeek V4 Flash(high effort)进行对抗性辅助审查;我已核对 election 状态机、root ownership、测试、精确 head 8ca108e3e 和失败 CI 日志。本轮未运行本地测试。

@Astro-Han

Copy link
Copy Markdown
Contributor

@1625567290 Hi, Could we state that if there is any ai tools used in the PR?

@Astro-Han
Astro-Han merged commit 803edc1 into apache:mainAug 20, 2026
18 of 19 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(runtime-host): investigate owned candidate connection flake

2 participants

@1625567290@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fix(runtime-host): give owned-candidate connect a CI-safe election window by 1625567290 · Pull Request #3221 · apache/maka · GitHub
Skip to content

fix(runtime-host): give owned-candidate connect a CI-safe election window - #3221

Merged
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake
Aug 20, 2026
Merged

fix(runtime-host): give owned-candidate connect a CI-safe election window#3221
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake

Conversation

@1625567290

Copy link
Copy Markdown
Contributor

Summary

owned Host exits promptly after its first connection closes flakes under the full Runtime Host suite. CI run 32101082229 finished 962/963 tests and failed this one at 2.19s with failed instead of connected. The 2s election window is enough on an idle machine and too short under suite load. The promptness claim is Host exit after the first connection closes, not the connect itself.

  • Widen that test's election window to 8s and keep settle(500) as the prompt-exit check.
  • Put the concrete failed:<reason> on the assertion so a later miss is diagnosable.
  • A late candidate report after the election ends no longer blocks return; the missed election stays startup_timeout.

Fixes#3190

Verification

  • node --test packages/runtime-host/dist/__tests__/owned-candidate.test.js — 7/7

Checklist

  • Tests cover the change and fail without it
  • Focused lint/typecheck and the affected suites pass locally
  • Full workspace lint/format/typecheck

Does this PR entail a change in behavior?

  • Yes — owned connect under a missed election returns without waiting on a late candidate report, and the flaking test uses an 8s election window

@coderabbitai

coderabbitaiBot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b6dc0fd2-ecac-44d4-a27f-7360a9c69d26

📥 Commits

Reviewing files that changed from the base of the PR and between 95e0072 and 8ca108e.

📒 Files selected for processing (1)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/tests/owned-candidate.test.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.


📝 Walkthrough

What problem this solves

This PR reduces CI timing flakes in the Runtime Host owned-candidate connection test.

  • It increases the election window from 2 seconds to 8 seconds.
  • It keeps settle(500) as the prompt-exit check after the first connection closes.
  • It preserves and asserts the concrete failed:<reason> result.
  • It returns startup_timeout when a candidate reports after a missed election.
  • It classifies missing or failed launch.spawned results as host_unresponsive.
  • It releases late spawned hosts instead of terminating them with SIGKILL.

Source of truth

The PR extends the existing owned-candidate connection, election, and launch-cleanup paths. It does not create a parallel connection path or add a public API.

Scope and simplification

The changes are the smallest coherent solution for the reported timing failure and late-startup behavior.

Asynchronous releaseOwnedLaunch cleanup is necessary because a late candidate must not block the startup_timeout result. It also avoids terminating a host that another client may adopt.

The diagnostic formatter, polling helper, and regression tests preserve failure details and cover the timing cases. No deletion or simplification is apparent without weakening diagnostics or regression coverage.

Validation and risks

Focused verification passed: 7/7 tests in owned-candidate.test.js.

The tests cover the longer election window, concrete failure reporting, unresolved candidates, and late candidate adoption. Full-suite validation is not reported, so required checks remain unverified.

Complexity delta

  • Authorities: No new source of truth.
  • States: Clarifies existing host_unresponsive, existing_host, and startup_timeout results.
  • Branches: Adds handling for rejected or missing launch.spawned results and asynchronous release cleanup.
  • Configuration: Increases the test election deadline from 2 seconds to 8 seconds.
  • Public surface: No exported or public entities change.
  • Test maintenance: Adds regression coverage, diagnostics, and a polling helper.

Total maintenance complexity stays justified. The added branches and tests directly cover the reported failure modes.

Review-relevant risks

The connection behavior changes when launch.spawned fails, returns no host, or reports after a missed election. The cleanup behavior also changes from termination to release. These changes can affect Runtime Host error classification, host ownership, and user-visible connection outcomes. Material changes in these areas require independent human review under repository policy.

No security, licensing, release, or governance effect was identified in the current diff.

Required checks remain unverified beyond the reported focused test result. The person performing the merge reviews the final diff. A maintainer makes the final determination.

Walkthrough

The runtime-host client now classifies owned host startup failures and distinguishes launch failures from existing-host outcomes. Tests cover startup timeouts, late candidates, election timing, cleanup, and connection diagnostics.

Changes

Owned runtime-host connection handling

Layer / File(s)Summary
Classify and clean up startup failures
packages/runtime-host/src/client/connect-or-spawn.ts
Owned host startup failures and missing hosts produce host_unresponsive. Connected results produce existing_host only when no launch occurs. Failure cleanup releases owned launches and handles rejected spawn promises.
Cover timeout and late-candidate behavior
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Tests cover unresolved candidates, late candidate adoption, candidate cleanup, pending promises, and the extended election deadline.
Add connection diagnostics and polling
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Test helpers format connection results and poll for delayed candidate startup.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk:⚪ Minimal · up to 8ca10

This localized change widens the owned-candidate election window and makes missed-election failures diagnosable; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check nameStatusExplanationResolution
Out of Scope Changes check⚠️ WarningThe connect-or-spawn.ts changes alter production candidate-handling behavior beyond the linked issue's test-flake investigation scope.Separate the production behavior change into a linked issue or explain its direct necessity for issue #3190.
Ai Use Disclosure⚠️ WarningThe PR description selects neither disclosure statement, and all five PR commits lack a valid Generated-by trailer.Add exactly one required declaration. If generative tooling made a substantive contribution, name the tool and scope; if it authored material content, add a trailer that survives squash/amend. See “Human ownership and AI attribution” in...
✅ Passed checks (3 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly describes the main change: extending the owned-candidate election window for CI reliability.
Description check✅ PassedThe description covers the problem, fix, issue reference, verification, tests, and behavior change, but omits the required AI use section.
Linked Issues check✅ PassedThe PR preserves the failure reason and increases the election deadline, addressing the timing-flake objectives in issue #3190.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Stabilize owned-candidate election handling under CI load

🐞 Bug fix🧪 Tests🕐 10-20 Minutes

Grey Divider

AI Description

• Preserve startup timeouts when candidate spawn failures arrive after the election deadline.
• Extend the disconnect test election window to eight seconds for loaded CI.
• Add regression coverage and concrete diagnostics for owned connection failures.
Diagram

sequenceDiagram
actor Caller
participant Owned as Owned Connect
participant Election as Host Election
participant Candidate as Owned Candidate
Caller->>Owned: request connection
Owned->>Election: connect or spawn
Election->>Candidate: launch candidate
Note over Election: deadline expires
Election-->>Owned: startup_timeout
Candidate--x Owned: late spawn failure
Owned-->>Caller: preserve timeout
Loading
High-Level Assessment

The focused wrapper change is appropriate: it treats a rejected spawn report as an unavailable owned host while preserving an already-determined election failure. Returning immediately without observing the candidate promise could reduce latency further, but risks losing ownership and cleanup of a process that eventually spawns; introducing launch cancellation would be disproportionate for this fix.

Files changed (2) +46 / -4

Bug fix (1) +6 / -2
connect-or-spawn.tsPreserve owned election failures after spawn rejection+6/-2

Preserve owned election failures after spawn rejection

• Handles rejected owned-candidate spawn promises without falling through to the generic exception path. Existing election failures are returned unchanged, while connected results without a usable launched host are classified as 'host_unresponsive' or 'existing_host'.

packages/runtime-host/src/client/connect-or-spawn.ts

Tests (1) +40 / -2
owned-candidate.test.tsCover late candidate reports and stabilize the disconnect lifecycle test+40/-2

Cover late candidate reports and stabilize the disconnect lifecycle test

• Adds a regression test proving that a late rejected candidate report does not replace 'startup_timeout'. Expands the disconnect test election window from two to eight seconds and reports concrete failure reasons while retaining the 500ms host-exit assertion.

packages/runtime-host/src/tests/owned-candidate.test.ts

@qodo-code-review

qodo-code-reviewBot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0)📘 Rule violations (0)📜 Skill insights (0)

Grey Divider


Action required

1. Late report still blocks return✓ Resolved🐞 Bug≡ Correctness
Description
After the election produces startup_timeout, connectOwnedRuntimeHostWithDependencies() still
awaits launch.spawned, so a late or never-settling candidate report delays or permanently blocks
the result beyond the election deadline. This violates the stated behavior that a missed election
returns without waiting for the late report.
Code

packages/runtime-host/src/client/connect-or-spawn.ts[130]

+ const host = await launch?.spawned.catch(() => undefined);
Relevance

●●● Strong

Accepted precedent supports fixing blocking async cleanup; this is a deterministic missed-deadline
correctness bug matching PR intent.

PR-#2674

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The election waits for launch.spawned only through the deadline and then returns
startup_timeout, but the outer owned wrapper immediately awaits the original unbounded promise
again. The added test delays rejection for 20 ms after a 1 ms deadline yet checks only the returned
reason, so it passes even though the function waits for that late report; replacing the test promise
with a never-settling promise reproducibly prevents completion.

packages/runtime-host/src/client/connect-or-spawn.ts[249-260]
packages/runtime-host/src/client/connect-or-spawn.ts[293-297]
packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
packages/runtime-host/src/tests/owned-candidate.test.ts[50-74]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`connectOwnedRuntimeHostWithDependencies()` unconditionally awaits the owned candidate's `spawned` promise after the bounded election completes. Consequently, a non-connected election result cannot return until that late promise settles, and a never-settling report hangs indefinitely.
## Issue Context
The election already bounds candidate waiting with `settleBeforeDeadline()` and returns `startup_timeout`. Preserve that result immediately when it is non-connected; attach non-blocking cleanup handlers for a candidate that reports later rather than awaiting it. Extend the regression test to prove completion occurs near the election deadline with a delayed or never-settling report.
## Fix Focus Areas
- packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
- packages/runtime-host/src/__tests__/owned-candidate.test.ts[50-74]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⚖️ Balanced: This changes runtime connection/election behavior and error handling in an asynchronous host lifecycle path, so it carries genuine correctness risk, but the logic is localized enough for a careful single-pass review.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 92e25a18-7c46-472b-b5f2-0caa3a90d1eb

📥 Commits

Reviewing files that changed from the base of the PR and between 21929e3 and f1f11fb.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 2 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated
Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning at the election deadline instead of awaiting a never-settling spawn is the right goal, but the new fire-and-forget cleanup transfers an unsafe kill decision to the timed-out client. A late candidate can become the shared State Root host and accept another client before this cleanup fires; settle(1_000) then kills that live host. The owner of the process must decide whether it is idle, rather than a client that already lost the election. The regression test also needs to cover a candidate that resolves late, not only one that rejects.

AI-assisted review: Codex coordinated two independent reviewer passes and an OpenCode Go DeepSeek V4 Flash high-effort adversarial pass. I verified the exact head f944c30984deeeff827db3871f3cf122c4c249de, the launcher kill behavior, the State Root connection path, and current CI. No local tests were run in this review.

中文审查

不再等待永不结束的 spawn、按 election deadline 返回,这个目标正确;但新的后台清理把危险的 kill 决策交给了已经超时的客户端。迟到 candidate 可能已经成为同一 State Root 的共享 Host,并接入另一个客户端,随后仍会被 settle(1_000) 杀死。进程是否空闲应由 Host 自身判断,而不是由已经输掉选举的客户端决定。回归测试也需要覆盖 candidate 迟到但成功 resolve 的路径,而不只是 reject。

本次为 AI 辅助审查:Codex 协调两轮独立 reviewer 审查及 OpenCode Go DeepSeek V4 Flash high 对抗性复核,并核验了精确 head、launcher kill 行为、State Root 连接路径和当前 CI。本轮未运行本地测试。

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated
Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/runtime-host/src/__tests__/owned-candidate.test.ts (1)

301-317: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consolidate the duplicate connection-result formatters.

ownedConnectFailure and adoptedConnectFailure have identical branches. Use one helper with a union of the two result types. This removes duplicate maintenance when result kinds change.

Disposition: optional.

As per path instructions, “Flag concrete cases where code can be deleted or simplified.”

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e4a9d78f-8cc3-422b-a298-18cdecac0b9d

📥 Commits

Reviewing files that changed from the base of the PR and between f944c30 and 95e0072.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts
…ndow
The promptness claim is Host exit after the first connection closes,
not the connect itself. A two-second election misses spawn plus
handshake under a full suite and only reports `failed`. Widen the
window, keep the concrete reason on the assertion, and map a
deadline throw to startup_timeout.
Fixesapache#3190
Election already bounds candidate startup. Awaiting launch.spawned
after a non-connected result hangs connectOwned when the candidate
never reports. Return that result immediately and settle a late
host in the background.
Fixesapache#3190
A timed-out owned connect no longer settles a late candidate. That
path sent SIGKILL after 1s and could terminate a host another client
had already adopted. Release the process and let idle-grace /
initial-connection policy exit it when unused.
Fixesapache#3190
…tling
The adopt regression only checked that the late pid stayed alive. Wrap
the delivered candidate so a settle() cleanup cannot pass that check
and kill the adopted host after the assertion.
Fixesapache#3190
@1625567290
1625567290force-pushed the fix/host-owned-candidate-connection-flake branch from 95e0072 to 8ca108eCompareAugust 18, 2026 17:05

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head resolves the earlier ownership race correctly. A missed election now returns at its own deadline, preserves the specific failure reason, and releases a late owned candidate to the Runtime Host lifecycle instead of killing a process another client may have adopted. The implementation extends the existing OwnedCandidateAttempt seam and does not create another election authority.

I found no reproducible P0–P3 issue on this head. I also checked two adversarial candidates and rejected them: root ownership prevents the pid-mismatch candidate from simultaneously being the adopted authority, and the test's deliberately unresolved spawn promise occupies the first election until its deadline, so the real candidate cannot make that call enter the connected branch.

The current windows_recovery failure is in the unchanged Local IPC trust fixture (insecure_endpoint_directory), while the affected Runtime Host checks pass. This approval is for the code; the PR is not merge-ready until required CI is green.

AI-assisted review disclosure: Codex performed the final review using two independent reviewer passes and OpenCode Go DeepSeek V4 Flash (high effort) as an adversarial advisory pass. I verified the election state machine, root ownership, tests, exact head 8ca108e3e, and the failing CI log. No local tests were run.

中文评论

当前 head 已正确解决此前的 ownership race:missed election 会在自己的 deadline 返回并保留具体失败原因;迟到的 owned candidate 会交还 Runtime Host lifecycle,而不是杀掉可能已被其他 client 接管的进程。实现直接扩展既有 OwnedCandidateAttempt seam,没有新增 election authority。

当前 head 未发现可复现的 P0–P3。我也复核并排除了两个对抗性候选:root ownership 保证 pid mismatch 的 candidate 不可能同时成为被接管的权威;测试中故意不 resolve 的 spawn promise 会占用第一次 election 直到 deadline,因此真实 candidate 不会让该调用进入 connected 分支。

当前 windows_recovery 失败来自未改动的 Local IPC trust fixture(insecure_endpoint_directory),受影响的 Runtime Host checks 已通过。本次 approve 针对代码;必需 CI 全绿前不能合并。

AI 辅助审查说明:Codex 使用两轮独立 reviewer 审查,并以 OpenCode Go DeepSeek V4 Flash(high effort)进行对抗性辅助审查;我已核对 election 状态机、root ownership、测试、精确 head 8ca108e3e 和失败 CI 日志。本轮未运行本地测试。

@Astro-Han

Copy link
Copy Markdown
Contributor

@1625567290 Hi, Could we state that if there is any ai tools used in the PR?

@Astro-Han
Astro-Han merged commit 803edc1 into apache:mainAug 20, 2026
18 of 19 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(runtime-host): investigate owned candidate connection flake

2 participants

@1625567290@Astro-Han
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); fix(runtime-host): give owned-candidate connect a CI-safe election window by 1625567290 · Pull Request #3221 · apache/maka · GitHub
Skip to content

fix(runtime-host): give owned-candidate connect a CI-safe election window - #3221

Merged
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake
Aug 20, 2026
Merged

fix(runtime-host): give owned-candidate connect a CI-safe election window#3221
Astro-Han merged 5 commits into
apache:mainfrom
1625567290:fix/host-owned-candidate-connection-flake

Conversation

@1625567290

Copy link
Copy Markdown
Contributor

Summary

owned Host exits promptly after its first connection closes flakes under the full Runtime Host suite. CI run 32101082229 finished 962/963 tests and failed this one at 2.19s with failed instead of connected. The 2s election window is enough on an idle machine and too short under suite load. The promptness claim is Host exit after the first connection closes, not the connect itself.

  • Widen that test's election window to 8s and keep settle(500) as the prompt-exit check.
  • Put the concrete failed:<reason> on the assertion so a later miss is diagnosable.
  • A late candidate report after the election ends no longer blocks return; the missed election stays startup_timeout.

Fixes#3190

Verification

  • node --test packages/runtime-host/dist/__tests__/owned-candidate.test.js — 7/7

Checklist

  • Tests cover the change and fail without it
  • Focused lint/typecheck and the affected suites pass locally
  • Full workspace lint/format/typecheck

Does this PR entail a change in behavior?

  • Yes — owned connect under a missed election returns without waiting on a late candidate report, and the flaking test uses an 8s election window

@coderabbitai

coderabbitaiBot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b6dc0fd2-ecac-44d4-a27f-7360a9c69d26

📥 Commits

Reviewing files that changed from the base of the PR and between 95e0072 and 8ca108e.

📒 Files selected for processing (1)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/tests/owned-candidate.test.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.


📝 Walkthrough

What problem this solves

This PR reduces CI timing flakes in the Runtime Host owned-candidate connection test.

  • It increases the election window from 2 seconds to 8 seconds.
  • It keeps settle(500) as the prompt-exit check after the first connection closes.
  • It preserves and asserts the concrete failed:<reason> result.
  • It returns startup_timeout when a candidate reports after a missed election.
  • It classifies missing or failed launch.spawned results as host_unresponsive.
  • It releases late spawned hosts instead of terminating them with SIGKILL.

Source of truth

The PR extends the existing owned-candidate connection, election, and launch-cleanup paths. It does not create a parallel connection path or add a public API.

Scope and simplification

The changes are the smallest coherent solution for the reported timing failure and late-startup behavior.

Asynchronous releaseOwnedLaunch cleanup is necessary because a late candidate must not block the startup_timeout result. It also avoids terminating a host that another client may adopt.

The diagnostic formatter, polling helper, and regression tests preserve failure details and cover the timing cases. No deletion or simplification is apparent without weakening diagnostics or regression coverage.

Validation and risks

Focused verification passed: 7/7 tests in owned-candidate.test.js.

The tests cover the longer election window, concrete failure reporting, unresolved candidates, and late candidate adoption. Full-suite validation is not reported, so required checks remain unverified.

Complexity delta

  • Authorities: No new source of truth.
  • States: Clarifies existing host_unresponsive, existing_host, and startup_timeout results.
  • Branches: Adds handling for rejected or missing launch.spawned results and asynchronous release cleanup.
  • Configuration: Increases the test election deadline from 2 seconds to 8 seconds.
  • Public surface: No exported or public entities change.
  • Test maintenance: Adds regression coverage, diagnostics, and a polling helper.

Total maintenance complexity stays justified. The added branches and tests directly cover the reported failure modes.

Review-relevant risks

The connection behavior changes when launch.spawned fails, returns no host, or reports after a missed election. The cleanup behavior also changes from termination to release. These changes can affect Runtime Host error classification, host ownership, and user-visible connection outcomes. Material changes in these areas require independent human review under repository policy.

No security, licensing, release, or governance effect was identified in the current diff.

Required checks remain unverified beyond the reported focused test result. The person performing the merge reviews the final diff. A maintainer makes the final determination.

Walkthrough

The runtime-host client now classifies owned host startup failures and distinguishes launch failures from existing-host outcomes. Tests cover startup timeouts, late candidates, election timing, cleanup, and connection diagnostics.

Changes

Owned runtime-host connection handling

Layer / File(s)Summary
Classify and clean up startup failures
packages/runtime-host/src/client/connect-or-spawn.ts
Owned host startup failures and missing hosts produce host_unresponsive. Connected results produce existing_host only when no launch occurs. Failure cleanup releases owned launches and handles rejected spawn promises.
Cover timeout and late-candidate behavior
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Tests cover unresolved candidates, late candidate adoption, candidate cleanup, pending promises, and the extended election deadline.
Add connection diagnostics and polling
packages/runtime-host/src/__tests__/owned-candidate.test.ts
Test helpers format connection results and poll for delayed candidate startup.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk:⚪ Minimal · up to 8ca10

This localized change widens the owned-candidate election window and makes missed-election failures diagnosable; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check nameStatusExplanationResolution
Out of Scope Changes check⚠️ WarningThe connect-or-spawn.ts changes alter production candidate-handling behavior beyond the linked issue's test-flake investigation scope.Separate the production behavior change into a linked issue or explain its direct necessity for issue #3190.
Ai Use Disclosure⚠️ WarningThe PR description selects neither disclosure statement, and all five PR commits lack a valid Generated-by trailer.Add exactly one required declaration. If generative tooling made a substantive contribution, name the tool and scope; if it authored material content, add a trailer that survives squash/amend. See “Human ownership and AI attribution” in...
✅ Passed checks (3 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly describes the main change: extending the owned-candidate election window for CI reliability.
Description check✅ PassedThe description covers the problem, fix, issue reference, verification, tests, and behavior change, but omits the required AI use section.
Linked Issues check✅ PassedThe PR preserves the failure reason and increases the election deadline, addressing the timing-flake objectives in issue #3190.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Stabilize owned-candidate election handling under CI load

🐞 Bug fix🧪 Tests🕐 10-20 Minutes

Grey Divider

AI Description

• Preserve startup timeouts when candidate spawn failures arrive after the election deadline.
• Extend the disconnect test election window to eight seconds for loaded CI.
• Add regression coverage and concrete diagnostics for owned connection failures.
Diagram

sequenceDiagram
actor Caller
participant Owned as Owned Connect
participant Election as Host Election
participant Candidate as Owned Candidate
Caller->>Owned: request connection
Owned->>Election: connect or spawn
Election->>Candidate: launch candidate
Note over Election: deadline expires
Election-->>Owned: startup_timeout
Candidate--x Owned: late spawn failure
Owned-->>Caller: preserve timeout
Loading
High-Level Assessment

The focused wrapper change is appropriate: it treats a rejected spawn report as an unavailable owned host while preserving an already-determined election failure. Returning immediately without observing the candidate promise could reduce latency further, but risks losing ownership and cleanup of a process that eventually spawns; introducing launch cancellation would be disproportionate for this fix.

Files changed (2) +46 / -4

Bug fix (1) +6 / -2
connect-or-spawn.tsPreserve owned election failures after spawn rejection+6/-2

Preserve owned election failures after spawn rejection

• Handles rejected owned-candidate spawn promises without falling through to the generic exception path. Existing election failures are returned unchanged, while connected results without a usable launched host are classified as 'host_unresponsive' or 'existing_host'.

packages/runtime-host/src/client/connect-or-spawn.ts

Tests (1) +40 / -2
owned-candidate.test.tsCover late candidate reports and stabilize the disconnect lifecycle test+40/-2

Cover late candidate reports and stabilize the disconnect lifecycle test

• Adds a regression test proving that a late rejected candidate report does not replace 'startup_timeout'. Expands the disconnect test election window from two to eight seconds and reports concrete failure reasons while retaining the 500ms host-exit assertion.

packages/runtime-host/src/tests/owned-candidate.test.ts

@qodo-code-review

qodo-code-reviewBot commented Aug 18, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0)📘 Rule violations (0)📜 Skill insights (0)

Grey Divider


Action required

1. Late report still blocks return✓ Resolved🐞 Bug≡ Correctness
Description
After the election produces startup_timeout, connectOwnedRuntimeHostWithDependencies() still
awaits launch.spawned, so a late or never-settling candidate report delays or permanently blocks
the result beyond the election deadline. This violates the stated behavior that a missed election
returns without waiting for the late report.
Code

packages/runtime-host/src/client/connect-or-spawn.ts[130]

+ const host = await launch?.spawned.catch(() => undefined);
Relevance

●●● Strong

Accepted precedent supports fixing blocking async cleanup; this is a deterministic missed-deadline
correctness bug matching PR intent.

PR-#2674

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The election waits for launch.spawned only through the deadline and then returns
startup_timeout, but the outer owned wrapper immediately awaits the original unbounded promise
again. The added test delays rejection for 20 ms after a 1 ms deadline yet checks only the returned
reason, so it passes even though the function waits for that late report; replacing the test promise
with a never-settling promise reproducibly prevents completion.

packages/runtime-host/src/client/connect-or-spawn.ts[249-260]
packages/runtime-host/src/client/connect-or-spawn.ts[293-297]
packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
packages/runtime-host/src/tests/owned-candidate.test.ts[50-74]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
`connectOwnedRuntimeHostWithDependencies()` unconditionally awaits the owned candidate's `spawned` promise after the bounded election completes. Consequently, a non-connected election result cannot return until that late promise settles, and a never-settling report hangs indefinitely.
## Issue Context
The election already bounds candidate waiting with `settleBeforeDeadline()` and returns `startup_timeout`. Preserve that result immediately when it is non-connected; attach non-blocking cleanup handlers for a candidate that reports later rather than awaiting it. Extend the regression test to prove completion occurs near the election deadline with a delayed or never-settling report.
## Fix Focus Areas
- packages/runtime-host/src/client/connect-or-spawn.ts[130-138]
- packages/runtime-host/src/__tests__/owned-candidate.test.ts[50-74]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⚖️ Balanced: This changes runtime connection/election behavior and error handling in an asynchronous host lifecycle path, so it carries genuine correctness risk, but the logic is localized enough for a careful single-pass review.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 92e25a18-7c46-472b-b5f2-0caa3a90d1eb

📥 Commits

Reviewing files that changed from the base of the PR and between 21929e3 and f1f11fb.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 2 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated
Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning at the election deadline instead of awaiting a never-settling spawn is the right goal, but the new fire-and-forget cleanup transfers an unsafe kill decision to the timed-out client. A late candidate can become the shared State Root host and accept another client before this cleanup fires; settle(1_000) then kills that live host. The owner of the process must decide whether it is idle, rather than a client that already lost the election. The regression test also needs to cover a candidate that resolves late, not only one that rejects.

AI-assisted review: Codex coordinated two independent reviewer passes and an OpenCode Go DeepSeek V4 Flash high-effort adversarial pass. I verified the exact head f944c30984deeeff827db3871f3cf122c4c249de, the launcher kill behavior, the State Root connection path, and current CI. No local tests were run in this review.

中文审查

不再等待永不结束的 spawn、按 election deadline 返回,这个目标正确;但新的后台清理把危险的 kill 决策交给了已经超时的客户端。迟到 candidate 可能已经成为同一 State Root 的共享 Host,并接入另一个客户端,随后仍会被 settle(1_000) 杀死。进程是否空闲应由 Host 自身判断,而不是由已经输掉选举的客户端决定。回归测试也需要覆盖 candidate 迟到但成功 resolve 的路径,而不只是 reject。

本次为 AI 辅助审查:Codex 协调两轮独立 reviewer 审查及 OpenCode Go DeepSeek V4 Flash high 对抗性复核,并核验了精确 head、launcher kill 行为、State Root 连接路径和当前 CI。本轮未运行本地测试。

Comment threadpackages/runtime-host/src/client/connect-or-spawn.ts Outdated
Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts Outdated

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/runtime-host/src/__tests__/owned-candidate.test.ts (1)

301-317: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consolidate the duplicate connection-result formatters.

ownedConnectFailure and adoptedConnectFailure have identical branches. Use one helper with a union of the two result types. This removes duplicate maintenance when result kinds change.

Disposition: optional.

As per path instructions, “Flag concrete cases where code can be deleted or simplified.”

Source: Path instructions


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e4a9d78f-8cc3-422b-a298-18cdecac0b9d

📥 Commits

Reviewing files that changed from the base of the PR and between f944c30 and 95e0072.

📒 Files selected for processing (2)
  • packages/runtime-host/src/__tests__/owned-candidate.test.ts
  • packages/runtime-host/src/client/connect-or-spawn.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/runtime-host/src/client/connect-or-spawn.ts

Included review availability: Your plan includes up to 3 reviews per rolling hour; 0 remain after this review.

Comment threadpackages/runtime-host/src/__tests__/owned-candidate.test.ts
…ndow
The promptness claim is Host exit after the first connection closes,
not the connect itself. A two-second election misses spawn plus
handshake under a full suite and only reports `failed`. Widen the
window, keep the concrete reason on the assertion, and map a
deadline throw to startup_timeout.
Fixesapache#3190
Election already bounds candidate startup. Awaiting launch.spawned
after a non-connected result hangs connectOwned when the candidate
never reports. Return that result immediately and settle a late
host in the background.
Fixesapache#3190
A timed-out owned connect no longer settles a late candidate. That
path sent SIGKILL after 1s and could terminate a host another client
had already adopted. Release the process and let idle-grace /
initial-connection policy exit it when unused.
Fixesapache#3190
…tling
The adopt regression only checked that the late pid stayed alive. Wrap
the delivered candidate so a settle() cleanup cannot pass that check
and kill the adopted host after the assertion.
Fixesapache#3190
@1625567290
1625567290force-pushed the fix/host-owned-candidate-connection-flake branch from 95e0072 to 8ca108eCompareAugust 18, 2026 17:05

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head resolves the earlier ownership race correctly. A missed election now returns at its own deadline, preserves the specific failure reason, and releases a late owned candidate to the Runtime Host lifecycle instead of killing a process another client may have adopted. The implementation extends the existing OwnedCandidateAttempt seam and does not create another election authority.

I found no reproducible P0–P3 issue on this head. I also checked two adversarial candidates and rejected them: root ownership prevents the pid-mismatch candidate from simultaneously being the adopted authority, and the test's deliberately unresolved spawn promise occupies the first election until its deadline, so the real candidate cannot make that call enter the connected branch.

The current windows_recovery failure is in the unchanged Local IPC trust fixture (insecure_endpoint_directory), while the affected Runtime Host checks pass. This approval is for the code; the PR is not merge-ready until required CI is green.

AI-assisted review disclosure: Codex performed the final review using two independent reviewer passes and OpenCode Go DeepSeek V4 Flash (high effort) as an adversarial advisory pass. I verified the election state machine, root ownership, tests, exact head 8ca108e3e, and the failing CI log. No local tests were run.

中文评论

当前 head 已正确解决此前的 ownership race:missed election 会在自己的 deadline 返回并保留具体失败原因;迟到的 owned candidate 会交还 Runtime Host lifecycle,而不是杀掉可能已被其他 client 接管的进程。实现直接扩展既有 OwnedCandidateAttempt seam,没有新增 election authority。

当前 head 未发现可复现的 P0–P3。我也复核并排除了两个对抗性候选:root ownership 保证 pid mismatch 的 candidate 不可能同时成为被接管的权威;测试中故意不 resolve 的 spawn promise 会占用第一次 election 直到 deadline,因此真实 candidate 不会让该调用进入 connected 分支。

当前 windows_recovery 失败来自未改动的 Local IPC trust fixture(insecure_endpoint_directory),受影响的 Runtime Host checks 已通过。本次 approve 针对代码;必需 CI 全绿前不能合并。

AI 辅助审查说明:Codex 使用两轮独立 reviewer 审查,并以 OpenCode Go DeepSeek V4 Flash(high effort)进行对抗性辅助审查;我已核对 election 状态机、root ownership、测试、精确 head 8ca108e3e 和失败 CI 日志。本轮未运行本地测试。

@Astro-Han

Copy link
Copy Markdown
Contributor

@1625567290 Hi, Could we state that if there is any ai tools used in the PR?

@Astro-Han
Astro-Han merged commit 803edc1 into apache:mainAug 20, 2026
18 of 19 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(runtime-host): investigate owned candidate connection flake

2 participants

@1625567290@Astro-Han