Skip to content

feat(remote-comms): Add kernel incarnation detection protocol - #788

Merged
sirtimid merged 28 commits into
mainfrom
sirtimid/issue-689-incarnation-detection
Feb 3, 2026
Merged

feat(remote-comms): Add kernel incarnation detection protocol#788
sirtimid merged 28 commits into
mainfrom
sirtimid/issue-689-incarnation-detection

Conversation

@sirtimid

@sirtimidsirtimid commented Jan 29, 2026

Copy link
Copy Markdown
Member

Depends on:#800

Closes#689

Summary

Integrates the handshake protocol for kernel incarnation detection (foundation established in #800). When kernels connect, they exchange incarnation IDs to detect when a peer has lost its state (e.g., storage cleared) but retained the same peer ID.

Key Changes

Handshake Integration (transport.ts)

  • Integrates handshake module into connection flow
  • Outbound connections: perform handshake after channel opens
  • Inbound connections: handle handshake from peer
  • Reconnection: re-perform handshake on reconnect

Incarnation Change Detection

  • setRemoteIncarnation() returns true if incarnation changed from a known previous value
  • On change detection, onRemoteGiveUp callback triggers promise rejection for pending work

Promise Rejection Flow

  • onRemoteGiveUp(peerId)RemoteManager.#handleRemoteGiveUp()
  • Rejects all kernel promises where the remote is the decider
  • Calls RemoteHandle.rejectPendingRedemptions() for pending URL redemptions

Wire-up Changes

  • Kernel.ts: Initialize incarnation ID and pass to RemoteManager
  • RemoteManager.ts: Pass incarnation ID to transport layer
  • PlatformServicesClient/Server: Pass incarnation ID through RPC
  • reconnection-lifecycle.ts: Perform handshake during reconnection

Files Changed

FileChange
transport.tsIntegration of handshake into connection flow
reconnection-lifecycle.tsHandshake during reconnection
Kernel.tsIncarnation ID initialization and reset handling
RemoteManager.tsPass incarnation ID to transport
remote-comms.tsWire incarnation ID through call chain
PlatformServicesClient.tsPass incarnation ID through RPC
PlatformServicesServer.tsAccept incarnation ID parameter

Testing

  • Unit tests for transport integration
  • Unit tests for reconnection lifecycle with handshake
  • E2E test: detects incarnation change when peer restarts with fresh state

🤖 Generated with Claude Code


Note

High Risk
Introduces a new connection-level handshake and reconnection behavior in transport.ts, altering how channels are accepted/registered and when remotes are failed, which can affect network compatibility and message delivery under failure/restart scenarios.

Overview
Adds an incarnation-ID handshake protocol to remote comms so peers can detect when a connection is re-established to a kernel that lost state (same peerId, new incarnationId) and proactively fail outstanding work.

Plumbs optional incarnationId through the full initialization stack (KernelStoreRemoteManagerremote-comms → platform services RPC → initTransport) and updates kernel reset behavior to preserve/clear incarnationId appropriately.

Updates transport lifecycle to handshake before channel registration for outbound, inbound, and reconnection flows; on incarnation change it triggers onRemoteGiveUp to reject pending promises. Expands unit coverage and adds an e2e test that restarts a peer with fresh storage and asserts the caller sees Remote connection lost.

Written by Cursor Bugbot for commit f150b82. This will update automatically on new commits. Configure here.

@github-actions

github-actionsBot commented Jan 29, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

StatusCategoryPercentageCovered / Total
🔵Lines87.64%
⬇️ -0.09%
6030 / 6880
🔵Statements87.51%
⬇️ -0.09%
6127 / 7001
🔵Functions86.07%
⬇️ -0.01%
1551 / 1802
🔵Branches83.96%
⬇️ -0.01%
2189 / 2607
File Coverage
FileStmtsBranchesFunctionsLinesUncovered Lines
Changed Files
packages/kernel-browser-runtime/src/PlatformServicesClient.ts96.66%
🟰 ±0%
81.48%
⬇️ -2.52%
89.47%
🟰 ±0%
96.66%
🟰 ±0%
106, 128
packages/kernel-browser-runtime/src/PlatformServicesServer.ts95.23%
🟰 ±0%
88.88%
🟰 ±0%
80.95%
🟰 ±0%
95.23%
🟰 ±0%
140, 163, 194, 404
packages/nodejs/src/kernel/PlatformServices.ts93.97%
🟰 ±0%
90.47%
🟰 ±0%
88.23%
🟰 ±0%
93.97%
🟰 ±0%
143-146, 184, 211-216
packages/ocap-kernel/src/Kernel.ts93.5%
⬆️ +2.37%
85%
⬆️ +1.67%
92.3%
🟰 ±0%
93.5%
⬆️ +2.37%
116, 225-228, 410, 480, 524
packages/ocap-kernel/src/remotes/kernel/RemoteManager.ts98.3%
⬆️ +0.03%
100%
🟰 ±0%
100%
🟰 ±0%
98.3%
⬆️ +0.03%
133
packages/ocap-kernel/src/remotes/kernel/remote-comms.ts100%
🟰 ±0%
100%
🟰 ±0%
100%
🟰 ±0%
100%
🟰 ±0%
packages/ocap-kernel/src/remotes/platform/reconnection-lifecycle.ts86.07%
⬇️ -2.50%
88.57%
⬆️ +0.70%
80%
🟰 ±0%
86.07%
⬇️ -2.50%
110-114, 134-135, 213-218, 256-257
packages/ocap-kernel/src/remotes/platform/transport.ts83.87%
⬇️ -2.72%
81.92%
⬆️ +0.39%
75.86%
⬆️ +0.86%
83.87%
⬇️ -2.72%
110, 149-150, 155-158, 199-208, 241, 275-293, 317, 401, 463, 487-492, 495-496, 537, 567, 586-588, 597
packages/ocap-kernel/src/rpc/platform-services/initializeRemoteComms.ts100%
🟰 ±0%
100%
🟰 ±0%
100%
🟰 ±0%
100%
🟰 ±0%
Generated in workflow #3548 for commit f150b82 by the Vitest Coverage Report Action

@sirtimid
sirtimid marked this pull request as ready for review January 29, 2026 13:58
@sirtimid
sirtimid requested a review from a team as a code ownerJanuary 29, 2026 13:58
Comment threadpackages/ocap-kernel/src/Kernel.ts Outdated
Comment threadpackages/ocap-kernel/src/remotes/platform/handshake.ts Outdated
@sirtimidsirtimid mentioned this pull request Jan 29, 2026
2 tasks
Comment threadpackages/ocap-kernel/src/types.ts
@sirtimid
sirtimidforce-pushed the sirtimid/issue-689-incarnation-detection branch from 62c07ec to 606f03fCompareJanuary 29, 2026 18:52
Comment threadpackages/ocap-kernel/src/remotes/platform/handshake.ts Outdated
Comment threadpackages/ocap-kernel/src/remotes/platform/handshake.ts
Comment threadpackages/ocap-kernel/src/remotes/platform/peer-state-manager.ts Outdated
@sirtimid
sirtimidforce-pushed the sirtimid/issue-689-incarnation-detection branch 2 times, most recently from 34dc2ca to 44b8e91CompareFebruary 2, 2026 15:21
Comment threadpackages/ocap-kernel/src/remotes/platform/reconnection-lifecycle.test.ts Outdated
@sirtimid
sirtimidforce-pushed the sirtimid/issue-689-incarnation-detection branch from 44b8e91 to 97a32ecCompareFebruary 2, 2026 15:33
@sirtimid
sirtimid changed the base branch from main to sirtimid/issue-689-foundationFebruary 2, 2026 15:35
Comment threadpackages/ocap-kernel/src/remotes/platform/transport.ts
Comment threadpackages/nodejs/test/e2e/remote-comms.test.ts Outdated
@sirtimid
sirtimidforce-pushed the sirtimid/issue-689-incarnation-detection branch from 1bb35d7 to d516d6cCompareFebruary 2, 2026 18:57
Comment threadpackages/nodejs/test/e2e/remote-comms.test.ts
github-merge-queueBot pushed a commit that referenced this pull request Feb 3, 2026
…odule (#800)
## Summary
- Add `getOrCreateIncarnationId()` to store for unique kernel instance
tracking
- Add `setRemoteIncarnation()` to peer state manager for tracking remote
incarnations
- Add handshake module with inbound/outbound handshake functions
- Extend `initializeRemoteComms` RPC schema to accept `incarnationId`
parameter
## Details
This is **part 1 of a 2-PR series** to add incarnation detection support
(issue #689).
**No behavior change** - This PR adds the capability without using it
yet. The `incarnationId` parameter is accepted by the RPC schema but not
yet wired through to the transport layer.
Part 2 (PR #788) will wire up the handshake protocol and integrate
incarnation detection into the reconnection lifecycle.
## Test plan
- [x] Unit tests for `getOrCreateIncarnationId()` in store
- [x] Unit tests for `setRemoteIncarnation()` in peer state manager
- [x] Unit tests for handshake module (inbound/outbound flows)
- [x] Unit tests for `initializeRemoteComms` schema accepting
`incarnationId`
- [x] All existing tests pass
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Medium risk because it introduces new network handshake/timeout
behavior and extends persisted state (kernel/peer incarnation IDs), even
though the new handshake isn’t yet integrated into the live comms flow.
> > **Overview**
> Adds incarnation-ID infrastructure to support detecting peer restarts.
The kernel store now persists a UUID `incarnationId` via
`provideIncarnationId()`, and `PeerStateManager` tracks
`remoteIncarnationId` per peer with `setRemoteIncarnation()` returning
whether it changed.
> > Introduces a new `handshake` module that exchanges
`handshake`/`handshakeAck` messages over a `Channel`, with read/write
timeouts and validation via `isHandshakeMessage()`. Extends
`initializeRemoteComms` RPC params schema to accept an optional
`incarnationId` (currently parsed/validated but intentionally not
forwarded to the hook), with accompanying unit tests.
> > <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
27fde8b. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Base automatically changed from sirtimid/issue-689-foundation to mainFebruary 3, 2026 10:51
@sirtimid
sirtimidforce-pushed the sirtimid/issue-689-incarnation-detection branch from 1c79a77 to 848fe35CompareFebruary 3, 2026 10:55
Comment threadpackages/ocap-kernel/src/remotes/platform/transport.ts

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Comment threadpackages/ocap-kernel/src/remotes/platform/handshake.test.ts Outdated
maxRetryAttempts,
);

// Create handshake dependencies (only if incarnation ID is configured)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When would we not configure an incarnation id?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's optional for unit tests that don't need handshake behavior, in production the kernel always provides an incarnation ID.

sirtimidand others added 2 commits February 3, 2026 21:49
Add Handshake and HandshakeAck message types to RemoteMessageBase
for kernel incarnation detection protocol. These types will be used
to exchange incarnation IDs during connection establishment.
Part of #689
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Generate unique incarnation ID (UUID) in Kernel constructor
- Store incarnation ID in memory only (not persisted)
- Pass incarnation ID through RemoteManager to remote-comms
- Update PlatformServices type and RPC handlers to accept incarnationId
The incarnation ID is used to detect when a peer has lost its state
and reconnected with the same peer ID.
Part of #689
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
sirtimidand others added 22 commits February 3, 2026 21:49
- Move handshake to connection level in new handshake.ts module
- Persist incarnation ID to store via getOrCreateIncarnationId()
- Remove incarnationId parameter from RemoteManager (uses store)
- Consolidate identity reset logic in Kernel.resetKernelState()
- Remove Handshake/HandshakeAck from RemoteMessageBase (handled at transport)
The incarnation ID now persists across kernel restarts but resets when
storage is cleared, correctly detecting actual state loss vs. restarts.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove reconnectionHolder indirection pattern in transport.ts
Use definite assignment assertion instead of holder object
- Remove duplicate writeWithTimeout from handshake.ts, import from channel-utils
- Fix PlatformServicesServer to accept incarnationId parameter
- Remove redundant e2e tests for handshake exchange and todo test
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The isHandshakeMessage type guard was only checking for the method
property but not validating that params.incarnationId exists and is
a string. This could cause runtime errors when a malformed message
with the correct method but missing params or incarnationId was
received.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The previous implementation used AbortSignal.timeout() which creates
an internal timer that continues running even after the read succeeds.
This could cause minor memory leaks as the timer and event listener
weren't properly cleaned up.
Now using AbortController with a manual setTimeout/clearTimeout to
ensure the timer is always cleaned up in the finally block.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace the definite assignment assertion (!) with a safer holder
pattern that throws a clear error if handleConnectionLoss is called
before initialization. This provides runtime safety if code is
refactored incorrectly in the future, while keeping TypeScript happy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove unnecessary delay(3000) in incarnation detection test since
await will naturally wait for the promise to settle
- Add cleanup for the fresh database file created during the test
to avoid accumulating test artifacts on disk
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement the 5 tests that were marked as .todo() due to async
handshake flow complexity:
- times out after 10 seconds when write hangs
- handles timeout errors and triggers connection loss handling
- error message includes correct timeout duration
- handles multiple concurrent writes with timeout
- reuses existing channel when inbound connection arrives during
reconnection dial
All tests now use proper mocking with makeAbortSignalMock to
simulate timeout behavior in a controllable way.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Extract the abort handler to a named function so it can be removed
in the finally block, preventing a minor memory leak where the
event listener would remain registered after the read completes.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
RemoteManager independently gets the incarnation ID from kernelStore,
making the Kernel field redundant.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The HandshakeDeps interface defined getRemoteIncarnation but neither
performOutboundHandshake nor performInboundHandshake used it.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ient
The PlatformServicesClient.initializeRemoteComms method was missing the
incarnationId parameter, causing the handshake feature to silently fail
in the browser runtime.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ages
If JSON.parse returns null, accessing parsed.method throws a TypeError
before the ?? 'unknown' fallback can evaluate. Use optional chaining to
safely handle null values.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove unused getRemoteIncarnation method from PeerStateManager
- Change Logger import to type-only import in handshake.ts
- Update tests to use getState() instead of removed method
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace file-based database with in-memory database in the incarnation
detection e2e test. Each new :memory: SQLite database is a fresh instance
that generates a new incarnationId, which is exactly what the test needs.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… handshake
Move onRemoteGiveUp callback invocation outside the try-catch block for
handshake errors. This ensures callback errors don't incorrectly mark
a successful handshake as failed, which would cause unnecessary
connection closures and reconnection attempts.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…test
Address review feedback: check for "Remote connection lost" error message
instead of just checking if the response is an Error instance.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Handshake failures during reconnection were incorrectly treated as
non-retryable errors, causing immediate give-up even for transient
network issues like timeouts.
Changed behavior: instead of throwing an error when handshake fails,
return null to signal retry. This is consistent with other retryable
conditions in tryReconnect.
Also fix duplicate closeChannel key in test file.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Assign the fresh database to kernelDatabase2 so that:
1. The afterEach cleanup closes the correct database
2. The old database isn't double-closed
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ing handshake
Co-authored-by: Cursor <cursoragent@cursor.com>
- Remove duplicate test cases in handshake.test.ts that were covered by it.each
- Add comment explaining when incarnation ID is not configured (tests only)
- Add missing incarnationId parameter to NodeJS PlatformServices
- Update NodeJS PlatformServices tests for new parameter
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@sirtimid
sirtimidforce-pushed the sirtimid/issue-689-incarnation-detection branch from cb58190 to f150b82CompareFebruary 3, 2026 20:54
@sirtimid
sirtimid enabled auto-merge February 3, 2026 20:59

@FUDCoFUDCo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Comment on lines +522 to +529
if (resetIdentity) {
// Full reset including identity - used when recovering from mnemonic
this.#kernelStore.reset();
} else {
// Preserve identity keys so network address survives restart
this.#kernelStore.reset({
except: ['keySeed', 'peerId', 'ocapURLKey', 'incarnationId'],
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

@sirtimid
sirtimid added this pull request to the merge queueFeb 3, 2026
Merged via the queue into main with commit 353bfecFeb 3, 2026
35 checks passed
@sirtimid
sirtimid deleted the sirtimid/issue-689-incarnation-detection branch February 3, 2026 23:34
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remote comms: Kernel incarnation detection

3 participants

@sirtimid@FUDCo@rekmarks