fix(server): reconcile stalled provider starts by rynfar · Pull Request #192 · pylon-code/pylon

rynfar · 2026-08-30T23:14:11Z

Problem

A native provider could remain logically present while teardown or startup was stuck, leaving a durable turn indefinitely starting. Provider lifecycle events could also race a timeout, retry, replacement, or restart and overwrite the exact admission that currently owned the thread.

Fixes#175.

Fix

Persist a server-clock admission record and reconcile it with exact request, message, provider instance, and immutable session-incarnation CAS commands.
Bound admission work and per-instance startup inventory without interrupting a legitimate long-running turn after its exact start is accepted.
Reject stale, pre-bind, failed-lineage, and replaced-session runtime events across Codex, Claude, Cursor, Grok, OpenCode, Prime ACP, and Prime daemon paths.
Add migration 048 with ambiguity-safe backfill, missing-session recovery, and full-replay parity.
Logically evict Prime sessions before bounded cleanup, preserve ordered events under backpressure, and share one exception-safe native-disposal completion across explicit and scope callers.
Bound replacement gates, multi-session shutdown, and retained admission-fiber cleanup.
Preserve exact pending admission lineage when a runtime-mode change races provider startup, and defer that mode to the next safe session ensure.
Keep temporarily unknown startup inventory on the original admission deadline and accept a late exact start without duplicate settlement.
Treat expected stale session-lifecycle CAS misses as accepted no-ops rather than invariant failures.
Document provider admission timeout and daemon teardown behavior.

Validation

721 focused tests across contracts, orchestration, migration, all provider adapters, ProviderService, and Prime daemon paths
360 exact final Prime runtime/adapter tests after the disposal-latch amendment
Server and contracts typechecks
Targeted lint and format checks
git diff --check origin/pylon...HEAD
Repeated independent adversarial reviews of concurrency, migration, provider boundaries, backpressure, and Effect cleanup
Exact post-review repair: 86 focused orchestration tests and 415 combined ProviderService/Prime adapter/runtime tests
Exact post-review server typecheck plus targeted lint, format, and diff checks

Browser verification was not needed because this change is server-only.

Model: openai-codex/gpt-5.6-sol
Harness: Prime Agent RLM

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

github-actions · 2026-08-30T23:24:53Z

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.7 KiB	+449 B (+3.3%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+56 B (+0.8%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.8 KiB	+393 B (+6.0%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	58.7 KiB	+3.1 KiB (+5.6%)	66.4 KiB	✅
Codex	Live turn messages	10	11	+1 (+10.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.5 KiB	+246 B (+1.8%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	+50 B (+0.7%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+196 B (+3.0%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	58.0 KiB	+1.6 KiB (+2.9%)	66.4 KiB	✅
Claude	Live turn messages	10	9	−1 (−10.0%)	21	✅

Baseline: e62ad6a · PR result: ccbd0e9 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

Codex decoded thread snapshot: 109.5 KiB
Claude decoded thread snapshot: 110.2 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Bound Prime native teardown outside thread permits and gate replacement sessions on teardown completion. Correlate pending turn admissions, time out blocked provider preparation, and reconcile overdue starts on boot. Fixes#175

rynfar · 2026-08-30T23:14:11Z

Problem

A native provider could remain logically present while teardown or startup was stuck, leaving a durable turn indefinitely starting. Provider lifecycle events could also race a timeout, retry, replacement, or restart and overwrite the exact admission that currently owned the thread.

Fixes#175.

Fix

Persist a server-clock admission record and reconcile it with exact request, message, provider instance, and immutable session-incarnation CAS commands.
Bound admission work and per-instance startup inventory without interrupting a legitimate long-running turn after its exact start is accepted.
Reject stale, pre-bind, failed-lineage, and replaced-session runtime events across Codex, Claude, Cursor, Grok, OpenCode, Prime ACP, and Prime daemon paths.
Add migration 048 with ambiguity-safe backfill, missing-session recovery, and full-replay parity.
Logically evict Prime sessions before bounded cleanup, preserve ordered events under backpressure, and share one exception-safe native-disposal completion across explicit and scope callers.
Bound replacement gates, multi-session shutdown, and retained admission-fiber cleanup.
Preserve exact pending admission lineage when a runtime-mode change races provider startup, and defer that mode to the next safe session ensure.
Keep temporarily unknown startup inventory on the original admission deadline and accept a late exact start without duplicate settlement.
Treat expected stale session-lifecycle CAS misses as accepted no-ops rather than invariant failures.
Document provider admission timeout and daemon teardown behavior.

Validation

721 focused tests across contracts, orchestration, migration, all provider adapters, ProviderService, and Prime daemon paths
360 exact final Prime runtime/adapter tests after the disposal-latch amendment
Server and contracts typechecks
Targeted lint and format checks
git diff --check origin/pylon...HEAD
Repeated independent adversarial reviews of concurrency, migration, provider boundaries, backpressure, and Effect cleanup
Exact post-review repair: 86 focused orchestration tests and 415 combined ProviderService/Prime adapter/runtime tests
Exact post-review server typecheck plus targeted lint, format, and diff checks

Browser verification was not needed because this change is server-only.

Model: openai-codex/gpt-5.6-sol
Harness: Prime Agent RLM

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

github-actions · 2026-08-30T23:24:53Z

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.7 KiB	+449 B (+3.3%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+56 B (+0.8%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.8 KiB	+393 B (+6.0%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	58.7 KiB	+3.1 KiB (+5.6%)	66.4 KiB	✅
Codex	Live turn messages	10	11	+1 (+10.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.5 KiB	+246 B (+1.8%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	+50 B (+0.7%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+196 B (+3.0%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	58.0 KiB	+1.6 KiB (+2.9%)	66.4 KiB	✅
Claude	Live turn messages	10	9	−1 (−10.0%)	21	✅

Baseline: e62ad6a · PR result: ccbd0e9 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

Codex decoded thread snapshot: 109.5 KiB
Claude decoded thread snapshot: 110.2 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Bound Prime native teardown outside thread permits and gate replacement sessions on teardown completion. Correlate pending turn admissions, time out blocked provider preparation, and reconcile overdue starts on boot. Fixes#175

rynfar · 2026-08-30T23:14:11Z

Problem

A native provider could remain logically present while teardown or startup was stuck, leaving a durable turn indefinitely starting. Provider lifecycle events could also race a timeout, retry, replacement, or restart and overwrite the exact admission that currently owned the thread.

Fixes#175.

Fix

Persist a server-clock admission record and reconcile it with exact request, message, provider instance, and immutable session-incarnation CAS commands.
Bound admission work and per-instance startup inventory without interrupting a legitimate long-running turn after its exact start is accepted.
Reject stale, pre-bind, failed-lineage, and replaced-session runtime events across Codex, Claude, Cursor, Grok, OpenCode, Prime ACP, and Prime daemon paths.
Add migration 048 with ambiguity-safe backfill, missing-session recovery, and full-replay parity.
Logically evict Prime sessions before bounded cleanup, preserve ordered events under backpressure, and share one exception-safe native-disposal completion across explicit and scope callers.
Bound replacement gates, multi-session shutdown, and retained admission-fiber cleanup.
Preserve exact pending admission lineage when a runtime-mode change races provider startup, and defer that mode to the next safe session ensure.
Keep temporarily unknown startup inventory on the original admission deadline and accept a late exact start without duplicate settlement.
Treat expected stale session-lifecycle CAS misses as accepted no-ops rather than invariant failures.
Document provider admission timeout and daemon teardown behavior.

Validation

721 focused tests across contracts, orchestration, migration, all provider adapters, ProviderService, and Prime daemon paths
360 exact final Prime runtime/adapter tests after the disposal-latch amendment
Server and contracts typechecks
Targeted lint and format checks
git diff --check origin/pylon...HEAD
Repeated independent adversarial reviews of concurrency, migration, provider boundaries, backpressure, and Effect cleanup
Exact post-review repair: 86 focused orchestration tests and 415 combined ProviderService/Prime adapter/runtime tests
Exact post-review server typecheck plus targeted lint, format, and diff checks

Browser verification was not needed because this change is server-only.

Model: openai-codex/gpt-5.6-sol
Harness: Prime Agent RLM

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

github-actions · 2026-08-30T23:24:53Z

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.7 KiB	+449 B (+3.3%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+56 B (+0.8%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.8 KiB	+393 B (+6.0%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	58.7 KiB	+3.1 KiB (+5.6%)	66.4 KiB	✅
Codex	Live turn messages	10	11	+1 (+10.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.5 KiB	+246 B (+1.8%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	+50 B (+0.7%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+196 B (+3.0%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	58.0 KiB	+1.6 KiB (+2.9%)	66.4 KiB	✅
Claude	Live turn messages	10	9	−1 (−10.0%)	21	✅

Baseline: e62ad6a · PR result: ccbd0e9 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

Codex decoded thread snapshot: 109.5 KiB
Claude decoded thread snapshot: 110.2 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Bound Prime native teardown outside thread permits and gate replacement sessions on teardown completion. Correlate pending turn admissions, time out blocked provider preparation, and reconcile overdue starts on boot. Fixes#175

rynfar · 2026-08-30T23:14:11Z

Problem

A native provider could remain logically present while teardown or startup was stuck, leaving a durable turn indefinitely starting. Provider lifecycle events could also race a timeout, retry, replacement, or restart and overwrite the exact admission that currently owned the thread.

Fixes#175.

Fix

Persist a server-clock admission record and reconcile it with exact request, message, provider instance, and immutable session-incarnation CAS commands.
Bound admission work and per-instance startup inventory without interrupting a legitimate long-running turn after its exact start is accepted.
Reject stale, pre-bind, failed-lineage, and replaced-session runtime events across Codex, Claude, Cursor, Grok, OpenCode, Prime ACP, and Prime daemon paths.
Add migration 048 with ambiguity-safe backfill, missing-session recovery, and full-replay parity.
Logically evict Prime sessions before bounded cleanup, preserve ordered events under backpressure, and share one exception-safe native-disposal completion across explicit and scope callers.
Bound replacement gates, multi-session shutdown, and retained admission-fiber cleanup.
Preserve exact pending admission lineage when a runtime-mode change races provider startup, and defer that mode to the next safe session ensure.
Keep temporarily unknown startup inventory on the original admission deadline and accept a late exact start without duplicate settlement.
Treat expected stale session-lifecycle CAS misses as accepted no-ops rather than invariant failures.
Document provider admission timeout and daemon teardown behavior.

Validation

721 focused tests across contracts, orchestration, migration, all provider adapters, ProviderService, and Prime daemon paths
360 exact final Prime runtime/adapter tests after the disposal-latch amendment
Server and contracts typechecks
Targeted lint and format checks
git diff --check origin/pylon...HEAD
Repeated independent adversarial reviews of concurrency, migration, provider boundaries, backpressure, and Effect cleanup
Exact post-review repair: 86 focused orchestration tests and 415 combined ProviderService/Prime adapter/runtime tests
Exact post-review server typecheck plus targeted lint, format, and diff checks

Browser verification was not needed because this change is server-only.

Model: openai-codex/gpt-5.6-sol
Harness: Prime Agent RLM

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

github-actions · 2026-08-30T23:24:53Z

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.7 KiB	+449 B (+3.3%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+56 B (+0.8%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.8 KiB	+393 B (+6.0%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	58.7 KiB	+3.1 KiB (+5.6%)	66.4 KiB	✅
Codex	Live turn messages	10	11	+1 (+10.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.5 KiB	+246 B (+1.8%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	+50 B (+0.7%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+196 B (+3.0%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	58.0 KiB	+1.6 KiB (+2.9%)	66.4 KiB	✅
Claude	Live turn messages	10	9	−1 (−10.0%)	21	✅

Baseline: e62ad6a · PR result: ccbd0e9 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

Codex decoded thread snapshot: 109.5 KiB
Claude decoded thread snapshot: 110.2 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Bound Prime native teardown outside thread permits and gate replacement sessions on teardown completion. Correlate pending turn admissions, time out blocked provider preparation, and reconcile overdue starts on boot. Fixes#175

rynfar · 2026-08-30T23:14:11Z

Problem

A native provider could remain logically present while teardown or startup was stuck, leaving a durable turn indefinitely starting. Provider lifecycle events could also race a timeout, retry, replacement, or restart and overwrite the exact admission that currently owned the thread.

Fixes#175.

Fix

Persist a server-clock admission record and reconcile it with exact request, message, provider instance, and immutable session-incarnation CAS commands.
Bound admission work and per-instance startup inventory without interrupting a legitimate long-running turn after its exact start is accepted.
Reject stale, pre-bind, failed-lineage, and replaced-session runtime events across Codex, Claude, Cursor, Grok, OpenCode, Prime ACP, and Prime daemon paths.
Add migration 048 with ambiguity-safe backfill, missing-session recovery, and full-replay parity.
Logically evict Prime sessions before bounded cleanup, preserve ordered events under backpressure, and share one exception-safe native-disposal completion across explicit and scope callers.
Bound replacement gates, multi-session shutdown, and retained admission-fiber cleanup.
Preserve exact pending admission lineage when a runtime-mode change races provider startup, and defer that mode to the next safe session ensure.
Keep temporarily unknown startup inventory on the original admission deadline and accept a late exact start without duplicate settlement.
Treat expected stale session-lifecycle CAS misses as accepted no-ops rather than invariant failures.
Document provider admission timeout and daemon teardown behavior.

Validation

721 focused tests across contracts, orchestration, migration, all provider adapters, ProviderService, and Prime daemon paths
360 exact final Prime runtime/adapter tests after the disposal-latch amendment
Server and contracts typechecks
Targeted lint and format checks
git diff --check origin/pylon...HEAD
Repeated independent adversarial reviews of concurrency, migration, provider boundaries, backpressure, and Effect cleanup
Exact post-review repair: 86 focused orchestration tests and 415 combined ProviderService/Prime adapter/runtime tests
Exact post-review server typecheck plus targeted lint, format, and diff checks

Browser verification was not needed because this change is server-only.

Model: openai-codex/gpt-5.6-sol
Harness: Prime Agent RLM

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

github-actions · 2026-08-30T23:24:53Z

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.7 KiB	+449 B (+3.3%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+56 B (+0.8%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.8 KiB	+393 B (+6.0%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	58.7 KiB	+3.1 KiB (+5.6%)	66.4 KiB	✅
Codex	Live turn messages	10	11	+1 (+10.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.5 KiB	+246 B (+1.8%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	+50 B (+0.7%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+196 B (+3.0%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	58.0 KiB	+1.6 KiB (+2.9%)	66.4 KiB	✅
Claude	Live turn messages	10	9	−1 (−10.0%)	21	✅

Baseline: e62ad6a · PR result: ccbd0e9 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

Codex decoded thread snapshot: 109.5 KiB
Claude decoded thread snapshot: 110.2 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Bound Prime native teardown outside thread permits and gate replacement sessions on teardown completion. Correlate pending turn admissions, time out blocked provider preparation, and reconcile overdue starts on boot. Fixes#175

rynfar · 2026-08-30T23:14:11Z

Problem

A native provider could remain logically present while teardown or startup was stuck, leaving a durable turn indefinitely starting. Provider lifecycle events could also race a timeout, retry, replacement, or restart and overwrite the exact admission that currently owned the thread.

Fixes#175.

Fix

Persist a server-clock admission record and reconcile it with exact request, message, provider instance, and immutable session-incarnation CAS commands.
Bound admission work and per-instance startup inventory without interrupting a legitimate long-running turn after its exact start is accepted.
Reject stale, pre-bind, failed-lineage, and replaced-session runtime events across Codex, Claude, Cursor, Grok, OpenCode, Prime ACP, and Prime daemon paths.
Add migration 048 with ambiguity-safe backfill, missing-session recovery, and full-replay parity.
Logically evict Prime sessions before bounded cleanup, preserve ordered events under backpressure, and share one exception-safe native-disposal completion across explicit and scope callers.
Bound replacement gates, multi-session shutdown, and retained admission-fiber cleanup.
Preserve exact pending admission lineage when a runtime-mode change races provider startup, and defer that mode to the next safe session ensure.
Keep temporarily unknown startup inventory on the original admission deadline and accept a late exact start without duplicate settlement.
Treat expected stale session-lifecycle CAS misses as accepted no-ops rather than invariant failures.
Document provider admission timeout and daemon teardown behavior.

Validation

721 focused tests across contracts, orchestration, migration, all provider adapters, ProviderService, and Prime daemon paths
360 exact final Prime runtime/adapter tests after the disposal-latch amendment
Server and contracts typechecks
Targeted lint and format checks
git diff --check origin/pylon...HEAD
Repeated independent adversarial reviews of concurrency, migration, provider boundaries, backpressure, and Effect cleanup
Exact post-review repair: 86 focused orchestration tests and 415 combined ProviderService/Prime adapter/runtime tests
Exact post-review server typecheck plus targeted lint, format, and diff checks

Browser verification was not needed because this change is server-only.

Model: openai-codex/gpt-5.6-sol
Harness: Prime Agent RLM

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

github-actions · 2026-08-30T23:24:53Z

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.7 KiB	+449 B (+3.3%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+56 B (+0.8%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.8 KiB	+393 B (+6.0%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	58.7 KiB	+3.1 KiB (+5.6%)	66.4 KiB	✅
Codex	Live turn messages	10	11	+1 (+10.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.5 KiB	+246 B (+1.8%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	+50 B (+0.7%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+196 B (+3.0%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	58.0 KiB	+1.6 KiB (+2.9%)	66.4 KiB	✅
Claude	Live turn messages	10	9	−1 (−10.0%)	21	✅

Baseline: e62ad6a · PR result: ccbd0e9 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

Codex decoded thread snapshot: 109.5 KiB
Claude decoded thread snapshot: 110.2 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Bound Prime native teardown outside thread permits and gate replacement sessions on teardown completion. Correlate pending turn admissions, time out blocked provider preparation, and reconcile overdue starts on boot. Fixes#175

rynfar · 2026-08-30T23:14:11Z

Problem

A native provider could remain logically present while teardown or startup was stuck, leaving a durable turn indefinitely starting. Provider lifecycle events could also race a timeout, retry, replacement, or restart and overwrite the exact admission that currently owned the thread.

Fixes#175.

Fix

Persist a server-clock admission record and reconcile it with exact request, message, provider instance, and immutable session-incarnation CAS commands.
Bound admission work and per-instance startup inventory without interrupting a legitimate long-running turn after its exact start is accepted.
Reject stale, pre-bind, failed-lineage, and replaced-session runtime events across Codex, Claude, Cursor, Grok, OpenCode, Prime ACP, and Prime daemon paths.
Add migration 048 with ambiguity-safe backfill, missing-session recovery, and full-replay parity.
Logically evict Prime sessions before bounded cleanup, preserve ordered events under backpressure, and share one exception-safe native-disposal completion across explicit and scope callers.
Bound replacement gates, multi-session shutdown, and retained admission-fiber cleanup.
Preserve exact pending admission lineage when a runtime-mode change races provider startup, and defer that mode to the next safe session ensure.
Keep temporarily unknown startup inventory on the original admission deadline and accept a late exact start without duplicate settlement.
Treat expected stale session-lifecycle CAS misses as accepted no-ops rather than invariant failures.
Document provider admission timeout and daemon teardown behavior.

Validation

721 focused tests across contracts, orchestration, migration, all provider adapters, ProviderService, and Prime daemon paths
360 exact final Prime runtime/adapter tests after the disposal-latch amendment
Server and contracts typechecks
Targeted lint and format checks
git diff --check origin/pylon...HEAD
Repeated independent adversarial reviews of concurrency, migration, provider boundaries, backpressure, and Effect cleanup
Exact post-review repair: 86 focused orchestration tests and 415 combined ProviderService/Prime adapter/runtime tests
Exact post-review server typecheck plus targeted lint, format, and diff checks

Browser verification was not needed because this change is server-only.

Model: openai-codex/gpt-5.6-sol
Harness: Prime Agent RLM

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

github-actions · 2026-08-30T23:24:53Z

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.7 KiB	+449 B (+3.3%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+56 B (+0.8%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.8 KiB	+393 B (+6.0%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	58.7 KiB	+3.1 KiB (+5.6%)	66.4 KiB	✅
Codex	Live turn messages	10	11	+1 (+10.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.5 KiB	+246 B (+1.8%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	+50 B (+0.7%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+196 B (+3.0%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	58.0 KiB	+1.6 KiB (+2.9%)	66.4 KiB	✅
Claude	Live turn messages	10	9	−1 (−10.0%)	21	✅

Baseline: e62ad6a · PR result: ccbd0e9 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

Codex decoded thread snapshot: 109.5 KiB
Claude decoded thread snapshot: 110.2 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Bound Prime native teardown outside thread permits and gate replacement sessions on teardown completion. Correlate pending turn admissions, time out blocked provider preparation, and reconcile overdue starts on boot. Fixes#175

rynfar · 2026-08-30T23:14:11Z

Problem

A native provider could remain logically present while teardown or startup was stuck, leaving a durable turn indefinitely starting. Provider lifecycle events could also race a timeout, retry, replacement, or restart and overwrite the exact admission that currently owned the thread.

Fixes#175.

Fix

Persist a server-clock admission record and reconcile it with exact request, message, provider instance, and immutable session-incarnation CAS commands.
Bound admission work and per-instance startup inventory without interrupting a legitimate long-running turn after its exact start is accepted.
Reject stale, pre-bind, failed-lineage, and replaced-session runtime events across Codex, Claude, Cursor, Grok, OpenCode, Prime ACP, and Prime daemon paths.
Add migration 048 with ambiguity-safe backfill, missing-session recovery, and full-replay parity.
Logically evict Prime sessions before bounded cleanup, preserve ordered events under backpressure, and share one exception-safe native-disposal completion across explicit and scope callers.
Bound replacement gates, multi-session shutdown, and retained admission-fiber cleanup.
Preserve exact pending admission lineage when a runtime-mode change races provider startup, and defer that mode to the next safe session ensure.
Keep temporarily unknown startup inventory on the original admission deadline and accept a late exact start without duplicate settlement.
Treat expected stale session-lifecycle CAS misses as accepted no-ops rather than invariant failures.
Document provider admission timeout and daemon teardown behavior.

Validation

721 focused tests across contracts, orchestration, migration, all provider adapters, ProviderService, and Prime daemon paths
360 exact final Prime runtime/adapter tests after the disposal-latch amendment
Server and contracts typechecks
Targeted lint and format checks
git diff --check origin/pylon...HEAD
Repeated independent adversarial reviews of concurrency, migration, provider boundaries, backpressure, and Effect cleanup
Exact post-review repair: 86 focused orchestration tests and 415 combined ProviderService/Prime adapter/runtime tests
Exact post-review server typecheck plus targeted lint, format, and diff checks

Browser verification was not needed because this change is server-only.

Model: openai-codex/gpt-5.6-sol
Harness: Prime Agent RLM

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

github-actions · 2026-08-30T23:24:53Z

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	13.3 KiB	13.7 KiB	+449 B (+3.3%)	15.1 KiB	✅
Codex	Thread snapshot wire	6.9 KiB	6.9 KiB	+56 B (+0.8%)	7.3 KiB	✅
Codex	Live turn WebSocket wire	6.4 KiB	6.8 KiB	+393 B (+6.0%)	7.8 KiB	✅
Codex	Live turn WebSocket decoded	55.6 KiB	58.7 KiB	+3.1 KiB (+5.6%)	66.4 KiB	✅
Codex	Live turn messages	10	11	+1 (+10.0%)	21	✅
Claude	Total thread wire	13.3 KiB	13.5 KiB	+246 B (+1.8%)	15.1 KiB	✅
Claude	Thread snapshot wire	6.9 KiB	6.9 KiB	+50 B (+0.7%)	7.3 KiB	✅
Claude	Live turn WebSocket wire	6.4 KiB	6.6 KiB	+196 B (+3.0%)	7.8 KiB	✅
Claude	Live turn WebSocket decoded	56.4 KiB	58.0 KiB	+1.6 KiB (+2.9%)	66.4 KiB	✅
Claude	Live turn messages	10	9	−1 (−10.0%)	21	✅

Baseline: e62ad6a · PR result: ccbd0e9 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

Codex decoded thread snapshot: 109.5 KiB
Claude decoded thread snapshot: 110.2 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Bound Prime native teardown outside thread permits and gate replacement sessions on teardown completion. Correlate pending turn admissions, time out blocked provider preparation, and reconcile overdue starts on boot. Fixes#175

github-actionsBot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XXL labels Aug 30, 2026

rynfar force-pushed the fix/provider-start-reconciliation branch from d09f6ac to 12b7c80Compare August 30, 2026 23:29

rynfar mentioned this pull request Aug 30, 2026
Scheduled agent runs: write the safety contract before any implementation #177
Open

rynfar force-pushed the fix/provider-start-reconciliation branch from 12b7c80 to 9867ad2Compare August 31, 2026 03:09

rynfar mentioned this pull request Aug 31, 2026
Prime provider parity: tasks, automation, and upstream watch #114
Open
23 tasks

rynfar added 2 commits August 30, 2026 21:36

fix(server): reconcile stalled provider starts
8d3fd19
Bound Prime native teardown outside thread permits and gate replacement sessions on teardown completion. Correlate pending turn admissions, time out blocked provider preparation, and reconcile overdue starts on boot. Fixes#175

fix(server): preserve provider admission lineage
ccbd0e9

rynfar force-pushed the fix/provider-start-reconciliation branch from 9867ad2 to ccbd0e9Compare August 31, 2026 03:44

rynfar mentioned this pull request Aug 31, 2026
Prime Agent active turns cannot survive a Pylon server restart #84
Closed

rynfar merged commit 4ce0070 into pylonAug 31, 2026
15 checks passed

rynfar deleted the fix/provider-start-reconciliation branch August 31, 2026 03:53