Skip to content

fix(storage): serialize operational schema migrations - #2445

Merged
Astro-Han merged 29 commits into
mainfrom
fix/2357-serialize-operational-migrations
Aug 13, 2026
Merged

fix(storage): serialize operational schema migrations#2445
Astro-Han merged 29 commits into
mainfrom
fix/2357-serialize-operational-migrations

Conversation

@Astro-Han

@Astro-HanAstro-Han commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Refs #2357.

Released operational databases need one migration owner: classify every schema scope, recheck under the write lock, migrate, and publish the new registry as one SQLite transaction.

This PR:

  • runs operational schema upgrades under one BEGIN IMMEDIATE transaction and rolls back every scope on failure;
  • migrates released v0.1.6-v0.1.10 Plan Reminder records into the current Scheduled Task authority when every persisted fact is representable;
  • fails closed before mutation when released Automation exists or Reminder state cannot be represented losslessly, leaving the old database available to the previous Maka release;
  • preserves released Automation provenance in Session messages, admissions, AgentRuns, and RuntimeEvents after refactor: remove duplicate runtime architecture #2742 removed the execution authority;
  • canonically settles orphaned legacy Automation executions during Runtime Host recovery and shows their provenance as read-only history;
  • reports migration blockers and permanent storage failures through stable Candidate exit codes to actionable CLI and Desktop errors.

The upgrade is one-way after a successful migration. It does not restore historical backups, preserve Automation as a second authority, convert legacy Automation into Scheduled Tasks, or support an old process continuing to write after migration.

Verification

  • Built @maka/core, @maka/storage, @maka/runtime, @maka/runtime-host, Desktop main, and @maka/ui from clean artifacts.
  • Focused Operational migration and concurrency tests: 32 passed.
  • Focused persisted-codec and Core execution tests: 27 passed.
  • Focused Runtime Host production-composition tests: 11 passed.
  • Focused Runtime Host kernel and Candidate lifecycle tests: 47 passed.
  • Focused Desktop owner tests: 16 passed.
  • Focused CLI context tests: 5 passed.
  • npm run windows:inventory.
  • npm run format:check.

Coverage includes concurrent initialization, transaction rollback, a frozen real v0.1.6 database, successful Reminder migration and reopen, Automation fail-closed behavior, unrepresentable Reminder history preservation, legacy Automation message recovery, canonical closure of an orphaned legacy admission, Candidate failure transport, and single actionable CLI/Desktop presentation.

Review focus

Please verify the released database lifecycle from classification through migration or rollback and reopen. In particular:

  • successful migration preserves every representable Reminder fact;
  • Automation and unrepresentable Reminder state fail before mutation while their authority still exists;
  • databases already opened by refactor: remove duplicate runtime architecture #2742 retain readable legacy Automation history and recover without restoring that authority;
  • migration and storage failures reach CLI/Desktop once with actionable guidance.

Checklist

  • Tests cover the change and fail without it
  • Format, typecheck for affected workspaces, and focused suites pass locally

Does this PR entail a change in behavior?

  • Yes - described under Summary above
  • No

AI assistance

OpenAI Codex materially assisted implementation, tests, analysis, and review. A human contributor must review the final diff and commit messages and owns the submission and merge decision.

@Astro-Han
Astro-Hanforce-pushed the fix/2357-serialize-operational-migrations branch from c87604f to bb810a0CompareAugust 7, 2026 14:53
@Astro-Han
Astro-Han requested a review from zhiiwAugust 7, 2026 15:41
@Astro-Han

Astro-Han commented Aug 7, 2026

Copy link
Copy Markdown
ContributorAuthor

@likun666661@zhiiw Could you please review this replacement for #2363?

The implementation has been reduced to two storage-only commits: one transaction for every operational scope and the registry, plus a lifetime shared lock for every live owner with exclusive migration.

Please focus on whether the result is optimal from first principles and Occam's razor, whether the migration and cross-process locking model is correct, whether the tests are sufficient, and whether any remaining complexity can be removed or refactored into a cleaner solution.

@zhiiwzhiiw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two blocking correctness issues are noted inline.

Comment threadpackages/storage/src/operational-state-schema-lock.ts Outdated
Comment threadpackages/storage/src/operational-state-store.ts Outdated
@Astro-Han
Astro-Hanforce-pushed the fix/2357-serialize-operational-migrations branch from bb810a0 to 0c90963CompareAugust 8, 2026 07:34
@Astro-Han

Copy link
Copy Markdown
ContributorAuthor

@likun666661@zhiiw Could you please review the latest revision?

The previously identified migration-turn race and v0.1.6 backup restoration issue have been fixed. The latest head is fee391a67.

Please re-evaluate the complete PR for correctness, first-principles simplicity, locking and migration behavior, test quality, and any remaining unnecessary complexity.

@zhiiwzhiiw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Four remaining correctness and cross-platform test issues are noted inline.

Comment threadpackages/storage/src/__tests__/operational-state-schema-lock.test.ts Outdated
Comment threadpackages/storage/src/index.ts
Comment threadpackages/storage/src/operational-state-schema-lock.ts Outdated
Comment threadpackages/storage/src/__tests__/operational-state-schema-lock.test.ts Outdated
@Astro-Han
Astro-Hanforce-pushed the fix/2357-serialize-operational-migrations branch 5 times, most recently from d2814b3 to eb4f1afCompareAugust 8, 2026 14:52

@zhiiwzhiiw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three remaining durability, fail-closed consumer, and Windows test issues are noted inline.

Comment threadpackages/storage/src/operational-state-store.ts Outdated
Comment threadpackages/storage/src/usage-stats-store.ts
Comment threadpackages/storage/src/__tests__/operational-state-schema-lock.test.ts Outdated
@Astro-Han
Astro-Hanforce-pushed the fix/2357-serialize-operational-migrations branch 6 times, most recently from 3601d6c to 7c168dfCompareAugust 9, 2026 05:54
@Astro-Han
Astro-Han requested a review from zhiiwAugust 9, 2026 05:56
@Astro-Han

Copy link
Copy Markdown
ContributorAuthor

@likun666661@zhiiw All previous review threads are now addressed and resolved, and the PR history and body reflect the final six-commit design. Could you re-review the latest head, focusing on the complete upgrade and restore lifecycle and any remaining removable complexity?

Comment threadpackages/storage/src/operational-state-store.ts Outdated
@Astro-Han
Astro-Han requested a review from zhiiwAugust 10, 2026 17:19
@Astro-Han

Copy link
Copy Markdown
ContributorAuthor

@likun666661@zhiiw All previous review threads are addressed and resolved. The latest fix at 67ac7dc1d rejects damaged registryless state while preserving fresh and Runtime-only bootstrap paths. Could you re-review the current head and approve if the migration and restore contracts now look correct?

AI assistance disclosure: This automated request was prepared by OpenAI Codex on behalf of Astro-Han.

@Astro-Han
Astro-Hanforce-pushed the fix/2357-serialize-operational-migrations branch 4 times, most recently from 593085e to 28af3ceCompareAugust 11, 2026 02:22
@Astro-Han
Astro-Han marked this pull request as ready for review August 11, 2026 03:21
@Astro-Han
Astro-Hanforce-pushed the fix/2357-serialize-operational-migrations branch from c786848 to 5d17f72CompareAugust 13, 2026 10:56
@Astro-Han

Copy link
Copy Markdown
ContributorAuthor

@M4n5ter@likun666661 Could you please perform the final review of the rebased head 5d17f72e4?

The rebase integrates #2922 at the existing ownership boundary:

  • Session metadata v22 → v23 now runs through the caller-owned migrator inside the unified operational BEGIN IMMEDIATE transaction.
  • Target-schema verification automatically includes the transcript payload/chunk tables.
  • Backup validation retains perf(runtime-host): replace transcript snapshots with bounded pages #2922’s payload/chunk consistency checks.
  • The rollback test now explicitly verifies that the Session metadata version and both v23 tables remain unchanged when publication fails.
  • The Runtime Host compatibility epoch is now 20, preserving both main’s transcript protocol change and this PR’s legacy Automation provenance change.

Focused Storage and Runtime Host validations pass. Required CI is still running.

OpenAI Codex materially assisted with the rebase, conflict resolution, tests, and analysis. I reviewed the resulting changes and own the submission.


中文对照:

@M4n5ter@likun666661,麻烦对 rebase 后的最新 HEAD 5d17f72e4 做最终审查。

本次 rebase 已在现有 owner 边界内整合 #2922

  • Session metadata v22 → v23 现在通过 caller-owned migrator,在统一的 operational BEGIN IMMEDIATE 事务内执行。
  • Target-schema verification 会自动包含 transcript payload/chunk tables。
  • Backup validation 保留 perf(runtime-host): replace transcript snapshots with bounded pages #2922 的 payload/chunk 一致性检查。
  • 回滚测试现在会明确验证:发布失败时,Session metadata 版本及两张 v23 表均保持不变。
  • Runtime Host compatibility epoch 更新为 20,同时保留 main 的 transcript 协议变化和本 PR 的 legacy Automation provenance 变化。

Storage 和 Runtime Host 的聚焦验证均已通过。Required CI 仍在运行。

OpenAI Codex 对 rebase、冲突处理、测试和分析提供了实质协助。我已审阅最终改动,并对本次提交负责。

@M4n5terM4n5ter left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

English

Re-reviewed the rebased head 5d17f72. The Session metadata v22 → v23 migration is correctly integrated into the unified operational transaction, including target-schema verification and rollback. Backup consistency checks and compatibility epoch 20 are also preserved.

No remaining blocking issues. Approved.

中文

已重新审查 rebase 后的 5d17f72。Session metadata v22 → v23 迁移已正确纳入统一的 operational transaction,包括目标 schema 验证和失败回滚;backup 一致性校验与 compatibility epoch 20 也均得到保留。

没有剩余阻塞问题,可以批准。

AI assistance disclosure: OpenAI Codex assisted with this review. I reviewed the analysis and own the final review decision.

@Astro-Han
Astro-Han merged commit ce6534d into mainAug 13, 2026
11 checks passed
@Astro-Han
Astro-Han deleted the fix/2357-serialize-operational-migrations branch August 13, 2026 15:18
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Astro-Han@M4n5ter@likun666661@zhiiw