Uh oh!
There was an error while loading. Please reload this page.
fix(platform-wallet-storage): reject duplicate (wallet_id, identity_index) on write - #4441
Conversation
…slot `identity_index` is an HD derivation-path component, so `(wallet_id, identity_index)` names exactly one identity. A duplicate was accepted at write time: the displaced identity's keys and contacts lost their owner, and the next `load()` failed the WHOLE wallet's saved state with `OrphanedIdentityEntry`. `store()` now probes for slot occupancy BEFORE the changeset joins the shared per-wallet buffer, so the error names the write that caused it and cannot drop a changeset another caller staged for the same wallet. The probe keys occupancy on the flush scope (the slot the write actually lands in, since the upsert promotes a NULL wallet_id into it), skips tombstoned rows so a freed slot stays reusable, and treats ids in `cs.removed` as holding nothing — `apply` inserts before it tombstones, so "tombstone A@N + insert B@N" in one changeset stays legal. Colliding entries inside a single changeset are rejected too, without picking a winner. A wallet-less identity carrying an index is refused outright: out-of-wallet identities are keyed by identity id alone. Two typed variants, both non-transient and `Constraint`-kind: `IdentityIndexConflict` and `WalletlessIdentityIndex`. Repairing duplicates already on disk stays out of scope. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`restore_from` replaces the destination file wholesale, so the write-path identity-slot check never sees those bytes. The source is validated for structure only — a duplicate inside it survives the restore and surfaces at the next `load()`. Also note that `delete_wallet`'s pre-flush applies a changeset outside `store()`, so its slot check was made against possibly-stale disk state. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e too `store` validates each changeset before it enters the buffer, but the buffer merges. Under `FlushMode::Manual`, `store(A@1)` and `store(B@1)` are each valid against a clean disk and merge into one contradictory changeset that the flush wrote unchallenged. `identities::apply` now runs the same check against what is actually about to hit disk — which is also the only cover for `delete_wallet`'s pre-flush, the one apply path that never passes through `store`. Flush-time rejection is fatal by the existing classification, so the offending wallet's buffered changeset is dropped rather than restored: it can never be applied, and restoring it would fail every future flush of that wallet identically. Blast radius is that wallet alone — each wallet flushes in its own transaction. Same reasoning made `delete_wallet` tolerate a constraint failure in its pre-flush: pending writes that can never be persisted would otherwise make the wallet permanently undeletable, which is the state a user most wants to be able to delete. They are dropped with a warning and the cascade proceeds; they name only the wallet whose rows are about to go. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Marvin QA pass on fix/pws-reject-duplicate-identity-index (identity-index write-path guard, memcan TODO 4fbe235c-1bcd-4a32-baea-5b4f9d67e526). - sqlite_identity_index_concurrency.rs: two threads racing store() on the SAME wallet + SAME index can both pass the pre-buffer probe (disk state hasn't moved yet), merge into one contradictory buffered changeset, and the flush-time backstop then rejects it whole — but only ONE racing store() call drives the actual flush; the other's take_for_flush finds nothing and returns Ok(()) while its identity never reaches disk. Violates R3 and the documented "durable on Ok" contract of FlushMode::Immediate. Reproduced (rare, jammer-assisted, not CI-grade). - sqlite_delete_wallet_constraint_carveout.rs: delete_wallet's pre-flush Constraint-kind tolerance (persister.rs) is scoped to the PersistenceErrorKind::Constraint bucket, not to the two new IdentityIndexConflict/WalletlessIdentityIndex variants it was written for. Any native SQLite ConstraintViolation (FK/CHECK/UNIQUE/NOT NULL) in a wallet's drained pre-flush buffer is classified Constraint too, so it is silently dropped and the delete proceeds — a regression from the pre-PR behavior where any apply failure hard-aborted delete_wallet. Demonstrated via an ordinary identity_keys FK violation, unrelated to identity-index uniqueness. QA-only: demonstrates findings, does not fix production code. Full findings routed to the lead / Bilby. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`delete_wallet`'s pre-flush carve-out matched the `Constraint` KIND, which `persistence_kind()` also returns for every native SQLite ConstraintViolation anywhere in the schema — FK, CHECK, UNIQUE, NOT NULL. So an FK violation in a wallet's drained buffer, corruption that hard-aborted the delete before this branch existed, was silently dropped with a `warn!` at the exact moment an operator removes state. Match the two variants the carve-out was written for instead. An unpersistable identity slot still can't make a wallet undeletable; anything else aborts the delete with the buffered changeset restored, so the evidence is still there to look at. Marvin's QA probe flips with it: the FK violation now fails loudly, and the restored changeset is shown to still flush once its FK target exists. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The store-time probe ran under the connection mutex and the merge ran under the buffer's, so two threads claiming one slot could both find it free, both merge, and hand the flush a changeset it can only refuse. The flush refused it whole — including the entry belonging to whichever caller had already been told `Ok(())`. Silent loss reported as success. The check moves inside the buffer's critical section, as a closure `Buffer::store_checked` runs before it merges, and judges the MERGED view — buffered plus incoming, built with the same `Merge` impl the flush will use, so it sees what `apply` sees. Racing callers now serialize on the buffer lock: the second reads the first as the occupant and is refused at store time, where the error still names the write that caused it. `flush_inner` closes the mirror-image window by taking the connection before it drains the buffer and holding it through the write. Mid-flush a changeset is in neither the buffer nor the database; a probe that ran there would read a free slot that isn't, and `store` needs that same lock to check an identity write. The flush-time backstop stays for what remains genuinely outside this process: a sibling persister on the same file. Three tests that used to assemble a contradictory buffer through two stores now stage it that way instead — through a peer's row — because two stores can no longer do it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`delete_wallet`'s pre-flush said its changeset carried only a store-time slot check made against disk that may have moved. It hasn't been true since the check started running inside this very transaction; say what guards it now and what the tolerance below covers. `identities::apply` gets the same treatment: an in-process pair of stores can no longer merge into a contradiction, so the backstop's stated reason is a peer process on the same file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ydration' into fix/pws-reject-duplicate-identity-index # Conflicts: # packages/rs-platform-wallet-storage/src/sqlite/error.rs # packages/rs-platform-wallet-storage/src/sqlite/persister.rs # packages/rs-platform-wallet-storage/src/sqlite/schema/identities.rs # packages/rs-platform-wallet-storage/tests/sqlite_error_classification.rs
⛔ Blockers found — Opus deferred (commit c0264e4) |
Uh oh!
There was an error while loading. Please reload this page.
thepastaclaw
left a comment
There was a problem hiding this comment.
Preliminary review — Codex only
The write guard correctly rejects straightforward duplicate identity slots and closes the original in-process check-to-merge race. However, the delete carve-out can lose buffered writes after a failed deletion, and Immediate mode still allows another flush to discard an accepted identity changeset without reporting the failure to its originating store call; atomic reindexing is also rejected despite having a unique final state. Source: Codex general, Rust-quality, and security reviewer evidence (exact reviewer backend model IDs were not present in the supplied projection); final verifier backend: cliproxy/gpt-5.6-sol; openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.
Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.
Review provenance
- Codex reviewers:
gpt-5.6-sol— general (completed),gpt-5.6-sol— rust-quality (completed),gpt-5.6-sol— security-auditor (completed) - Verifier:
gpt-5.6-sol— verifier - Sonnet: not run (deferred by blocker gate)
🔴 2 blocking | 🟡 2 suggestion(s)
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-platform-wallet-storage/src/sqlite/persister.rs`:
- [BLOCKING] packages/rs-platform-wallet-storage/src/sqlite/persister.rs:743-754: Retain the drained changeset until deletion commits
The identity-conflict carve-out consumes `cs` and leaves `drained_slot` empty before the wallet has been deleted. If a later operation fails—such as automatic backup creation, opening or executing the delete transaction, or committing it—the outer `restore_buffer` call has nothing to restore. The wallet remains, but the previously accepted Manual-mode changeset, including any unrelated valid sub-changesets bundled with the conflicting identity write, is permanently lost. Put `cs` back into `drained_slot` after rolling back the pre-flush transaction; the existing `drop(drained_slot.take())` after the delete commit will still discard it on the successful path.
- [BLOCKING] packages/rs-platform-wallet-storage/src/sqlite/persister.rs:1169-1185: Concurrent flush can hide rejection from the identity writer
Immediate-mode `store()` releases the connection after `store_checked()` and reacquires it only in `flush_inner()`. During that gap, another thread can drain the accepted changeset. For example, process A checks a free slot and buffers identity A; process B independently commits identity B to that slot; then another thread in process A calls `flush(wallet_id)`, drains A, detects B at the transactional backstop, and fatally drops A. When A's original store call resumes, its own `flush_inner()` sees an empty buffer and returns `Ok(())`. The caller is therefore told that A is durable even though its mapping was discarded, contradicting the documented Immediate-mode guarantee and allowing it to continue using ambiguous HD-derived key material. Serialize the checked merge and its Immediate-mode flush as one operation—by retaining exclusive ownership of the connection/write sequence or using an equivalent per-wallet operation lock—so no other flush can consume the changeset before its originating `store()` receives the result.
In `packages/rs-platform-wallet-storage/src/sqlite/schema/identities.rs`:
- [SUGGESTION] packages/rs-platform-wallet-storage/src/sqlite/schema/identities.rs:178-196: Treat identities reindexed by the same changeset as vacating their old slots
The disk occupant is treated as vacating a slot only when it appears in `cs.removed`, even though `apply()` also updates `identity_index` for entries in `cs.identities`. A changeset such as `{A -> 2, B -> 1}` is therefore rejected when A currently occupies index 1, despite its final state being unique; a two-way swap is rejected for the same reason. Evaluate every on-disk occupant and ignore one when the changeset removes it or assigns it a different final index. Iterating all matching rows also preserves the intended behavior for legacy databases that may already contain more than one occupant.
In `packages/rs-platform-wallet-storage/tests/sqlite_identity_index_concurrency.rs`:
- [SUGGESTION] packages/rs-platform-wallet-storage/tests/sqlite_identity_index_concurrency.rs:95-138: Replace the scheduling stress loop with a deterministic interleaving test
The barrier aligns the two store calls only at their start; it does not force either thread into the former probe-to-merge window. A regression can therefore pass all 500 attempts depending on scheduling, while each attempt creates a database and six threads and four threads continuously contend on the connection mutex. Add a test-only synchronization point that pauses the first caller while it holds the relevant critical section, starts the second caller, and then releases the first. That would exercise the ordering guarantee deterministically with substantially less timing-dependent CI load.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
72395da
into
feat/platform-wallet-storage-rehydrationUh oh!
There was an error while loading. Please reload this page.
TL;DR
Adds a write-path guard in
rs-platform-wallet-storagethat rejects a second identity claimingan already-occupied
(wallet_id, identity_index)slot — atstore()time and again as aflush-time backstop — with no schema change and no migration.
User story
As an operator of a wallet built on
rs-platform-wallet-storage, I need(wallet_id, identity_index)to always identify exactly one identity, because that pair drives HDderivation-path lookups: a silent duplicate would let two different identities resolve to the
same derived key material.
Scenario
Walready has identityAstored atidentity_index = 3.store()identityBalso atidentity_index = 3for walletW.Wnow has two identities claiming index 3,and downstream derivation-path logic has no way to know which one is authoritative.
store()returnsWalletStorageError::IdentityIndexConflictimmediately.The same probe also runs as a backstop inside the flush transaction, closing the
FlushMode::Manualwindow where two individually-validstore()calls could still merge intoan on-disk duplicate before either lands.
Detailed discussion
Design. Implements the locked design from memcan TODO
4fbe235c-1bcd-4a32-baea-5b4f9d67e526(R1–R15, domain invariants I1–I7). Deliberately no defence-in-depth — the probe is the only
enforcement, no schema constraint, no migration — so every writer of
identitiesrows had to bere-derived and checked by hand, not just the obvious
store()path.Core pieces:
check_index_conflicts(&Connection/&Transaction, &WalletId, &IdentityChangeSet)— callableagainst either a bare connection or a transaction, so the same probe covers both the store-time
check and the flush-time backstop.
WalletStorageErrorvariants,IdentityIndexConflictandWalletlessIdentityIndex(both non-transient,
Constraint-kind), with explicit arms in every wildcard-free match tablein the crate.
Buffer::store_checked(wallet_id, cs, check)— runs the probe and the merge as one criticalsection under the buffer lock.
SqlitePersister::store()builds a merged view (buffered ∪ incoming, via the realMergeimpl) and checks that before accepting a write.
flush_inner()now holds the connection across take-for-flush and the write itself, closingthe drain-to-commit window.
A race the fix itself introduced, and how it was closed. The first version of the store-time
probe ran under the connection lock while the buffer merge ran under a separate lock — two
threads racing
store()on the same wallet with conflicting indices could both pass thedisk-state probe, both merge into one contradictory buffered changeset, and have the flush-time
backstop's fatal-drop path silently discard whichever caller's entry didn't "win," even though
that caller had already received
Ok(()). Caught by adversarial QA(
tests/sqlite_identity_index_concurrency.rs, ~1-in-several-hundred repro window) and closed bymaking probe+merge share one critical section per wallet (above), verified both by construction
(lock order re-derived independently twice:
conn → buffer, one direction,Bufferneverreaches for
conn) and by stress testing (500+ iterations, three-way races, store-vs-deleteraces).
delete_wallet's error-tolerance carve-out. The flush-time backstop's fatal-drop path meanta wallet with a contradictory buffered changeset could become permanently undeletable. The fix
added a carve-out — originally matching the whole
Constrainterror-kind bucket, which QA showedwould silently swallow unrelated corruption (any FK/CHECK/UNIQUE/NOT-NULL violation) at delete
time too. Narrowed to match exactly the two new variants.
QA. Two full adversarial passes plus a focused re-verification pass on the new locking
machinery specifically, all independently ledger-verified (test logs, clippy,
git diff-treescope checks) rather than trusted from agent reports. Zero outstanding findings.
Out of scope, logged separately:
IdentityChangeSet::merge'sand_modifybranch (in thesibling
rs-platform-walletcrate) doesn't copyidentity_indexon a reindex-while-bufferedmerge — confirmed real, confirmed not to create a false negative in this guard, filed as memcan
TODO
e97aa67c-7ee0-4d41-a7bb-f4e39c695164(low priority) rather than fixed inline, to keep thisPR's diff scoped to
rs-platform-wallet-storage.Testing:
cargo test -p platform-wallet-storage(new tests insqlite_identity_index_uniqueness.rs,sqlite_identity_index_concurrency.rs,sqlite_delete_wallet_constraint_carveout.rs, plus updates to the two wildcard-freeerror-classification test files),
cargo clippy -p platform-wallet-storage --all-targets,cargo fmt.