store: fix restore atomicity, swapInTempDB close error, single-pass CRC32 - #405
Conversation
…pdate format comments Co-authored-by: bootjp <1306365+bootjp@users.noreply.github.com> Agent-Logs-Url: https://github.com/bootjp/elastickv/sessions/4edb95f7-2270-4eda-9b31-cd1d7fac0ddf
| // It writes all entries into a temporary sibling directory and only swaps the | ||
| // temp directory into s.dir after a full successful read, preserving the | ||
| // existing DB on any failure. | ||
| func (s *pebbleStore) restorePebbleNativeAtomic(r io.Reader) error { |
There was a problem hiding this comment.
🚫 [golangci]reported by reviewdog 🐶
calculated cyclomatic complexity for function restorePebbleNativeAtomic is 11, max is 10 (cyclop)
| // read the spool file directly without needing a LimitReader. Entries are | ||
| // written into a temporary Pebble directory and only swapped into place after | ||
| // decoding succeeds, preserving the existing store on failure. | ||
| func (s *pebbleStore) restoreFromLegacyGob(r io.Reader) error { |
There was a problem hiding this comment.
🚫 [golangci]reported by reviewdog 🐶
calculated cyclomatic complexity for function restoreFromLegacyGob is 13, max is 10 (cyclop)
bootjp
commented
Mar 22, 2026
@copilot fix golangci-lint |
Co-authored-by: bootjp <1306365+bootjp@users.noreply.github.com> Agent-Logs-Url: https://github.com/bootjp/elastickv/sessions/9c40de5f-6d02-45a5-a99f-9be3f8234f2c
Fixed in e1dbc8a. The |
| @@ -662,11 +663,18 @@ func (s *pebbleStore) restoreOneEntry(r io.Reader, batch *pebble.Batch) (bool, e | |||
| } | |||
| func (s *pebbleStore) restoreBatchLoop(r io.Reader) error { | |||
Uh oh!
There was an error while loading. Please reload this page.
Several correctness and durability issues in the Pebble FSM restore paths identified in review. Key problems: native Pebble restore wiped the existing DB before validating the incoming snapshot;
swapInTempDBsilently ignoreddb.Close()errors; legacy gob restore read the payload twice (spool then re-read for CRC verify); and the format history comment was misleading about pre-magic Pebble snapshots.Changes
Native Pebble restore — atomic temp-dir swap
restorePebbleNativeAtomic: writes entries into a unique temp dir (os.MkdirTemp), persistslastCommitTS, then swaps into place viaswapInTempDB— existing DB is preserved on any failure.restoreBatchLoopInto(r io.Reader, db *pebble.DB)as a standalone function so both the direct and atomic restore paths share the same batch loop without touchings.db.Restorenow dispatches native Pebble snapshots to the atomic variant instead of callingreopenFreshDBup front.swapInTempDB— check Close errorrestoreFromLegacyGob— single-pass CRC32Replaced the two-pass approach (spool-all then seek-back-verify) with a rolling
checksumSize-byte tail buffer: the CRC32 is computed incrementally during the spool, the 4-byte trailer never reaches the spool file, and the decoder reads the file once. Removed the now-deadverifyCRC32Trailerhelper. AddedspoolBufSize = 32*1024constant.Comments / backward compat clarity
restorePebbleNativedoc comment: thepebbleSnapshotMagicheader was present from day one; there are no pre-magic Pebble snapshots.Restoredefault branch clarifying that unknown streams fall to legacy gob with a clear decode error.Test coverage
Added
TestPebbleStore_Restore_NativePebbleAtomicwhich truncates a valid snapshot mid-stream and asserts the original DB contents remain accessible after the failed restore.📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.