Add live point-in-time logical backup producer

[bootjp]

Summary

• add the live point-in-time logical backup producer and CLI over the backup control-plane RPCs
• renew the distributed pin while streaming, stop renewal before publishing MANIFEST.json, and always end the backup session
• encode DynamoDB, S3, Redis, and SQS scopes with checksums and expected-key validation
• add crash, renewal-race, cross-adapter, and TTL-expiry acceptance coverage
• add the operator backup/restore runbook and promote the Phase 1 design document to Implemented

Stack

• base: Add live logical backup control plane #1119
• pin substrate prerequisite: Add live backup pin FSM substrate #1056

Risk

The data path is opt-in through the new backup CLI and admin RPCs. The main correctness risks are publishing a completion marker after pin loss and reading from a changing route catalog; both are covered by pinned-route and renewal-quiescence tests.

Verification

• go test ./internal/backup ./cmd/elastickv-backup -count=1
• go test -race ./internal/backup -run 'Test(RunLiveBackup|ProducerCrash|CrossAdapterConsistency|ExpectedKeysBaseline)' -count=1
• go test ./adapter -run 'Test(BeginBackup|RenewBackup|StreamBackup|ListBackup|BackupProtocol|InternalForwardAdminProposal)' -count=1
• go test ./kv -run 'Test(Backup|BeginBackup|Pin|BackupScanner|ShardStoreScanAt_RoutesFilesystem)' -count=1
• make -C proto gen followed by a clean generated diff
• golangci-lint run ./... --timeout=15m --allow-parallel-runners -j 2
• git diff --check
• no references remain to docs/design/2026_04_29_proposed_logical_backup.md

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c40d109a-4524-4fcd-a5f3-86745d5fad92

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[bootjp]

Current signed HEAD is 2d468c5a169c93696b0037d66c644b5378685898.

Verification:

• producer and CLI tests pass
• focused producer acceptance tests pass under -race
• control-plane adapter and backup scanner tests pass
• proto breaking/generation checks pass with no generated diff
• full lint reports 0 issues
• git diff --check passes
• the former proposed design path has no remaining references

Please review the latest head.

@codex review
@coderabbitai review

[coderabbitai]

@bootjp: Reviewing the latest signed HEAD (2d468c5a169c93696b0037d66c644b5378685898) now.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[gemini-code-assist]

Code Review

This pull request implements Phase 1 of the logical backup feature, enabling point-in-time consistent logical backups from a running elastickv cluster. It introduces the elastickv-backup CLI tool, adds server-side configurations for active backup pins, implements the live backup producer lifecycle (including streaming and renewal), and updates design documents and runbooks. The review feedback highlights a potential nil pointer dereference on Windows when loading system certificates, portability issues with using os.Link for manifest publication on filesystems that do not support hard links, and noisy warning logs during graceful shutdown due to unsuppressed context cancellation errors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[bootjp]

Updated the latest head d5a5934945c060183a6409fd0cc81c2010a7672b for all three findings:

• fall back to a new certificate pool when SystemCertPool returns either an error or a nil pool
• publish the final manifest with same-directory os.Rename, while preserving the pre-existing marker guard
• keep graceful cancellation warning-free and retain warnings for genuine renewal failures

Verification:

• go test ./internal/backup ./cmd/elastickv-backup -count=1
• go test -race ./internal/backup -run 'Test(RunLiveBackup|ProducerCrash|CrossAdapterConsistency|ExpectedKeysBaseline)' -count=1
• GOOS=windows GOARCH=amd64 go test -c -o /tmp/elastickv-backup-windows.test.exe ./cmd/elastickv-backup
• golangci-lint run ./internal/backup/... ./cmd/elastickv-backup/... --timeout=5m

Caller audit for the manifest error-semantics change found one production caller (RunLiveBackup), which propagates finalization errors, leaves no manifest, and still executes EndBackup.

@codex review

[bootjp]

Current head d5a5934 is fully green and all prior inline findings are resolved. @codex review

[bootjp]

History-only author-compliance repair completed at 83f3a9f6aa23fd632aba203228284ddda27c4d53.

• final tree is byte-identical to prior head d5a5934945c060183a6409fd0cc81c2010a7672b
• the PR now exposes one linear commit authored and committed by bootjp <contact@bootjp.me>
• GitHub reports the commit signature as verified/valid
• parent is sanitized Add live logical backup control plane #1119 head 6a85349d2deec210799349b420d35c9563666e55

No runtime or source-tree semantics changed. CI is running on the reconstructed head.

@codex review

[bootjp]

Current reconstructed head 83f3a9f6aa23fd632aba203228284ddda27c4d53 is fully green. PR-visible authorship and signature verification pass, the final tree remains identical to the pre-rewrite head, and there are no current-head root findings. Please complete the latest-head review. @codex review

[bootjp]

Latest signed head 83f3a9f6aa23fd632aba203228284ddda27c4d53 remains mergeable with all required checks green and no current-head root findings. @codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:83f3a9f6aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bootjp]

Current signed HEAD 56277d2cbc957487a967e03263167fb3cb5ed38f addresses all five current-head findings.

Evidence:

• live producer modes unsupported by the shipped native restore path fail before dial and at the library entrypoint
• TLS server-name-only and CA plus skip-verify combinations fail closed
• non-empty one/two-record scopes require at least one returned record
• server baseline and stream now exclude decoder-dropped S3 generation/incomplete-upload and SQS side rows; producer counts only after decoder acceptance
• runbook and implemented design no longer recommend unrestorable modes

Verification:

• go test ./cmd/elastickv-backup ./internal/backup ./adapter -run 'Test(ParseFlags|LoadTransportCredentials|ScopeForKey|RunLiveBackup|ExpectedKeysBaseline|ValidateLiveBackupRestoreCompatibility|Backup|StreamBackup|ListBackupScopes)' -count=1 -timeout=300s
• focused -race tests for the backup CLI and producer
• golangci-lint run ./cmd/elastickv-backup ./internal/backup ./adapter --timeout=5m --allow-parallel-runners reports 0 issues
• git diff --check

Caller audit for the ScopeForKey semantic change found the baseline scan, server stream, producer stream validator, and tests; baseline and stream now share the same retained-record classification. CI is running for this head.

@codex review

[chatgpt-codex-connector]

💡 Codex Review

elastickv/internal/backup/live.go

Lines 66 to 67 in 56277d2

	casebytes.HasPrefix(key, []byte(DDBTableGenPrefix)):
	returnddbScopeFromDirectSegment(key, DDBTableGenPrefix)

Exclude DynamoDB generation counters from scopes

When a DynamoDB table has been deleted, the live adapter leaves the !ddb|meta|gen| counter behind while the table schema is gone, and the decoder's HandleTableGen is a no-op. Classifying that counter as a scoped table makes BeginBackup/ListAdaptersAndScopes select a deleted table and lets the live manifest publish dynamodb.tables for a scope that has no _schema.json; the native restore path then emits no table even though the manifest says it was dumped. Treat this like the other generation counters that are already unscoped, or only count it when the schema record is present.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bootjp]

Latest signed HEAD 7d4834f1211074152a9238e98dc23d11c31ad358 addresses all current-head findings.

• DynamoDB generation counters are now unscoped, so deleted-table counters cannot create manifest-only tables.
• Each pin renewal is bounded by TTL/3 and the remaining expiry; deadline failure cancels the dump and blocks manifest publication.
• Default all-scope selection includes every non-empty Begin baseline scope even when the later List response omits it.
• S3 validation now counts only records retained after finalize: bucket metadata, active manifests, and manifest-selected chunks.

Verification:

• go test ./internal/backup ./cmd/elastickv-backup -count=1 -timeout=300s
• focused adapter backup tests
• focused producer tests under -race
• golangci-lint run ./internal/backup ./cmd/elastickv-backup ./adapter --timeout=5m --allow-parallel-runners reports 0 issues
• git diff --check

Caller audit for ScopeForKey covers the server baseline, server stream, producer validator, and tests; all now exclude the same generation counter.

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:7d4834f121

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bootjp]

Evidence for current HEAD ed0498de2d24eeabd7b07c25a637cfb2d8438841:

• Finalized retained counts now cover SQS, DynamoDB, Redis, and S3.
• Deleted SQS generation-only state, schema-less DynamoDB items, Redis orphan TTLs, and Redis stream entries without metadata cannot mask a short stream or publish a misleading manifest.
• Redis derivable delta, claim, and score indexes are excluded consistently from Begin baseline and Stream filtering.
• Caller audit covered all ScopeForKey, FinalizedScopeCounts, and retained-count production callers.
• All three current review threads were replied to and resolved.

Validation:

• go test ./internal/backup ./cmd/elastickv-backup -count=1 -timeout=300s
• focused go test -race ./internal/backup
• golangci-lint run ./internal/backup ./cmd/elastickv-backup --timeout=5m --allow-parallel-runners (0 issues)
• signed author and committer: bootjp <contact@bootjp.me>

Requesting a fresh latest-head review.

@codex review

[bootjp]

Rebase evidence for current HEAD 53e64025e8efaa147b7759cfb2d2ad434dfb9a64:

• Rebased onto Add live logical backup control plane #1119 HEAD f30bd74b5e64064d6a846ac640cf35d639a9d03b.
• git range-diff reports all four producer commits unchanged (=).
• All four PR-visible commits retain author and committer bootjp <contact@bootjp.me> and GitHub signature verification is verified: true.
• The retained-count findings remain fixed in rebased commit 53e64025.

Post-rebase validation:

• go test ./internal/backup ./cmd/elastickv-backup ./kv . -count=1 -timeout=600s
• go test ./adapter -run 'Backup|ForwardLeaseRead|ForwardAdminProposal' -count=1 -timeout=300s
• focused producer race tests
• golangci-lint run ./internal/backup ./cmd/elastickv-backup ./adapter ./kv . --timeout=5m --allow-parallel-runners (0 issues)
• git diff --check

The broad unfiltered adapter suite reached its 600-second timeout in Raft integration cleanup; the focused backup/forwarding adapter suite passed and GitHub CI is now running on this head.

Requesting a fresh latest-head review after the dependency rebase.

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:53e64025e8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bootjp]

Rebase evidence for current HEAD 7f11f84:

• Rebased onto Add live logical backup control plane #1119 HEAD dd9f499.
• Conflict resolution kept both the Add live logical backup control plane #1119 live-backup safety fixes and Add live point-in-time logical backup producer #1128 producer restore-compatibility fixes.
• The default all-scope path includes non-empty BeginBackup baseline scopes in addition to ListAdaptersAndScopes results, so baseline scopes omitted by List are not dropped.
• All rebased commits are signed by bootjp contact@bootjp.me.

Validation:

• go test ./internal/backup -run 'Test(RunLiveBackup|ExpectedKeysBaseline|ScopeForKey|LiveDecoder|Finalize|SQS|DynamoDB|Redis|S3)' -count=1\n- go test ./cmd/elastickv-backup ./adapter ./kv -run 'Test(StreamBackup|ListBackupScopes|RenewBackup|SnapshotBackupGroups|BeginBackup|FSMSnapshotRejectsActiveBackupPin|ActiveTimestampTrackerOldestBackupForGroup)' -count=1\n- golangci-lint run ./cmd/elastickv-backup ./adapter ./kv ./internal/backup ./cmd/server --timeout=5m\n- go test ./... -run '^$'\n\n@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:7f11f84972

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bootjp]

Latest signed HEAD 688c1b28a4196f01908575932e9f26dfeb0fb2cf addresses the current-head findings.

• BeginBackup/ListAdaptersAndScopes now build expected counts from retained key/value decoder state, so S3, DynamoDB, SQS, and Redis cleanup leftovers use the same denominator as final validation.
• StreamBackup now checks the renewed server-side session deadline while streaming, so an original stream request token can continue after its embedded initial deadline once the pin has been renewed.
• Explicit --scope requests can use non-empty baseline scopes even when ListAdaptersAndScopes omits them.
• Scoped manifests now include only adapters selected by the requested scopes.
• The branch was rebased onto base 6608a1cd8f4e25c10b869c6d131ac6d17bf020ce; local merge-tree against the base is clean.

Validation:

• go test ./internal/backup ./adapter -run 'Test(LiveScopeCounterUsesRetainedCounts|RunLiveBackupAcceptsRequestedBaselineOnlyScope|RunLiveBackupScopedManifestOmitsUnselectedAdapters|BeginBackupExpectedKeysUseRetainedCounts|StreamBackupHonorsRenewedSessionDeadline|StreamBackupStopsWhenTokenExpiresMidStream|ListBackupScopesReportsScannerCloseError)' -count=1 -timeout=300s
• go test ./cmd/elastickv-backup ./internal/backup ./adapter ./kv -run 'Test(StreamBackup|ListBackupScopes|RenewBackup|SnapshotBackupGroups|BeginBackup|RunLiveBackup|ExpectedKeysBaseline|ScopeForKey|LiveDecoder|Finalize|SQS|DynamoDB|Redis|S3)' -count=1 -timeout=600s
• go test ./... -run '^$' -count=1 -timeout=600s
• golangci-lint run ./cmd/elastickv-backup ./adapter ./kv ./internal/backup ./cmd/server --timeout=5m --allow-parallel-runners
• git diff --check

All visible commits are signed by bootjp <contact@bootjp.me>. CI is running for this head.

@codex review

[bootjp]

Latest signed HEAD 7c134ba4123ec8782f6d9b6b7bb6cc48fd90e29d supersedes the previous update.

The prior CI test (ubuntu-latest) failure was the token-rotation regression test still expecting StreamBackup to reject an old embedded token deadline. That expectation is now aligned with the server-side session-deadline behavior: Renew/List still reject the old token, while StreamBackup uses the renewed session deadline.

Validation after this update:

• go test -race ./adapter -run 'TestBackupTokenDeadlineRotatesAndFailsClosed|TestStreamBackupHonorsRenewedSessionDeadline' -count=1 -timeout=300s
• go test ./internal/backup ./adapter -run 'Test(LiveScopeCounterUsesRetainedCounts|RunLiveBackupAcceptsRequestedBaselineOnlyScope|RunLiveBackupScopedManifestOmitsUnselectedAdapters|BeginBackupExpectedKeysUseRetainedCounts|BackupTokenDeadlineRotatesAndFailsClosed|StreamBackupHonorsRenewedSessionDeadline|StreamBackupStopsWhenTokenExpiresMidStream|ListBackupScopesReportsScannerCloseError)' -count=1 -timeout=300s
• go test ./cmd/elastickv-backup ./internal/backup ./adapter ./kv -run 'Test(StreamBackup|ListBackupScopes|RenewBackup|SnapshotBackupGroups|BeginBackup|RunLiveBackup|ExpectedKeysBaseline|ScopeForKey|LiveDecoder|Finalize|SQS|DynamoDB|Redis|S3)' -count=1 -timeout=600s
• go test ./... -run '^$' -count=1 -timeout=600s
• golangci-lint run ./cmd/elastickv-backup ./adapter ./kv ./internal/backup ./cmd/server --timeout=5m --allow-parallel-runners
• git diff --check

Current-head unresolved review threads are 0. CI is running again for this head.

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit:7c134ba412

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bootjp]

Latest signed HEAD 8f76d8c373b09bbdaf748103eb5548ebee6d8598 is rebased onto base b3b8ab9f7fb2d4c161d4226bcafe75e45c977b1d.

The rebase conflict was limited to adapter/admin_backup_test.go; the resolution keeps the base scope-filter recording and the retained-count key/value scanner test support together. Local merge-tree against the latest base is clean.

Validation after this rebase:

• go test ./internal/backup ./adapter -run 'Test(LiveScopeCounterUsesRetainedCounts|RunLiveBackupAcceptsRequestedBaselineOnlyScope|RunLiveBackupScopedManifestOmitsUnselectedAdapters|BeginBackupExpectedKeysUseRetainedCounts|BackupTokenDeadlineRotatesAndFailsClosed|StreamBackupHonorsRenewedSessionDeadline|StreamBackupStopsWhenTokenExpiresMidStream|StreamBackupUsesPinTimestampAndScopeFilter|ListBackupScopesReportsScannerCloseError)' -count=1 -timeout=300s
• go test -race ./adapter -run 'Test(BackupTokenDeadlineRotatesAndFailsClosed|StreamBackupHonorsRenewedSessionDeadline)' -count=1 -timeout=300s
• go test ./... -run '^$' -count=1 -timeout=600s
• golangci-lint run ./cmd/elastickv-backup ./adapter ./kv ./internal/backup ./cmd/server --timeout=5m --allow-parallel-runners
• git diff --check

Current-head unresolved review threads are 0. CI is running for this head.

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:8f76d8c373

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bootjp]

Latest signed HEAD db7ec7d0ce9143ff9211e1f8e0e4a46066732324 addresses the three latest-head findings and is rebased onto base d91744b04be1ce78e16f49a0e488d31f0b340cbc.

• Default no---scope manifests preserve enabled empty adapters while explicit scoped dumps still omit unselected adapters.
• BeginBackup now carries adapter/scope selection into the baseline count, so unselected adapters are skipped before decoding.
• Baseline counting uses a key-only pass plus metadata-only value reads; large blob/message payloads are not materialized for the preflight count.
• The rebase conflict resolution preserves live-session checks during ListAdaptersAndScopes and StreamBackup.

Validation:

• make -C proto gen
• go test ./internal/backup ./adapter -run 'Test(LiveScopeCounterUsesRetainedCounts|RunLiveBackupAcceptsRequestedBaselineOnlyScope|RunLiveBackupScopedManifestOmitsUnselectedAdapters|RunLiveBackupDefaultManifestPreservesEnabledEmptyAdapters|BeginBackupExpectedKeysUseRetainedCounts|BeginBackupExpectedKeysAvoidMaterializingBlobValues|BeginBackupBaselineSkipsUnselectedAdapterValues|ListBackupScopesFailsWhenSessionEndsDuringScan|StreamBackupFailsClosedWithoutPinnedRouteSnapshot|BackupTokenDeadlineRotatesAndFailsClosed|StreamBackupHonorsRenewedSessionDeadline|StreamBackupStopsWhenTokenExpiresMidStream|StreamBackupUsesPinTimestampAndScopeFilter|ListBackupScopesReportsScannerCloseError)' -count=1 -timeout=300s
• go test ./cmd/elastickv-backup ./internal/backup ./adapter ./kv -run 'Test(StreamBackup|ListBackupScopes|RenewBackup|SnapshotBackupGroups|BeginBackup|RunLiveBackup|ExpectedKeysBaseline|ScopeForKey|LiveDecoder|Finalize|SQS|DynamoDB|Redis|S3|ValidateBackupSnapshotAt|BackupScanners|FilteredBackupScanner|BackupTokenDeadline|ActiveTimestampTrackerOldestBackupForGroup)' -count=1 -timeout=600s
• go test ./... -run '^$' -count=1 -timeout=600s
• golangci-lint run ./cmd/elastickv-backup ./adapter ./kv ./internal/backup ./cmd/server --timeout=5m --allow-parallel-runners
• git diff --check

Caller audit covered BeginBackupRequest, scanBackupScopeCounts, NewLiveScopeCounter, liveManifestAdapters, and requireLiveBackupSession production call sites.

Current-head unresolved review threads are 0. CI is running for this head.

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:db7ec7d0ce

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Skip adapters excluded by explicit scopes

When a dump is narrowed only by Scopes (the CLI default still sends all adapters), this check treats every adapter as enabled and then calls ScopeForKey on keys from adapters that have no requested scope. In a Redis-only scoped dump, for example, a malformed or future-format S3/DynamoDB key can still abort BeginBackup even though that adapter will be omitted; derive/consult the adapter set present in selection.scopes before parsing the scoped key.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Avoid materializing Redis payloads in baseline counts

For Redis scopes this returns true for every Redis key, so the baseline metadata pass materializes user payload values such as large strings, hash/list values, and stream entries. On Redis-heavy backups, BeginBackup can therefore load a page of large payloads just to compute expected counts, defeating the key-only/metadata-only preflight path; restrict value reads to Redis metadata/TTL records that are actually needed and count payload-bearing keys from their keys.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Thread adapter filters into scope listing

After BeginBackup honors an --adapter redis selection, the producer still calls ListAdaptersAndScopes, and this server path scans every adapter with AllAdapters(). That means a disabled adapter can still abort an otherwise valid filtered dump (for example a malformed DynamoDB/S3 key fails classification during this list pass), even though its values were intentionally excluded from the baseline and will not be streamed; carry the requested adapter/scope selection into the listing RPC or avoid the full-adapter scan for filtered runs.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Gate scoped BeginBackup fields by protocol version

These new request fields are silently ignored by older live-backup servers because protobuf preserves forward compatibility, while backup_protocol_version still advertises v1. During a rolling upgrade or when the CLI contacts an old admin endpoint, an explicit scoped dump can receive an unfiltered baseline and then select extra baseline scopes (especially with the default all-adapter setting), producing a broader dump than requested; bump/gate the client-server backup protocol before relying on adapters and scopes.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Fail when baseline metadata keys disappear

When a key-only baseline pass records metadata keys and one of those keys is missing from this second value pass, the loop simply finishes and finalizes a lower retained count because observed keys are never tracked as consumed. If a backup pin is lost or compaction races between the key pass and the metadata pass, this can turn a missing table/bucket/queue metadata record into an omitted baseline instead of the expected shortfall failure; remove each consumed metadata key and fail if any remain after the scan.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[bootjp]

Codex レビューの 5 件のうち 4 件を修正しました（24bff020）。

対応済み

P2 adapter/admin_backup.go:748 — Skip adapters excluded by explicit scopes
Scopes のみで絞った場合でも CLI は既定で全 adapter を送るため、全 adapter が有効扱いになり、ストリームされない adapter の壊れたキー / 将来フォーマットのキーで BeginBackup 全体が失敗していました。narrowBackupAdaptersToScopes を追加し、要求された scope が名指しする adapter だけに集合を絞ってから ScopeForKey に到達するようにしています。

P2 adapter/admin_backup.go:537 — Thread adapter filters into scope listing
ListAdaptersAndScopes が AllAdapters() で再スキャンしていたため同じ中断が再発していました。backupSession に BeginBackup 時の selection を保持し、リスト側で再利用します。

P2 adapter/admin_backup.go:1250 — Fail when baseline metadata keys disappear
key-only パスが記録したメタデータキーが value パスに現れない場合、黙って少ない retained count で確定していました。pending セットで消費を追跡し、残っていればエラーにします（pin 喪失 / compaction レースが shortfall として顕在化）。フィルタ済みスキャナの述語が metadataKeys を読むため、pending は別コピーにしています。

P2 proto/admin.proto:187 — Gate scoped BeginBackup fields by protocol version
BeginBackupResponse.backup_protocol_version (field 7) を追加し、サーバは backup protocol v2 を広告 + エコーします。プロデューサ側は「narrowed な dump なのにサーバが v2 未満」のとき ErrLiveBackupScope で fail-closed し、pin も後始末します。ピア版ゲートの最小値は v1 のまま据え置きです — フィルタは BeginBackup を処理するノード上でのみ計算されるため、混在バージョンのクラスタでも rolling upgrade を壊しません。

未対応（意図的な先送り）

P2 internal/backup/live.go:360 — Avoid materializing Redis payloads in baseline counts
指摘は妥当ですが、Redis の retained count はエンコーダの value 解析に強く依存しています（HandleString / HandleHLL は値に埋め込まれた TTL エンベロープを読んで inlineTTLOwned を立て、その結果が後続の HandleTTL の orphan-TTL 計上を左右する、など）。安全に外せるのはコレクション要素レコードに限られ、5 種類のエンコーダのハンドラを個別に監査する必要があります。誤ると expected-key ベースラインが静かにズレ、この機能全体の shortfall 検知が壊れるため、本 PR には含めず別 PR で扱います。

Caller audit

意味論を変えた関数の呼び出し元:

• narrowBackupAdaptersToScopes → backupBaselineSelectionFromBeginRequest のみ。
• rememberBackupSession（シグネチャ変更）→ BeginBackup 1 箇所。
• scanBackupScopeMetadata（成功→エラーになり得る fail-closed 化）→ scanBackupScopeCounts のみ。その先は buildExpectedBackupBaseline（BeginBackup）と ListAdaptersAndScopes の 2 経路で、どちらも FailedPrecondition として伝播。
• backupSelectionForToken（新規、セッション欠落時にエラー）→ ListAdaptersAndScopes のみ。同関数は既に backupRouteSnapshotForToken で同条件を弾いています。

Test evidence

• 追加: TestBackupBaselineSelectionNarrowsAdaptersToScopes (table-driven 5 ケース), TestBeginBackupScopedDumpIgnoresUnrequestedAdapterKeys, TestBeginBackupAdvertisesScopedBaselineProtocol, TestListAdaptersAndScopesReusesBeginSelection, TestBeginBackupFailsWhenBaselineMetadataKeyDisappears, TestRunLiveBackupRejectsScopedDumpOnLegacyServer (table-driven 4 ケース)
• go test -race -run Backup ./adapter/ ./internal/backup/ ./cmd/elastickv-backup/ → ok
• go test -race ./internal/backup/ ./cmd/elastickv-backup/ → ok
• go test -race -run 'Backup|Admin' ./adapter/ → ok
• golangci-lint run adapter/... internal/backup/... → 0 issues

Self-review

1. データ損失: メタデータキー消失を fail-closed 化したので、静かに欠けたバックアップが公開される経路が 1 つ減りました。adapter 絞り込みは要求されていない adapter を数えないだけで、ストリーム対象は変わりません。
2. 並行性 / 分散障害: pending はフィルタ済みスキャナの述語が読む metadataKeys とは別マップにしてスキャン中の変更を回避。backupSelectionForToken は既存の backupStateMu 配下で、reapBackupSessionsLocked の呼び出し順も既存メソッドに合わせています。
3. パフォーマンス: scope 指定時のベースラインスキャンは対象 adapter が減る分だけ軽くなります。Redis ペイロードの materialize は上記のとおり未対応。
4. データ整合性: プロトコルバージョンのエコーにより、フィルタが無視されたまま広いダンプを掴む経路を塞ぎました。expected_keys と実ストリームの scope 集合が食い違わなくなります。
5. テスト網羅: 新規分岐（scope のみ / adapter のみ / 交差 / 全 adapter / 未知 adapter / v1 サーバ / v2 サーバ / メタデータ欠落）をすべてカバー。

@claude review

[claude]

Claude encountered an error after 7s —— View job

---

I'll analyze this and get back to you.

[bootjp]

前回の @claude review は 7 秒でエラー終了していたため再依頼します。

@claude review

[claude]

Claude finished @bootjp's task in 4m 9s —— View job

---

Claude review

• Gather context (PR diff, prior review threads, CLAUDE.md conventions)
• Review current HEAD (24bff02) diff for correctness/data-loss/concurrency/consistency issues
• Check outstanding Codex findings for whether they're actually resolved at current HEAD
• Post final review

Scope of this pass

I re-reviewed the branch head at 24bff02 ("backup: honor scoped baselines end to end"), which is the fix for the last Codex round's 4 P2 findings (adapter-narrowing abort, ListAdaptersAndScopes rescanning all adapters, silent baseline shortfall on metadata-key disappearance, and a v1 server silently dropping adapters/scopes). I traced each fix end-to-end against adapter/admin_backup.go, internal/backup/live_producer.go, internal/backup/live.go, proto/admin.proto, and the CLI defaults in cmd/elastickv-backup/main.go, and cross-checked the new tests.

Note: I could not run go build/go test/golangci-lint in this sandboxed job — every Bash invocation beyond a handful of pre-approved read-only git commands (status/log/diff/show) required interactive approval that wasn't available here. This review is based on static reading of the diff and existing tests, not a fresh test run. If you want CI-equivalent confidence, please rely on the go test/golangci-lint runs already reported in the PR thread.

Findings

No new correctness issues. Specifically:

• narrowBackupAdaptersToScopes (adapter/admin_backup.go:735) only narrows when scopes is non-empty, and backupBaselineSelectionFromBeginRequest already validates every scope's adapter is enabled before narrowing, so the result can't collapse to an unintended empty AdapterSet. Matches the 5 table-driven cases in TestBackupBaselineSelectionNarrowsAdaptersToScopes.
• ListAdaptersAndScopes now reuses session.selection via backupSelectionForToken instead of AllAdapters() (adapter/admin_backup.go:550), so a redis-only BeginBackup no longer re-triggers a full-adapter rescan (and the abort that comes with it) on ListAdaptersAndScopes. Confirmed by TestListAdaptersAndScopesReusesBeginSelection.
• The metadata-pass shortfall fix tracks pending as a copy of metadataKeys (kept separate specifically because the filtered scanner's predicate reads metadataKeys concurrently with the scan) and fails closed if any key never reappears in the value pass — turning a silent narrower baseline into an explicit FailedPrecondition. Confirmed by TestBeginBackupFailsWhenBaselineMetadataKeyDisappears.
• The protocol-version gate (backupProtocolVersionV2 echoed in BeginBackupResponse.backup_protocol_version, field 7 — no wire collision) is consumed by requireScopedBaselineSupport/beginBackupRequestIsScoped in the producer. I checked the default CLI path (--adapter defaults to all 4 names, no --scope) still resolves to "not scoped" so it stays compatible with a v1 peer, while any narrowing via --adapter or --scope correctly requires v2 and fails closed (with pin cleanup via cleanupInvalidBeginResponse) otherwise. The peer-version handshake in gateBackupPeerVersions intentionally keeps its v1 floor, so a mixed-version cluster doesn't get blocked cluster-wide by this. Covered by TestRunLiveBackupRejectsScopedDumpOnLegacyServer.
• liveManifestAdapters's new includeEnabledEmpty parameter (internal/backup/live_producer.go:637, driven by len(opts.Scopes) == 0) correctly distinguishes "no explicit --scope filter" (keep empty-but-enabled adapters in the manifest) from "explicit --scope filter" (only include adapters with selected scopes) — this was the remaining unresolved P2 from the previous Codex round on db7ec7d/8f76d8c373.

Deferred item, correctly disclosed:liveScopeCounterNeedsValue (internal/backup/live.go:350) still materializes every Redis value during baseline scans, not just metadata-bearing keys — the one Codex P2 finding the author explicitly chose not to fix in this PR, with a reasoned explanation (Redis retained-count logic is entangled with TTL-envelope parsing across 5 encoder handlers; a partial fix risks silently desyncing the expected-key baseline). That tradeoff looks sound to me — a performance-only issue traded for correctness safety, and it's called out for follow-up rather than silently left in.

Minor, non-blocking observations

• finalizedScopeCounts (internal/backup/live.go:364) has a dead branch: adapterFinalizesScopeCounts returns true for all four adapter names ScopeForKey can ever produce, so the count > 0 && !adapterFinalizesScopeCounts(...) condition inside the loop is always false and nothing from streamed is ever copied into out there — every count actually comes from d.addFinalizedScopeCounts. Not a bug (behavior is correct), just a vestigial branch that could be simplified in a later cleanup.
• beginBackupRequestIsScoped's allBackupAdapterCount = 4 is a hardcoded adapter count; if a fifth logical-backup adapter is ever added this comparison needs updating in lockstep with AllAdapters()/liveBackupAdapterNames. Low risk today, worth a one-line comment or a shared constant if adapters get added again.

Nothing here blocks merge from my read of the current head. The remaining open thread (Redis baseline value materialization) is already tracked by the author as a follow-up, not a silent gap.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[bootjp]

レビューの minor 指摘のうち、こちらの差分に由来する 1 件を修正しました（6fa53039）。

allBackupAdapterCount = 4 のハードコード — ご指摘のとおり AllAdapters() / liveBackupAdapterNames と歩調を合わせる必要がありました。定数をやめて len(liveBackupAdapterNames(AllAdapters())) から導出するようにし、TestAllBackupAdapterCountTracksAllAdapters で「全 adapter を列挙したリクエストが scoped と誤判定されないこと」も含めて固定しています。5 つ目の adapter が追加されても、未フィルタのダンプが誤って narrowed 扱いになりサーバに v2 を要求してしまう事故は起きません。

finalizedScopeCounts の dead branch (internal/backup/live.go:364) — 分析に同意します（adapterFinalizesScopeCounts は ScopeForKey が返し得る 4 つの adapter 名すべてで true なので、streamed からのコピーは実際には発生せず、カウントは全て addFinalizedScopeCounts 由来）。ただしこれは本 PR の差分ではない既存コードで、挙動も正しいため、Redis ペイロード materialize の follow-up PR と併せて整理します（差分を絞るため本 PR には含めません）。

Test evidence

• go test -race ./internal/backup/ ./cmd/elastickv-backup/ → ok
• golangci-lint run internal/backup/... → 0 issues

@claude review

[claude]

Claude finished @bootjp's task in 1m 11s —— View job

---

Claude review

• Gather context (latest diff at 6fa5303, prior review threads)
• Review the allBackupAdapterCount fix in internal/backup/live_producer.go
• Confirm the deferred finalizedScopeCounts dead-branch item is correctly left out of scope
• Check for any other issues introduced by this commit
• Post final review

Scope of this pass

Reviewed the single commit 6fa5303 ("backup: derive the all-adapter count from AllAdapters"), which addresses the last remaining minor from the previous Claude pass on 24bff02.

Note: I could not run go test/golangci-lint in this sandboxed job — Bash invocations beyond a handful of pre-approved read-only commands require interactive approval that wasn't available here. This review is based on static reading of the diff, not a fresh test run.

Findings

No issues. The fix is correct and minimal:

• internal/backup/live_producer.go:29-31 — allBackupAdapterCount is now var allBackupAdapterCount = len(liveBackupAdapterNames(AllAdapters())) instead of a hardcoded 4. AllAdapters() (internal/backup/decode.go:40) and liveBackupAdapterNames (internal/backup/live_producer.go:508) have no dependency on other package-level vars, so this initializes safely at package load with no init-order hazard, and nothing mutates it afterward — effectively a computed constant.
• beginBackupRequestIsScoped (internal/backup/live_producer.go:210-215) now compares against the derived count, so a request naming every currently-known adapter is still correctly classified as unscoped and won't spuriously demand backup protocol v2 from the server. TestAllBackupAdapterCountTracksAllAdapters pins exactly this: it asserts the derived value matches len(liveBackupAdapterNames(AllAdapters())) and that a request listing all adapters returns false from beginBackupRequestIsScoped.
• This directly closes the gap flagged in the prior review pass — if a fifth logical-backup adapter is added later, allBackupAdapterCount updates automatically instead of silently drifting.

Deferred item, correctly out of scope: I checked finalizedScopeCounts/adapterFinalizesScopeCounts (internal/backup/live.go:364-382) — confirmed this commit doesn't touch that file at all (git diff 24bff02 6fa5303 --stat shows only live_producer.go and its test). The dead branch (adapterFinalizesScopeCounts returns true for all four adapter names ScopeForKey can produce, so out[scope] = count never executes) is pre-existing, non-buggy, and reasonably bundled with the disclosed Redis-materialization follow-up rather than this PR.

Minor, non-blocking observation

liveBackupAdapterNames's make([]string, 0, 4) //nolint:mnd // four logical backup adapters (internal/backup/live_producer.go:509) still hardcodes 4 as a capacity hint. It's harmless today (append just reallocates once more if a fifth adapter is added — not a correctness issue), but since this commit's whole point was removing a hardcoded adapter count, it'd be a good candidate to fold into the same follow-up cleanup rather than leave a second "4" nearby.

Nothing here blocks merge.