Complete standalone hotspot split automation - #1104
Conversation
Warning Review limit reached
Next review available in:26 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (27)
📝 WalkthroughWalkthrough自動分割の検出器・スケジューラ・ランタイム制御を追加し、KeyViz のホットキー観測、カタログ監視、分配サーバー、管理 gRPC と接続しました。Top-K 隔離、証拠フェンス、Prometheus 指標、E2E テスト、設計文書も更新されています。 ChangesAutosplit 検出とスケジューリング
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Sequence Diagram(s)sequenceDiagram
participant AdminClient
participant AdminServer
participant RuntimeSwitch
participant Scheduler
AdminClient->>AdminServer: SetAutoSplitEnabled(enabled)
AdminServer->>RuntimeSwitch: SetEnabled(enabled)
AdminServer-->>AdminClient: Enabled()
Scheduler->>RuntimeSwitch: KillSwitch()
RuntimeSwitch-->>Scheduler: enabled state
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
bootjp
commented
Jul 18, 2026
@codex review |
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (48d83f0):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
To use Codex here, create a Codex account and connect to github. |
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (8b22ccf):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
bootjp
commented
Jul 18, 2026
@codex review |
To use Codex here, create a Codex account and connect to github. |
There was a problem hiding this comment.
Code Review
This pull request implements the standalone automatic hotspot range split scheduler (autosplit) for same-group splits, integrating it with the existing SplitRange and KeyViz sampler. It introduces a durable SplitAtHLC route lineage field to reconstruct cooldowns on leadership changes, updates the route catalog codec to v2 with backward compatibility, and adds a transaction commit timestamp patching mechanism (CommitTSValueOffset). A critical issue was identified in the scheduler's execution loop where sequential split decisions are executed with a stale catalog version, which will cause subsequent splits in the same cycle to fail due to optimistic concurrency control (OCC) mismatches. Updating the catalog version dynamically after each successful split is recommended to resolve this.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Uh oh!
There was an error while loading. Please reload this page.
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (a18b63d):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
bootjp
commented
Jul 18, 2026
@codex review |
To use Codex here, create a Codex account and connect to github. |
bootjp
commented
Jul 18, 2026
@codex review |
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (00e9a82):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
To use Codex here, create a Codex account and connect to github. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:00e9a82fb8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (4817c25):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
To use Codex here, create a Codex account and connect to github. |
bootjp
commented
Jul 18, 2026
Updated the scheduler follow-up on latest head. Addressed current findings around demo sampling, sampler route descriptor re-registration, and catalog-key leadership gating. Local checks: go test ./distribution/autosplit -count=1; go test . -count=1; go test ./cmd/server -count=1; go test ./kv -count=1; go test ./... -run '^$'; go test ./... -timeout=20m; golangci-lint --config=.golangci.yaml run --fix; git diff --check.\n\n@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:4817c25fdb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (9f3f8a9):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
To use Codex here, create a Codex account and connect to github. |
bootjp
commented
Jul 18, 2026
Current-head evidence for Requirement audit:
Current-head fixes for review roots:
Verification:
Commit signature: Good EDDSA signature; author and committer are |
bootjp
commented
Jul 18, 2026
@codex review |
Superseded by the verified current-head evidence comment: #1104 (comment) |
bootjp
commented
Jul 18, 2026
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:9f3f8a93c5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (3bd6a97):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
To use Codex here, create a Codex account and connect to github. |
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (9d4d857):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
bootjp
commented
Jul 23, 2026
Current HEAD: Resolved on this head:
Validation:
@codex review |
To use Codex here, create a Codex account and connect to github. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:9d4d857448
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| if *autoSplitDefaultBuckets <= keyviz.DefaultKeyBucketsPerRoute { | ||
| return errors.New("--autoSplitDefaultBuckets must be greater than 1") |
There was a problem hiding this comment.
When --autoSplit is enabled and the operator explicitly supplies --keyvizKeyBucketsPerRoute, --autoSplitDefaultBuckets is not used to build the sampler, but this validation still rejects the process if that unused default is set to 1 (or otherwise outside the allowed range). In that explicit-bucket configuration the node fails startup even though the effective sampler bucket count is valid, so the check should only apply when the auto-split default will actually be selected.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Resolved in 2a886f2. The autoSplitDefaultBuckets range check now runs only when that implied default bucket count will actually be used; explicit keyvizKeyBucketsPerRoute configs bypass unused invalid defaults. The demo path was updated the same way.
| snapshot.Version = result.CatalogVersion | ||
| refreshed, refreshErr := s.source.Snapshot(ctx) | ||
| if refreshErr != nil { | ||
| s.cfg.Logger.WarnContext(ctx, "autosplit: refresh after compound finalization failed", | ||
| slog.Any("err", refreshErr)) | ||
| break |
There was a problem hiding this comment.
When the catalog read transiently fails immediately after a pending compound finalization commits, this advances only snapshot.Version and then breaks with snapshot.Routes still describing the pre-finalization intermediate route. The rest of the tick reconciles/evaluates that mixed-version snapshot; since the distribution engine has already applied the committed children, observations resolve to the new child RouteIDs while the sampler has just been reconciled back to the old route, so writes to the newly-created ranges are dropped until another successful catalog refresh, and any spare cycle budget can be spent on stale route decisions.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Resolved in 2a886f2. After a pending compound finalization commits, a catalog refresh failure now stops the tick before sampler reconciliation, detector evaluation, or additional split scheduling can run against a mixed-version snapshot. Covered by TestSchedulerStopsTickAfterCompoundRefreshFailure.
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (2a886f2):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
To use Codex here, create a Codex account and connect to github. |
bootjp
commented
Jul 23, 2026
Current HEAD: Resolved on this head:
Validation:
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:2a886f2050
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| pending.intermediate = intermediate | ||
| s.pendingCompounds[parentID] = pending | ||
| attempted++ | ||
| result, err := s.executePendingCompound(ctx, snapshot.Version, pending) |
There was a problem hiding this comment.
When a compound split is pending and this node loses leadership for pending.intermediate.GroupID before the next tick, this path finalizes the second split before routesLedLocally has a chance to poll GroupLeadership and drop the pending entry. SplitRange itself only verifies catalog-key leadership, so a catalog leader can still mutate the catalog for a shard it no longer leads, using evidence from the prior shard leader term. Please gate pending finalization on current group leadership or run it after the group-leadership reconciliation.
Useful? React with 👍 / 👎.
| if snapshot.Version != engineVersion { | ||
| return nil |
There was a problem hiding this comment.
When a node catches up across more deltas than one watcher batch, applyCatalogDeltas advances the engine to the batch's last version, but notifyLatestSnapshotObserver loads the latest catalog snapshot and returns here because that version is still ahead of the engine. The route reconciler therefore is not notified about the route table the engine is actually serving until all batches are applied, so writes routed to newly-applied route IDs can be dropped from KeyViz/autosplit during catch-up. Notify with a snapshot at the applied engine version instead of skipping partial batches.
Useful? React with 👍 / 👎.
| for _, mut := range req.Mutations { | ||
| if mut == nil || isTxnMetaKey(mut.Key) { | ||
| continue |
There was a problem hiding this comment.
When a forwarded transaction cleanup reaches the leader as a Phase_ABORT request, the mutation list still contains the user keys after the txn metadata so the FSM can clear their intents; this loop only skips the metadata and records the remaining keys as committed writes. A follower-routed transaction that aborts can therefore create hot-key evidence for keys whose writes were rolled back, and the autosplit scheduler can split on cleanup traffic. Skip abort-phase requests, or otherwise restrict this observer to phases that represent user writes.
Useful? React with 👍 / 👎.
| runtime.reconciler.Reconcile(initialSnapshot.Routes) | ||
| runtime.autoSplitCfg.Reconciler = runtime.reconciler | ||
| } | ||
| coordinator.WithSamplerRouteResolver(runtime.sampler, demoSamplerRouteResolver(runtime.engine)) |
There was a problem hiding this comment.
When demo auto-split is enabled and clients write through a follower, this only wires the sampler into that follower's Coordinate; the demo Internal server is still constructed without WithInternalForwardWriteObserver. Since the demo scheduler runs only on the catalog/Raft leader and reads that node's local sampler, follower-ingressed writes never influence automatic splits unless clients happen to hit the current leader. Register coordinator.ObserveForwardedRequests with the demo Internal server as the production path does.
Useful? React with 👍 / 👎.
| if c == nil || c.sampler == nil || mut == nil { | ||
| return | ||
| } | ||
| c.sampler.Observe(routeID, mut.Key, keyviz.OpWrite, len(mut.Value), c.keyVizObserveLabel(label)) |
There was a problem hiding this comment.
For adapter-backed traffic whose storage keys normalize for routing, this records the raw mutation key even though routeID was found with routeKey(mut.Key) and the sampler slots were registered with catalog route boundaries in the normalized keyspace. With Redis, for example, user key z is observed as !redis|str|z, so after a split at m it sorts before the route start and all writes clamp into the first sub-bucket, causing autosplit to choose boundaries near m instead of the hot user key. Pass the normalized route key to the sampler for sharded observations so the evidence and route descriptors use the same ordering.
Useful? React with 👍 / 👎.
Two problems in the leader-side sampling of follower-forwarded writes. observeMutation passed the raw storage key to the sampler even though routeID came from routeKey and the sampler's sub-buckets are laid out on catalog route boundaries, which live in the normalized keyspace. A Redis user key "z" arrives as "!redis|str|z", which sorts before a route starting at "m", so every such write clamped into the first sub-bucket and autosplit picked a boundary near the route start rather than the hot user key. Sample the route key instead. The single-group path already did this through samplerConfig.routeForKey, so this aligns the sharded path with the established convention. ObserveForwardedRequests also counted ABORT cleanups. An abort still lists the user keys after the txn metadata so the FSM can clear their intents, but those writes were rolled back, so a follower-routed transaction that aborts could manufacture hot-key evidence and get the scheduler to split on cleanup traffic. Skip abort-phase requests in both the sharded and single-group observers. Claude-Session: https://claude.ai/code/session_013rNHooj7NF3giihWVba8QE
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
bootjp
commented
Aug 22, 2026
Codex P2 2 件 — 1. サンプリングキーの正規化 ( |
Claude finished @bootjp's task in 2m 39s —— View job Review of |
Three review findings. The compound finalization loop ran before routesLedLocally had a chance to reconcile group leadership, so a node that lost leadership for the intermediate route's group since the last tick could still finalize the second split. SplitRange only verifies catalog-key leadership, so a catalog leader could mutate the catalog for a shard it no longer leads using evidence from the prior shard leader term. Recheck leadership on each pending entry, and reject a changed term so a lose-and-regain between ticks does not carry the old term's evidence forward. notifyLatestSnapshotObserver returned without notifying whenever the persisted catalog was ahead of the engine, which is exactly the state during a multi-batch catch-up. The reconciler therefore never learned about the route table the engine was actually serving until every batch applied, so writes to newly applied route IDs went unattributed in KeyViz and autosplit for the whole catch-up. Notify with the engine's applied routes instead, via the new Engine.AppliedCatalogSnapshot. The demo Internal server was built without a forward-write observer, so follower-ingressed writes never reached the leader-local sampler the demo scheduler reads. Wire coordinator.ObserveForwardedRequests as main.internalTimestampOptions already does. Claude-Session: https://claude.ai/code/session_013rNHooj7NF3giihWVba8QE
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
bootjp
commented
Aug 22, 2026
残る Codex P2 3 件に対応しました（ 1. |
Claude finished @bootjp's task in 2m 22s —— View job Review of |
bootjp
commented
Aug 22, 2026
CI について: 失敗内容: 本 PR の変更とは無関係と判断した根拠:
依存バージョンを上げただけの renovate ブランチや main でも同様に落ちているため、コード変更に起因するものではなく、 再実行の結果が緑にならなければ、改めて調査します。 |
Only the two hotspot-split design docs conflicted; no code conflicts. main still described M3 automation as partial and referenced the doc under its _partial_ name. This branch is the one that completes M3 and renamed that doc to _implemented_, so the implemented status and the renamed path win. main's M2 SplitJob-catalog bullet is new here and had no counterpart on this side, so it is kept alongside the M3 bullet. Claude-Session: https://claude.ai/code/session_013rNHooj7NF3giihWVba8QE
TLA+ spec divergence review (auto-triggered)This PR touches files that the TLA+ safety spec has an anchor on (per Anchored files changed in this PR head (3e848ca):
What to check, by subsystem:
If the change is correct but requires a spec update, edit @claude review please verify TLA+ spec divergence per the checklist above. @codex review please verify TLA+ spec divergence per the checklist above. |
To use Codex here, create a Codex account and connect to github. |
bootjp
commented
Aug 22, 2026
新規の指摘・CI 失敗は無かったため、マージを塞いでいた 競合の範囲
解決の方針M3 のステータスは本ブランチ側（implemented）を採用しました。 片側だけ捨てる解決はしていません。 検証（ |
Claude encountered an error after 7s —— View job I'll analyze this and get back to you. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:3e848ca0c6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| func forwardedRequestRecordsUserWrites(req *pb.Request) bool { | ||
| return req.GetPhase() != pb.Phase_ABORT |
There was a problem hiding this comment.
For a follower-routed multi-shard transaction, prewriteTxn forwards a Phase_PREPARE request containing every user mutation and the commit path later forwards a Phase_COMMIT request containing those keys again. Because this predicate accepts both phases, Internal.Forward invokes this observer after each successful phase and the shard leader samples every committed user key twice; it can therefore cross the autosplit threshold at half the configured traffic rate. Restrict transactional sampling to one phase or otherwise deduplicate the transaction's keys.
Useful? React with 👍 / 👎.
| if !ok { | ||
| continue | ||
| } | ||
| c.observeMutation(routeID, mut, keyviz.LabelLegacy) |
There was a problem hiding this comment.
When KeyViz labels are enabled and an adapter write enters through a follower, the leader-side observer unconditionally records it as LabelLegacy, even though the ingress OperationGroup was labeled Redis, DynamoDB, S3, and so on. Since the forwarded protobuf currently carries no label, the leader's label-specific KeyViz data misattributes all follower-routed traffic; propagate the label with the forwarded request or preserve it through another forwarding-side mechanism.
Useful? React with 👍 / 👎.
bootjp
commented
Aug 22, 2026
CI 失敗 |
bootjp
commented
Aug 22, 2026
再実行後、別のテストが落ちたので追記します。こちらも本 PR とは無関係と判断し、再度再実行しました。
無関係と判断した根拠
補足: これで #1104 の CI では 2 本目の異なる flake です
どちらも本 PR が触っていない経路で、 |
bootjp
commented
Aug 22, 2026
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:3e848ca0c6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if ok { | ||
| r.registrar.RemoveRoute(route.RouteID) | ||
| } | ||
| r.registrar.RegisterRoute(route.RouteID, route.Start, route.End, route.GroupID) |
There was a problem hiding this comment.
When a catalog update changes the bounds of an existing RouteID, this immediate RemoveRoute/RegisterRoute sequence reclaims the retired MemSampler slot inside its grace window. Fresh evidence in MemSampler.RegisterRoute and TestSamplerGraceWindowReRegistrationKeepsStaleLayout confirms that reclamation intentionally preserves the old immutable sub-range layout while only replacing the displayed Start/End metadata. Subsequent observations are therefore bucketed using the old range, so KeyViz rows and autosplit p50 decisions can be attributed to incorrect boundaries; use a replacement path that creates a new layout for descriptor changes while draining the old counters separately.
Useful? React with 👍 / 👎.
| live := make(map[uint64]struct{}, len(routes)) | ||
| for _, route := range routes { | ||
| live[route.RouteID] = struct{}{} | ||
| registered, ok := r.registered[route.RouteID] | ||
| next := registeredRouteFromDescriptor(route) |
There was a problem hiding this comment.
When a split occurs with the sampler near MaxTrackedRoutes, this pass registers both new children while the removed parent still occupies an individual slot because obsolete routes are removed only in the later loop. For example, splitting 1 of 1023 routes with a 1024-route capacity transiently requires 1025 slots, so MemSampler.RegisterRoute folds one child into a virtual aggregate even though removing the parent immediately afterward leaves enough final capacity. The reconciler then records that child as registered and skips it on later equal snapshots, leaving its traffic permanently unavailable as individual KeyViz/autosplit evidence; remove obsolete RouteIDs before adding replacements or retry coarsened registrations after capacity is freed.
Useful? React with 👍 / 👎.
| for _, decision := range decisions { | ||
| nextCatalogVersion, err := s.executeDecision(ctx, catalogVersion, decision) |
There was a problem hiding this comment.
When the route's shard leadership transfers after routesLedLocally evaluates it but before this loop executes, the normal decision still calls SplitRange using evidence from the previous shard-leader term. Unlike catalog leadership, which DistributionServer.verifyCatalogLeader checks again, SplitRange does not verify leadership of decision.RouteGroupID; the catalog leader can therefore mutate a route it no longer leads. The pending-compound path already performs this execution-time group/term check, so ordinary decisions need the same fence immediately before each call.
Useful? React with 👍 / 👎.
Summary
Review roots addressed
Tests
Summary by CodeRabbit
新機能
バグ修正
ドキュメント