fix(drive): skip the ranked offset by counting instead of walking

[shumkov]

Issue being fixed or feature implemented

Ranked queries (SELECT <agg> GROUP BY <prop> ORDER BY <agg> LIMIT k OFFSET m) accept an
unbounded OFFSET, and the comment justifying that described the proving path only:

grovedb's paginated prover attests the skipped region from the counted subtree commitments
instead of walking it … There is no denial-of-service lever here to cap.

The unproved arm did not work that way. It skipped by stepping a storage iterator once per
skipped entry, so the skip alone cost Θ(min(offset, population)) — work proportional to a
number the caller chooses, on a path where nothing bounds it: ranked queries carry no fee, the
dispatcher does no cost accounting, a spawn_blocking query cannot be cancelled by client
disconnect or stream reset, and the gateway's only rate limit is per source IP across the whole
Platform service, so a query competes with state transitions rather than having its own budget.

The ranked surface exists only in v4.2.0-dev.1; no stable release contains it.

What was done?

grovedb gained a counted traversal for plain reads (dashpay/grovedb#792), which is the same
descent its prover already used, minus the proof. It reads each subtree's aggregate count off its
link and collapses any subtree that fits entirely inside the remaining offset rather than stepping
through it. This PR points the unproved executor at it.

Consequences:

• The skip is O(log n) at any offset, not proportional to it.
• An offset at or past the population is answered from the root's own count with no descent at
  all — the pathological input becomes the cheapest request on the surface rather than the most
  expensive.
• offset = 0 is untouched: it keeps the plain iterator path and never reads the tree, so the
  common unpaginated request costs exactly what it did before. grovedb pins that with an
  always-on equality test.
• The proof path is unchanged, and grovedb's proof suites needed no edits.

Measured by grovedb's own harness (seek/byte counters are the machine-independent signal; the full
grid is in that PR). At a million rows, a deep offset drops from a full linear scan to a
tree-depth descent — 22 seeks, ~3.7 KB, ~32 µs — and past the end to a flat 3 seeks / 366 B / 4 µs
at every population size. The counters scale as tree depth (11 → 15 → 18 → 22 across 1e3 → 1e6),
which is the shape the design predicts.

One corner measured and accepted rather than hidden: at a small positive offset with k = 100,
the counted path costs about 5× the linear read in wall-clock (~155 µs against ~30 µs) because
k tree point-gets are slower than k sequential iterator steps. Crossover to counted-wins sits
a few hundred rows in, worst measured cost is ~155 µs, and the alternative — a threshold hybrid
falling back to the linear skip below some offset — would make the skipped-region semantics depend
on the offset value. Uniform semantics won.

Pin: currently the grovedb branch rev, so this is reviewable now; to be re-pinned to the
develop merge commit before merge. That is a one-line change and does not invalidate review of
anything else here.

Supersedes this PR's own earlier approach. It previously served unproved reads by generating a
proof internally and verifying it to recover the entries. That worked, but it paid proof
construction, serialization and verification on every read, put a floor under the common
offset = 0 case, and its retry drew a blocking review for pairing new-state results with old
block metadata. The counted read removes the floor and, by having no proof envelope on the read
path, removes the retry and the state/metadata window with it. History was rewritten because the
old commits implemented an approach the diff no longer contains.

How Has This Been Tested?

• cargo test -p drive --lib → 3386 passed, 0 failed
• cargo test -p drive-abci --lib query:: → 623 passed, 0 failed
• cargo clippy --workspace --all-features → clean
• cargo fmt --check --all → clean

The informative result is which assertions moved. Across ~3,400 tests, four needed changing
and every one was a skipped value — three in drive, one on the wire in drive-abci. No entry or
ordering assertion moved, on any axis, in either direction, at any offset or k. That is the
claim this change stakes itself on: the counted read returns exactly what the linear walk
returned, and only the reported skip differs.

Read consistency

The counted page — root, descent and collect — is served from a single transaction raw iterator
with a pinned snapshot plus the transaction overlay, which is the same consistency mechanism the
linear scan it replaces relied on. This matters because the descent performs several reads where
the old scan performed one: without a pinned view, a block committing mid-descent could pair a
parent from the old state with a child from the new one, and merk does not verify a fetched child
against the parent's recorded link hash, so the result would be a silently mixed page rather than
an error. The proved path is not exposed the same way — a torn read there fails the verifier's
ancestor-chain reconciliation — which is why this was specific to the unproved read.

Cost of the guarantee: one extra seek (deep offset 22 → 23; offset 0 unchanged at 5; past-the-end
flat at 4).

Scope of the testing, stated rather than implied: the transaction-overlay behaviour is pinned by a
test. The commit-interleave case is not deterministically testable — there is no hook to pause a
fetch and force a commit mid-descent — so that half is argued from the mechanism, not proven by a
test.

A gate lesson worth keeping

This PR broke CI in a way cargo clippy --workspace --all-features structurally cannot catch, and that is worth writing down because the opposite advice is commonly given.

The pinned grovedb rev exported a type under any(minimal, verify) while the module holding it was gated on minimal alone. Any build enabling verifywithoutminimal failed with error[E0432]: unresolved import. --all-features turns every feature on, so the broken combination never occurs and the check passes; the failure only appears in a narrow cut, and it took CI's Check transport-free feature cut job — reproducible locally as cargo check -p drive --no-default-features --features verify — to surface it. The Kotlin native-library job hit the same error for the same reason.

So the two gates catch different classes and neither substitutes for the other:

• --workspace --all-features catches unbuilt sibling crates and feature-gated callers of a changed API. It is blind to feature-gating bugs.
• The narrow cuts (--no-default-features --features verify, and the other combinations CI builds) catch gating bugs. They are blind to most of what breadth catches.

If you are changing a #[cfg], adding a re-export, or bumping a dependency that does either, run the cut as well as the breadth build.

Breaking Changes

No API or wire format change. One wire-visible behaviour change on unproved responses.

RankedPage::skipped — GetDocumentsResponseV1.ResultData.Ranked.skipped on the wire — stops
echoing the request. The old read could not tell how far a short walk got, so the server echoed
the requested offset back. The counted descent tracks it, so both paths now report the same
quantity: the requested offset when the skip succeeded, and the ranking's total population when
the walk ran out of groups first. On a five-group ranking asked for a page well past the end, an
unproved response now reports 5 where it previously reported the offset.

A client asserting skipped == requested_offset sees a different value past the end. A client
using it as the rank base for entries[i] — its documented purpose — is unaffected, and gains a
population count it previously had to prove to obtain.

The value is not attested on the unproved path. It equals the attested one on an honest node,
and nothing forces a node to be honest — the same trust model as the entries beside it. The proto,
the Objective-C generated client (the only generated client carrying proto prose), the developer
book and the Rust docs all say so explicitly, so "the true population" is not read as a
cryptographic guarantee. One further nuance is documented at the field: the population comes from
the secondary's root aggregate while the per-node payload check only fires on visited nodes, so in
a corrupt secondary the unproved value can disagree with the true row count where the proved one
would not. On any valid secondary they are identical by construction.

Mixed-network note: the ranked surface first appears in v4.2.0-dev.1, so a network mixing
that tag with newer nodes returns the echoed offset from one and the population from the other for
the same unproved request. Devnet-only exposure, and the proto's new "do not assume this field
equals the offset you requested" advice is safe against both.

Comments corrected

Three comments asserted things the code did not do, one of them the justification for leaving
OFFSET uncapped. They land here rather than earlier on purpose: two of them state the policy, and
an accurate description of an uncapped cost lever is only safe to publish alongside the change that
removes it.

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated relevant unit/integration/functional/e2e tests
• I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
• I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

• I have assigned this pull request to a milestone

Summary by CodeRabbit

• Bug Fixes

  • Ranked pagination now reports the actual number of groups skipped when offsets extend beyond available results.
  • Proved and unproved reads return consistent skip counts, including for empty pages.
  • Ranked queries maintain efficient performance across all offsets using counted traversal.

• Documentation

  • Clarified pagination behavior, performance, skip counts, and proof attestation.
  • Updated API guidance to distinguish cryptographically attested proved counts from server-reported unproved counts.

[thepastaclaw]

🕓 Ready for review — 1 ahead in queue (commit fbae91a)
Queue position: 2/5 · 2 reviews active
ETA: start ~15:09 UTC · complete ~15:33 UTC (median 24m across 30 recent reviews; 2 slots)
Queued 34m ago · Last checked: 2026-08-24 14:50 UTC

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 128d3c8e-9c6e-4489-9575-6e11b275d9b3

📥 Commits

Reviewing files that changed from the base of the PR and between 7091b33 and 6f70cc5.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (8)

• packages/dapi-grpc/clients/platform/v0/objective-c/Platform.pbobjc.h
• packages/dapi-grpc/protos/platform/v0/platform.proto
• packages/rs-dpp/Cargo.toml
• packages/rs-drive-abci/Cargo.toml
• packages/rs-drive/Cargo.toml
• packages/rs-platform-version/Cargo.toml
• packages/rs-platform-wallet/Cargo.toml
• packages/rs-sdk/Cargo.toml

🚧 Files skipped from review as they are similar to previous changes (4)

• packages/rs-sdk/Cargo.toml
• packages/dapi-grpc/protos/platform/v0/platform.proto
• packages/rs-dpp/Cargo.toml
• packages/rs-drive/Cargo.toml

---

📝 Walkthrough

Walkthrough

Ranked pagination now reports the actual number of groups skipped. Proved and unproved reads use the same value, including past-end queries. Proved responses attest the value cryptographically. Grovedb dependencies, tests, and documentation use the updated behavior.

Changes

Ranked pagination

Layer / File(s)	Summary
Ranked execution semantics packages/rs-drive/src/query/drive_document_ranked_query/...	The executor returns Grovedb’s actual skipped count for count, sum, average, and page results. Documentation describes counted descent and past-end behavior.
Ranked pagination contracts packages/dapi-grpc/protos/platform/v0/platform.proto, packages/dapi-grpc/clients/platform/v0/objective-c/Platform.pbobjc.h, packages/rs-drive/src/query/drive_document_ranked_query/mod.rs, packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/v0/mod.rs, book/src/drive/ranked-index-examples.md	Public documentation defines the same skipped-count semantics for proved and unproved responses.
Past-end pagination validation packages/rs-drive-abci/src/query/document_query/v1/tests.rs, packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/ranked_index_e2e_tests.rs, packages/rs-drive/src/query/drive_document_ranked_query/tests.rs	Tests expect the ranking population instead of the requested offset.
Grovedb revision alignment packages/rs-dpp/Cargo.toml, packages/rs-drive-abci/Cargo.toml, packages/rs-drive/Cargo.toml, packages/rs-platform-version/Cargo.toml, packages/rs-platform-wallet/Cargo.toml, packages/rs-sdk/Cargo.toml	Grovedb dependencies now reference revision 0100cb833075621659a68ddd3696baecc98e55b8.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Mergeability Score:⚪ Minimal · up to 6f70c

The change replaces linear ranked-query offset skipping with counted traversal and updates the reported skipped value past the end; the supplied validation indicates the behavior is merge-ready after normal checks, with no actionable merge-blocking risk remaining.

Sequence Diagram(s)

sequenceDiagram
participant RankedQuery
participant Grovedb
participant RankedPage
RankedQuery->>Grovedb: Request indexed top-K page with offset
Grovedb-->>RankedQuery: Return entries and actual skipped count
RankedQuery->>RankedPage: Map entries and preserve skipped count
RankedPage-->>RankedQuery: Return ranked pagination response

Loading

Possibly related PRs

• dashpay/platform#4266: Directly introduced the ranked-pagination behavior refined by this change.

Suggested reviewers:lklimek, quantumexplorer, thepastaclaw

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: using counted traversal instead of walking to skip ranked offsets.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/ranked-unproved-read-through-prover

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

📖 Book Preview built successfully.

Download the preview from the workflow artifacts.
To view locally: download the artifact, unzip, and open index.html.

Updated at 2026-08-24T14:13:16.987Z

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

packages/rs-drive/src/query/drive_document_ranked_query/tests.rs (1)

1682-1694: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider a separate tolerance constant for proof size.

byte_slack is derived as a storage-loaded-bytes allowance (256 bytes per tree level). Line 1690 reuses it as a proof-size tolerance. The two quantities are unrelated, so a later change to the storage allowance silently changes this tripwire. Define a distinct constant for the proof-size comparison.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/rs-drive/src/query/drive_document_ranked_query/tests.rs` around
lines 1682 - 1694, Define a dedicated proof-size tolerance constant near the
proof-size assertion, rather than reusing byte_slack. Update the proof_bytes_at
comparison to use this new constant, while leaving byte_slack exclusively for
storage-loaded-bytes allowances.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/rs-drive-abci/src/query/document_query/v1/tests.rs`:
- Around line 3058-3090: Update empty_ranking_proof_rejection and its tests so
only the exact supported GroveError::CorruptedData message “Cannot create proof
for empty tree” is reclassified as QueryError::InvalidArgument. Replace the
substring-based contains predicate with exact message matching, and add a test
case containing the marker within unrelated text that must remain unmapped.
In `@packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs`:
- Around line 171-180: Update the proof-read flow in the ranked query execution
around verify_indexed_axis_top_k_paginated to use snapshot isolation; if
unavailable, add a bounded retry at the dispatcher only when the error is the
specific chain-mismatch verification failure. Preserve immediate propagation for
all other proof or GroveDB errors.
---
Nitpick comments:
In `@packages/rs-drive/src/query/drive_document_ranked_query/tests.rs`:
- Around line 1682-1694: Define a dedicated proof-size tolerance constant near
the proof-size assertion, rather than reusing byte_slack. Update the
proof_bytes_at comparison to use this new constant, while leaving byte_slack
exclusively for storage-loaded-bytes allowances.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 09b317c6-215d-46bb-8681-50ff49f5eb3c

📥 Commits

Reviewing files that changed from the base of the PR and between f05bf82 and f432daa.

📒 Files selected for processing (12)

• book/src/drive/ranked-index-examples.md
• packages/dapi-grpc/clients/platform/v0/objective-c/Platform.pbobjc.h
• packages/dapi-grpc/protos/platform/v0/platform.proto
• packages/rs-drive-abci/src/query/document_query/v1/mod.rs
• packages/rs-drive-abci/src/query/document_query/v1/tests.rs
• packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/ranked_index_e2e_tests.rs
• packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs
• packages/rs-drive/src/query/drive_document_ranked_query/executors/top_k_no_proof.rs
• packages/rs-drive/src/query/drive_document_ranked_query/mod.rs
• packages/rs-drive/src/query/drive_document_ranked_query/mode_detection.rs
• packages/rs-drive/src/query/drive_document_ranked_query/tests.rs
• packages/rs-drive/src/verify/document_ranked/verify_ranked_top_k_proof/v0/mod.rs

[codecov]

Codecov Report

❌ Patch coverage is 88.20961% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.39%. Comparing base (4be6fc1) to head (fbae91a).
⚠️ Report is 1 commits behind head on v4.2-dev.

Files with missing lines	Patch %	Lines
...perations/shielded/shielded_transfer_transition.rs	60.60%	13 Missing ⚠️
...vert_to_operations/shielded/unshield_transition.rs	86.95%	6 Missing ⚠️
packages/rs-drive/src/fees/op.rs	0.00%	4 Missing ⚠️
...grove_operations/grove_insert_empty_tree/v0/mod.rs	0.00%	4 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## v4.2-dev #4382 +/- ##
==========================================
Coverage 87.39% 87.39% ==========================================
Files 2735 2735 Lines 347672 347804 +132 ==========================================
+ Hits 303855 303975 +120 - Misses 43817 43829 +12 

Components	Coverage Δ
dpp	88.97% <91.30%> (-0.01%)	⬇️
drive	86.32% <88.15%> (-0.01%)	⬇️
drive-abci	89.72% <100.00%> (+0.01%)	⬆️
sdk	∅ <ø> (∅)
dapi-client	∅ <ø> (∅)
platform-version	∅ <ø> (∅)
platform-value	92.92% <ø> (ø)
platform-wallet	∅ <ø> (∅)
drive-proof-verifier	47.40% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[shumkov]

Responding to the nitpick from the review body (it has no thread of its own): separate tolerance constant for proof size — agreed and done in 577fe3d.

There's now a proof_size_slack, derived from what an envelope actually carries per level of the counted descent, with a comment recording why it must stay distinct from byte_slack: the latter bounds storage reads, the two are unrelated quantities, and sharing one constant would let a change to either silently move the other's tripwire. That was a fair catch.

For the record, the two inline comments are answered in their own threads: the retry suggestion was adopted, and the exact-match suggestion for the empty-tree marker was rejected because grovedb wraps merk's constant in its own prefix, so exact matching would prevent the mapper from ever firing.

[QuantumExplorer]

You should not use the proved path for this, instead there are unproved ways that will make this fast, even faster.

[thepastaclaw]

Preliminary review — Codex only

The prover-backed ranked read removes the offset-proportional walk, but the new handler-local retry crosses the state-publication boundary without refreshing the captured PlatformState. A retry that succeeds after the GroveDB commit can therefore return new-state results or proof bytes with the previous block's metadata and signature, so this requires changes before merge.
Source: reviewer backend model gpt-5.6-sol; final verifier backend model gpt-5.6-sol. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — general (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 1 blocking

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-drive-abci/src/query/document_query/v1/mod.rs`:
- [BLOCKING] packages/rs-drive-abci/src/query/document_query/v1/mod.rs:1367-1369: Retrying only Drive execution can pair new-state results with old block metadata
The retry re-executes the Drive request while retaining the `platform_state` reference captured by `QueryService` before the query began. This is unsafe in the exact commit-visible/guard-not-yet-published window the retry is intended to cover: `update_state_cache_v0` publishes the new PlatformState before the database transaction commits, `finalize_block` commits GroveDB, and only afterward stores the new `committed_block_height_guard`. A query that captured the old state before publication can have its first envelope torn by the commit, then successfully rebuild against the newly committed GroveDB state here. Because the guard still has the old height, the service post-check sees it equal to the captured old state's height and accepts the response. Lines 1409-1417 then attach metadata—and, for proved responses, the old block signature and block ID—from that old PlatformState to new-state data or proof bytes. The retry must restart at a boundary that reloads PlatformState and reruns the service consistency checks; a successful local retry cannot safely be wrapped with the existing state object.

[shumkov]

@coderabbitai review

Please review the current head (0f44e70130). Your last review was of 577fe3d3, which is a different implementation — this PR was rebuilt, not amended, so a re-read from scratch is worth more than a diff against your previous pass.

What changed since you last saw it:

• Prove-then-verify is gone. The unproved ranked read no longer generates a grovedb proof and verifies it to recover entries. grovedb exposed its counted descent to plain reads (feat: counted offset skip for unproved ranked paginated reads grovedb#792), so the executor now calls indexed_*_top_k_paginated directly and reads skipped off IndexedTopKPage.
• The handler-local retry is gone entirely — that was the substance of the previous blocking finding (a retry could pair new-state results with the previously captured PlatformState's block metadata and signature). It is not narrowed or made state-safe; with no proof envelope on the read path there is nothing to tear, so there is nothing to retry.
• The diff is much smaller: 17 files, +241/−154, down from +1067/−263.
• The behaviour change that remains is skipped on unproved responses: it reports the skip actually performed rather than echoing the requested offset, and the docs state plainly that on that path it is the node's unverified claim rather than an attested value.

Two of your earlier findings were adopted and remain relevant to the current code: the empty-tree mapper's narrowness, and the separate proof-size tolerance constant. Your suggestion to exact-match the empty-tree marker was declined, with reasoning in that thread — grovedb wraps merk's constant in its own prefix, so an equality predicate would never fire.

[coderabbitai]

@shumkov I will review the current head from scratch. I will assess the direct paginated read path, skipped behavior, error mapping, documentation, and test coverage.

---

🧠 Learnings used

Learnt from: QuantumExplorer
Repo: dashpay/platform PR: 4266
File: packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs:135-167
Timestamp: 2026-08-02T21:07:14.747Z
Learning: For ranked aggregate proofs, `DriveDocumentRankedQuery::execute_top_k_with_proof` in `packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs` receives Grovedb's empty-tree proof failure as `grovedb::Error::CorruptedData(String)`, not a dedicated typed error. It must preserve the generic `Error::GroveDB` mapping. `packages/rs-drive-abci/src/query/document_query/v1/mod.rs` narrowly maps that specific empty-tree message to a query validation error until Grovedb provides a typed variant.

---

Learnt from: shumkov
Repo: dashpay/platform PR: 4382
File: packages/rs-drive-abci/src/query/document_query/v1/tests.rs:3058-3090
Timestamp: 2026-08-12T19:16:31.122Z
Learning: In `packages/rs-drive-abci/src/query/document_query/v1/mod.rs`, `empty_ranking_proof_rejection` must match `"Cannot create proof for empty tree"` within a `GroveError::CorruptedData` message. GroveDB wraps Merk's marker with dynamic indexed-axis proof context, so the observed error payload is not equal to the bare marker. The mapping must remain limited to the GroveDB corrupted-data error variant; unrelated `CorruptedData`, `PathNotFound`, and non-GroveDB errors must remain unmapped.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[shumkov]

@coderabbitai review

Superseding my previous request: review 7091b33212, not 0f44e70130 — that head has been replaced by a dependency re-pin and my earlier request is stale. This is the settled head; no further pushes are planned before merge.

Your last substantive review was of 577fe3d3, which is a different implementation. This PR was rebuilt rather than amended, so a fresh read is worth more than a diff against your previous pass:

• Prove-then-verify is gone. The unproved ranked read no longer generates a grovedb proof and verifies it to recover entries. grovedb exposed its counted descent to plain reads (feat: counted offset skip for unproved ranked paginated reads grovedb#792), so the executor calls indexed_*_top_k_paginated directly and reads skipped off IndexedTopKPage.
• The handler-local retry is gone entirely — the substance of the previous blocking finding, where a retry could pair new-state results with the block metadata and signature of the PlatformState captured before the query. It is not narrowed or made state-safe: with no proof envelope on the read path there is nothing to tear, so there is nothing to retry.
• The diff is much smaller: 17 files, +241/−154, down from +1067/−263.
• The remaining behaviour change is skipped on unproved responses — it reports the skip actually performed rather than echoing the requested offset, and the docs state that on that path it is the node's unverified claim rather than an attested value.

Two of your earlier findings were adopted and still apply to the current code: the empty-tree mapper's narrowness, and a separate proof-size tolerance constant. Your suggestion to exact-match the empty-tree marker was declined with reasoning in that thread — grovedb wraps merk's constant in its own prefix, so an equality predicate would never fire on a real failure.

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo'== HEAD =='
git rev-parse HEAD
git status --short
echo'== target commit =='
git show --no-ext-diff --no-renames --format=fuller --stat 7091b33212
echo'== merge base and changed files =='
base="$(git merge-base 7091b33212 origin/master 2>/dev/null || git merge-base 7091b33212 origin/main)"printf'BASE=%s\n'"$base"
git diff --no-ext-diff --no-renames --name-status "$base" 7091b33212
echo'== changed implementation outline =='
ast-grep outline packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs --items all --view expanded
ast-grep outline packages/rs-drive-abci/src/query/document_query/v1/mod.rs --items all --view expanded
echo'== focused symbols and call sites =='
rg -n -C 5 --glob '*.rs' \
'indexed_.*_top_k_paginated|IndexedTopKPage|empty_ranking_proof_rejection|proof_size_slack|byte_slack|execute_top_k_with_proof|execute_top_k' \
packages/rs-drive packages/rs-drive-abci
echo'== target diff =='
git diff --no-ext-diff --no-renames --unified=80 "$base" 7091b33212 -- \
packages/rs-drive/src/query/drive_document_ranked_query \
packages/rs-drive-abci/src/query/document_query/v1 \
packages/rs-drive/Cargo.toml \
packages/rs-drive-abci/Cargo.toml \
packages/rs-dpp/Cargo.toml \
packages/rs-platform-version/Cargo.toml \
packages/rs-platform-wallet/Cargo.toml \
packages/rs-sdk/Cargo.toml \
packages/dapi-grpc/protos/platform/v0/platform.proto \
book/src/drive/ranked-index-examples.md

Length of output: 3619

---

🏁 Script executed:

#!/bin/bashset -u
echo'== refs =='
git for-each-ref --format='%(refname:short) %(objectname)' refs/remotes refs/heads | sort
echo'== recent ancestry =='
git log --oneline --decorate --graph -25 HEAD
echo'== candidate merge bases =='forrefin origin/develop develop upstream/develop origin/master master origin/main main;doif git rev-parse --verify -q "$ref">/dev/null;thenprintf'%s '"$ref"
git merge-base HEAD "$ref"||truefidoneecho'== all commits after a likely base, if origin/develop exists =='if git rev-parse --verify -q origin/develop >/dev/null;then
base="$(git merge-base HEAD origin/develop)"printf'BASE=%s\n'"$base"
git log --oneline --reverse "$base"..HEAD
echo'== changed files =='
git diff --no-ext-diff --no-renames --name-status "$base"..HEAD
fi

Length of output: 3542

---

🏁 Script executed:

#!/bin/bashset -euo pipefail
base="6495991c5e21f4f211ba326334ce3f717aaaf3bd"
head="7091b332120f35e7155a7867fa98a209c22bca6a"echo'== changed files =='
git diff --no-ext-diff --no-renames --name-status "$base""$head"echo'== implementation outlines =='
ast-grep outline packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs --items all --view expanded
ast-grep outline packages/rs-drive/src/query/drive_document_ranked_query/mod.rs --items all --view expanded
ast-grep outline packages/rs-drive-abci/src/query/document_query/v1/mod.rs --items all --view expanded
echo'== direct read calls, result handling, and empty-tree mapping =='
rg -n -C 8 --glob '*.rs' \
'indexed_(value|sum|count|average)_top_k_paginated|IndexedTopKPage|empty_ranking_proof_rejection|CorruptedData|proof_size_slack|byte_slack' \
packages/rs-drive/src/query/drive_document_ranked_query \
packages/rs-drive-abci/src/query/document_query/v1
echo'== source diff: ranked implementation and ABCI handling =='
git diff --no-ext-diff --no-renames --unified=100 "$base""$head" -- \
packages/rs-drive/src/query/drive_document_ranked_query \
packages/rs-drive-abci/src/query/document_query/v1
echo'== source diff: API docs and dependency manifests =='
git diff --no-ext-diff --no-renames --unified=60 "$base""$head" -- \
packages/dapi-grpc/protos/platform/v0/platform.proto \
packages/dapi-grpc/clients/platform/v0/objective-c/Platform.pbobjc.h \
book/src/drive/ranked-index-examples.md \
packages/rs-dpp/Cargo.toml \
packages/rs-drive/Cargo.toml \
packages/rs-drive-abci/Cargo.toml \
packages/rs-platform-version/Cargo.toml \
packages/rs-platform-wallet/Cargo.toml \
packages/rs-sdk/Cargo.toml \
Cargo.lock
echo'== stale GroveDB pins outside the lockfile =='
rg -n --glob 'Cargo.toml' --glob 'Cargo.lock' \
'a2791bbdca756d6a6113024aec48f09f7a33faa9|cc7b3997be5819c60eb09251f963c1ed1a89306f|e41d57e0|c4ceac67'.

Length of output: 199377

---

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo'== ranked response conversion =='
rg -n -C 12 --glob '*.rs' \
'DocumentRankedResponse|RankedPage|skipped:|set_skipped|RankedEntries' \
packages/rs-drive/src/query/drive_document_ranked_query \
packages/rs-drive-abci/src/query/document_query/v1/dispatch
echo'== ranked dispatch execution path =='
sed -n '80,190p' packages/rs-drive-abci/src/query/document_query/v1/dispatch/ranked.rs
sed -n '1,135p' packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs
sed -n '135,255p' packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs
echo'== proof-size tolerance definition and uses =='
rg -n -C 12 --glob '*.rs''proof_size_slack|byte_slack|proof.*slack|slack.*proof' packages/rs-drive packages/rs-drive-abci
echo'== offset test cases across axes =='
rg -n -C 8 --glob '*.rs' \
'at_offset\(|offset_pages|past_end|skipped.*[=,].*[0-9]|IndexedTopKPage' \
packages/rs-drive/src/query/drive_document_ranked_query/tests.rs \
packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/ranked_index_e2e_tests.rs \
packages/rs-drive-abci/src/query/document_query/v1/tests.rs
echo'== pinned GroveDB source candidates =='
find "${HOME:-/root}/.cargo/git/checkouts" -type f \
\( -name '*.rs' -o -name 'Cargo.toml'\) \
-print 2>/dev/null | grep -E '/grovedb[^/]*/|/grovedb-'| head -n 20 ||true
find /tmp -type f -name '*.rs' -print 2>/dev/null | grep -E 'grovedb|indexed.*top.*k'| head -n 20 ||true

Length of output: 113333

---

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo'== Avg call without overlapping output =='
nl -ba packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs | sed -n '125,155p'echo'== proof-size tolerance candidates across the repository =='
rg -n -i -C 8 \
'proof.{0,40}(size|slack|limit|bound|tolerance)|\ (size|slack|limit|bound|tolerance).{0,40}proof|\ byte_slack' \
. \
-g '!Cargo.lock' \
-g '!**/node_modules/**'||trueecho'== all changed lines not previously inspected =='
git diff --no-ext-diff --no-renames --unified=35 \
6495991c5e21f4f211ba326334ce3f717aaaf3bd \
7091b332120f35e7155a7867fa98a209c22bca6a -- \
packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/ranked_index_e2e_tests.rs \
packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/v0/mod.rs \
packages/rs-drive/src/query/drive_document_ranked_query/tests.rs \
packages/rs-drive-abci/src/query/document_query/v1/tests.rs
echo'== GroveDB source tree paths at the pinned revision =='
gh api "repos/dashpay/grovedb/git/trees/c4ceac676630727d01b532314a7e488edbeaf115?recursive=1" \
--jq '.tree[] | select(.path | test("indexed.*top.*k|top.*k.*indexed"; "i")) | [.path, .url] | `@tsv`' \
||true

Length of output: 1624

---

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo'== Avg call =='
sed -n '125,155p' packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs
echo'== proof-size tolerance candidates across the repository =='
rg -n -i -C 8 \
'proof.{0,40}(size|slack|limit|bound|tolerance)|(size|slack|limit|bound|tolerance).{0,40}proof|byte_slack' \
. \
-g '!Cargo.lock' \
-g '!**/node_modules/**'||trueecho'== remaining changed test lines =='
git diff --no-ext-diff --no-renames --unified=35 \
6495991c5e21f4f211ba326334ce3f717aaaf3bd \
7091b332120f35e7155a7867fa98a209c22bca6a -- \
packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/ranked_index_e2e_tests.rs \
packages/rs-drive/src/query/drive_document_ranked_query/tests.rs \
packages/rs-drive-abci/src/query/document_query/v1/tests.rs
echo'== GroveDB source tree paths at the pinned revision =='
gh api "repos/dashpay/grovedb/git/trees/c4ceac676630727d01b532314a7e488edbeaf115?recursive=1" \
--jq '.tree[] | select(.path | test("indexed.*top.*k|top.*k.*indexed"; "i")) | [.path, .url] | `@tsv`' \
||true

Length of output: 201544

---

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo'== exact proof-size tolerance identifiers =='
rg -n -C 6 --glob '*.rs' --glob '*.md' --glob 'Cargo.toml' \
'\bproof_size_slack\b|\bbyte_slack\b|\bproof.*slack\b|\bslack.*proof\b' \
packages/rs-drive \
packages/rs-drive-abci \
packages/rs-drive-proof-verifier \
packages/rs-sdk \
2>/dev/null ||trueecho'== pinned GroveDB commit =='
gh api "repos/dashpay/grovedb/commits/c4ceac676630727d01b532314a7e488edbeaf115" \
--jq '{sha: .sha, message: .commit.message, parents: [.parents[].sha]}' \
||trueecho'== final direct-read call inventory =='
rg -n -C 3 --glob '*.rs' \
'indexed_(count|sum|avg)_top_k_paginated\(' \
packages/rs-drive/src/query/drive_document_ranked_query

Length of output: 4821

---

@shumkov Fresh review of 7091b33212 is complete. I found no blocking issues.

I checked the direct Count, Sum, and Avg paginated reads. Each path returns GroveDB’s skipped value. The ABCI handler forwards that value unchanged to RankedEntries.skipped.

The Drive and ABCI tests cover the past-end case. They verify that an unproved response reports the reached population. The proved result reports the same value.

I also checked the narrow empty-tree mapping using the prior review context. It remains limited to Error::GroveDB(GroveError::CorruptedData(_)) and "Cannot create proof for empty tree".

All GroveDB dependencies now resolve to c4ceac676630727d01b532314a7e488edbeaf115. GitHub confirms that this commit follows cc7b3997 and includes counted-read regression coverage.

I did not run builds or tests because the review sandbox does not permit repository code execution.

---

🧠 Learnings used

Learnt from: QuantumExplorer
Repo: dashpay/platform PR: 4266
File: packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs:135-167
Timestamp: 2026-08-02T21:07:14.747Z
Learning: For ranked aggregate proofs, `DriveDocumentRankedQuery::execute_top_k_with_proof` in `packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs` receives Grovedb's empty-tree proof failure as `grovedb::Error::CorruptedData(String)`, not a dedicated typed error. It must preserve the generic `Error::GroveDB` mapping. `packages/rs-drive-abci/src/query/document_query/v1/mod.rs` narrowly maps that specific empty-tree message to a query validation error until Grovedb provides a typed variant.

---

Learnt from: shumkov
Repo: dashpay/platform PR: 4382
File: packages/rs-drive-abci/src/query/document_query/v1/tests.rs:3058-3090
Timestamp: 2026-08-12T19:16:31.122Z
Learning: In `packages/rs-drive-abci/src/query/document_query/v1/mod.rs`, `empty_ranking_proof_rejection` must match `"Cannot create proof for empty tree"` within a `GroveError::CorruptedData` message. GroveDB wraps Merk's marker with dynamic indexed-axis proof context, so the observed error payload is not equal to the bare marker. The mapping must remain limited to the GroveDB corrupted-data error variant; unrelated `CorruptedData`, `PathNotFound`, and non-GroveDB errors must remain unmapped.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[thepastaclaw]

Preliminary review — Codex only

The counted traversal removes offset-linear work, but positive-offset unproved reads now traverse with unsnapshotted point reads and can cross a block commit, returning mixed data under stale block metadata; this is blocking. The API request docs also still describe the skip as count-attested even though only proof results are attested, while the previous handler-local retry finding is fixed. Source: reviewer backend model gpt-5.6-sol; final verifier backend model gpt-5.6-sol; openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — general (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 1 blocking | 🟡 1 suggestion(s)

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs`:
- [BLOCKING] packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs:89-95: Counted reads lose the iterator's consistent view across a commit
For a positive offset below the population, these calls reach GroveDB's `counted_skip_page`, which loads children through successive `RefWalker::walk` point reads. The dispatcher passes `transaction = None`; at pinned GroveDB revision `c4ceac676630727d01b532314a7e488edbeaf115`, `TxRef::new` therefore creates an `OptimisticTransactionDB::transaction()` with default options, whose snapshot setting is false, and each child fetch ultimately calls `Transaction::get`. The replaced implementation collected the skipped and returned rows through one RocksDB iterator, which pins one view for that scan. If a block commits during the new descent, the resident parent can be from the old state while a fetched child comes from the new state. Merk's child load preserves the parent's recorded hash without checking it against the fetched child, and the counted traversal only cross-checks aggregate counts, so a same-population update can produce a mixed page rather than an error. In the existing interval after the GroveDB commit but before `committed_block_height_guard` is stored, the query service can also accept that page and attach metadata from the previously captured `PlatformState`. Run the counted traversal against a storage snapshot, or defer this switch until GroveDB exposes a snapshot-consistent counted-read API.
In `packages/dapi-grpc/protos/platform/v0/platform.proto`:
- [SUGGESTION] packages/dapi-grpc/protos/platform/v0/platform.proto:1128-1131: Request docs still describe the unproved skip as attested
The `offset` field applies to both proved and unproved ranked requests, but this paragraph still says the skip is count-attested and only describes GroveDB proving it. The implementation and the corrected response-field documentation distinguish the two paths: both use counted descent, but only the proved result attests the count. Update this paragraph and regenerate the Objective-C header, whose corresponding prose still says the response echoes an attested number.

[shumkov]

@thepastaclaw please re-review the current head, 6f70cc58ed.

Your blocking finding on 7091b332 — the counted descent losing the single-iterator consistent view — is addressed at the source rather than worked around here. grovedb 0100cb83 now serves the whole counted page (root, descent and collect) from one transaction raw iterator with a pinned snapshot plus the transaction overlay, the same mechanism the linear scan it replaced used, and drops RefWalker from the walk. Cost is one extra seek: deep offset 22 → 23, offset 0 unchanged at 5, past-the-end flat at 4.

Stated plainly because it bears on how much the fix is worth: the transaction-overlay half is pinned by a new grovedb test; the commit-interleave half is not deterministically testable — there is no hook to pause a fetch and force a commit mid-descent — so that half is argued from the mechanism rather than proven by a test.

Your suggestion on the offset request-side prose is also fixed, in both the proto and the Objective-C header.

[thepastaclaw]

Preliminary review — Codex only

The dependency re-pin fixes mixed node reads during descent, and the request-side documentation now correctly distinguishes counted unproved reads from cryptographically attested proved reads. However, GroveDB still discovers the secondary root key before creating the pinned iterator, so a root rotation between those steps can make a demoted node appear to be the complete ranking and silently truncate an unproved page; this remains blocking.
Source: reviewer backend model gpt-5.6-sol; final verifier backend model gpt-5.6-sol. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — general (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 1 blocking

1 carried-forward finding(s) already raised on this PR; not re-posting as new inline comments.

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs`:
- [BLOCKING] packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs:89-95: Counted reads lose the iterator's consistent view across a commit
(existing thread: https://github.com/dashpay/platform/pull/4382#discussion_r3778638772)
The new GroveDB revision pins root-node, descent, and collection reads to one raw iterator, but root-key discovery remains outside that pinned view. At GroveDB `0100cb83`, `indexed_axis_top_k_paginated_generic` calls `open_validated_axis_secondary` first; that function reads `secondary_root_key` from the parent through the snapshotless transaction. Only afterward does `counted_skip_page` create the raw iterator and re-fetch the node identified by that earlier key. If a commit rotates the secondary AVL root between these operations, the old root key can remain present in the new snapshot as a demoted child. The re-fetch therefore succeeds instead of triggering the absent-root guard, and the traversal treats that child's valid aggregate and links as the complete ranking. Internal count checks still pass because the node is a valid subtree, but the returned entries and `skipped` value are silently truncated. Root-key discovery must share the same pinned view as the secondary traversal, or the traversal must independently establish and validate the actual secondary root in its snapshot.

[QuantumExplorer]

Pin moved to grovedb develop HEAD f52adb64 (71d49c7994). Beyond the keys-only reads that close the last blocker, that revision carries the append-only storage accounting fix (dashpay/grovedb#822 via #825/#826 — write churn reported as replaced bytes, each note's permanent bytes charged once) and the keys-only projection on the unified PathQuery (#827). One adaptation: the genesis shielded seeding test passes the grove version to the commitment tree's save/commit_mmr.

Status of the shielded fee-floor tests at this pin, for the record: test_minimum_shielded_fee_covers_actual_grovedb_write_cost (transfer, unshield) are still red — estimate ~397M credits (storage 133M + processing 264M, the epoch's replaced bytes at the processing rate) vs the 131M flat fee — down from ~18.5B at the previous pin. That gap is the shielded pricing change queued after this PR (flat Shield, per-action storage constant from the corrected metering), not something this PR should absorb.
🤖 Posted by Claude Code

[thepastaclaw]

Preliminary review — Codex only

The keys-only indexed-axis APIs fix the prior snapshot-consistency blocker by returning ranked and HAVING key pairs directly from the pinned secondary view without resolving discarded primary values. The broadened GroveDB pin is nevertheless incompatible with the current shielded fee schedule: existing fee-floor tests fail, and an applied commitment-tree compaction can cost more than twice the flat fee, so changes are required before merge.

Source: reviewer backend model gpt-5.6-sol (general, security-auditor, rust-quality); final verifier backend model gpt-5.6-sol. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — security-auditor (completed), gpt-5.6-sol — rust-quality (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 1 blocking

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-drive/Cargo.toml`:
- [BLOCKING] packages/rs-drive/Cargo.toml:55-60: GroveDB pin leaves pool-paid shielded writes undercharged
The `f52adb64` re-pin brings GroveDB V4 append-only accounting and commitment-tree cost changes, but the fixed fees charged by `ShieldedTransfer` and `Unshield` remain unchanged. At this exact head, `cargo test -p drive test_minimum_shielded_fee_covers_actual_grovedb_write_cost -- --nocapture` fails both existing fee-floor tests: a one-action transfer charges 131,425,600 credits against a 397,515,800-credit estimate, while a one-action unshield charges 137,508,400 against 406,330,460. This is not merely harmless estimation slack: applying a complete one-action transfer when the note lands at the commitment-tree compaction boundary (position 2047) meters 275,228,400 credits—17,496,000 storage plus 257,732,400 processing—against the same 131,425,600-credit flat fee. `PaidFromShieldedPool` bypasses metered affordability validation, applies the writes, and books only `fees_to_add_to_pool`, so that valid transition consumes more than twice the GroveDB work it pays for, before accounting for the proof-verification work also included in the fee. Land a versioned shielded-pricing adjustment with this pin, or use a GroveDB revision that exposes the required keys-only reads without enabling accounting the current fee schedule cannot cover.

[QuantumExplorer]

315961430e — the shielded CI step got past the rs-drive fee floors at 570459d9 and then failed 5 drive-abci tests (4 × shield_from_asset_lock, 1 × shield::mainnet_halt_repro). Same root cause as the fee floors, one layer up:

What broke. Under GROVE_V4 (PV14) the pin's commitment-tree estimator is a deliberate per-append upper bound — a full epoch compaction + dense recompute charged on every note, ≈397M credits each. validate_fees_of_event v0 gates PaidFromAssetLockToPool on fee ≥ estimate, but a ShieldFromAssetLock's pool fee is the flatcompute_minimum_shielded_fee(n) + asset-lock base cost (the transform enforces lock ≥ shield + pool_fee; a client can't raise it short of over-donating into the implicit-fee cap). So the gate became a fixed comparison of two consensus constants and rejected every PV14 SFAL: "provided 212,851,200 but minimum required 793,403,740". PV13 is untouched — estimator and accounting are locked at GROVE_V3.

Fix (PV14 only, DRIVE_ABCI_METHOD_VERSIONS_V10 → validate_fees_of_event v1). The SFAL arm advertises the authoritative pool fee for gas_wanted and no longer estimate-gates — the epoch-amortized model every other pool-paid event (PaidFromShieldedPool) already follows, with the flat fee pinned against the amortized real cost by the rs-drive fee-floor tests and execution booking storage = min(actual, fee). Everything else — the transparent Shield (PaidFromAddressInputs) included — delegates to v0 unchanged: an address-funded shield must still hold ≥ the bound (929,279,120 credits at 494 notes, 1 action) and is charged the actual cost; flattening that one is the separate shielded-pricing PR.

Tests. Contrast unit test (same event: v0 rejects, v1 admits, advertised total = pool fee). mainnet_halt_repro::dropped_shield_must_not_mutate_state now runs under protocol 13 — the version mainnet was on at the halt and the last one whose locked estimator leaves the band open; its in-band headroom still reproduces the drop and the no-state-leak invariant. The PV14 band test (shield_fee_estimate_and_actual_must_not_leave_a_halting_band) re-measured locally: width 0 — first headroom that reaches execution == first that executes == 929,279,120 — so the estimator is an upper bound as designed; it stays #[ignore] purely for cost (≈60 fresh-platform runs, ~11 min). Clippy clean; the 5 formerly red tests + 26 neighbours pass locally.

[thepastaclaw]

Preliminary review — Codex only

The counted, keys-only ranked traversal removes the offset-linear read path while preserving snapshot consistency and pagination semantics. However, the GroveDB pin still leaves pool-paid shielded transitions undercharged because the new amortized tests count fee components reserved for proof and per-action computation toward GroveDB work a second time; two documentation/test-quality suggestions also remain.
Source: reviewers gpt-5.6-sol; final verifier claude-opus-4-6. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — security-auditor (completed), gpt-5.6-sol — rust-quality (completed), gpt-5.6-sol — ffi-engineer (completed), gpt-5.6-sol — general (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 1 blocking | 🟡 2 suggestion(s)

1 additional finding(s) omitted (not in diff).

1 carried-forward finding(s) already raised on this PR; not re-posting as new inline comments.

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-drive/src/state_transition_action/action_convert_to_operations/shielded/mod.rs`:
- [SUGGESTION] packages/rs-drive/src/state_transition_action/action_convert_to_operations/shielded/mod.rs:172-179: Round amortized fee-floor measurements upward
`avg_total` and `avg_storage` are used as fee floors but are calculated with truncating division. This can let the regression assertion pass when the flat fee is marginally below the exact epoch-wide cost. The current margin is large, but the test is intended to become a boundary tripwire as fee constants and GroveDB accounting evolve, so retain a conservative upper bound with ceiling division.
In `packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/v0/mod.rs`:
- [SUGGESTION] packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/v0/mod.rs:288-296: Stop telling rejected callers that OFFSET costs nothing
This user-facing validation error still says `OFFSET` costs nothing, but the implementation guarantees bounded work rather than zero work: offset zero uses the sequential fast path, a positive in-range offset performs a counted tree descent, and a past-end offset reads the root aggregate. The same obsolete shorthand remains in the ranked module heading (`offset is free`) and the test comment at `tests.rs:163`. Describe the actual guarantee—that skip work is bounded by tree depth and does not grow with the requested offset—in all three places.
In `packages/rs-drive/Cargo.toml`:
- [BLOCKING] packages/rs-drive/Cargo.toml:55-60: GroveDB pin leaves pool-paid shielded writes undercharged
(existing thread: https://github.com/dashpay/platform/pull/4382#discussion_r3835794228)
The epoch amortization addresses the once-per-2048-appends compaction spike, but the replacement assertion still compares the entire flat shielded fee with GroveDB write cost. The fee formula explicitly separates compute from storage: for one action, 100,000,000 credits price Halo 2 proof verification, 22,000,000 price per-action verification processing, and only 9,425,600 price the 344-byte storage allowance, producing the 131,425,600-credit flat fee. The measured epoch-average GroveDB work is approximately 28.05M credits, so the combined amortized resource charge is approximately 122M + 28.05M = 150.05M, about 18.62M above the charged fee. The assertion at `shielded_transfer_transition.rs:188-205` instead allows the same 122M compute component to cover both cryptographic work and database work. `PaidFromShieldedPool` and the new PV14 `PaidFromAssetLockToPool` path still apply the writes while booking no more than the flat fee. Introduce a versioned fee formula that adds the amortized GroveDB cost to the compute-only proof and per-action fees, and assert those components separately.

[QuantumExplorer]

Re-pinned to grovedb develop HEAD 753a11f1 (head 8835e1bd7a), picking up #828/#829/#830 — the fixed per-append commitment-tree cost model — plus the two #817 follow-up audits:

The pin (1e2bf4f93e). Under GROVE_V4 a CommitmentTreeInsert now meters the same figure at every position (compaction and frontier amortized into the model, dense maintenance O(height)), and the estimators price it tightly. Only fallout: CommitmentTree::open / compute_current_state_root take the grove version — threaded through the shielded snapshot paths and the genesis seeder test. GROVE_V3 (protocol 13) is locked, so mainnet replay is untouched.

Reverted the PV14 SFAL admission change (26676739e7). The v1/V10 gate-drop existed only because the previous pin's worst-case estimator (~397M/append) made flat pool fee ≥ estimate unsatisfiable. With the tight model the v0 gate passes with the untouched flat fee, so PV14 keeps DRIVE_ABCI_METHOD_VERSIONS_V9 and uniform estimate-gated admission — a smaller consensus surface than what I pushed at 315961430e. Measured at 494 notes: a transparent Shield's admission threshold and metered fee now coincide at 179,978,640 credits (band width 0, estimate == actual; at the previous pin the bound was 929,279,120 against a ~5× smaller actual). The PV13 halt-repro and the refreshed band test document both regimes.

#817 audits (8835e1bd7a). Canonical-reference axis rows bind the primary's committed value hash, so any update of a ranked-indexed document refreshes every axis row. New test per axis family (PCPSIT/PCIT/PSIT) drives both update shapes — in-place payload change and group move — through estimated and applied runs, requiring the dry-run estimate to never undercharge either fee component, then runs the canonical-row integrity walk so no stale row survives. Green at the new pin, as is the existing insert-parity test.

Everything verified locally: rs-drive ranked + shielded fee-floor suites (the epoch measurement dropped ~150 s → ~26 s thanks to #828), the full drive-abci shielded suite with the v0 gate restored (30/30), cargo check --workspace --tests, wasm32 (wasm-sdk + wasm-dpp), clippy, fmt.

[thepastaclaw]

Preliminary review — Codex only

The counted, keys-only ranked traversal appears correctly wired and preserves the intended snapshot-consistent pagination behavior. However, the GroveDB re-pin still leaves pool-paid shielded transitions undercharged because the fee-floor test lets compute-only verification fees subsidize database work; two inaccurate OFFSET-cost descriptions also remain in scope.
Source: reviewer backend model gpt-5.6-sol (general, security-auditor, rust-quality); final verifier backend model claude-opus-4-6. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — security-auditor (completed), gpt-5.6-sol — rust-quality (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 1 blocking | 🟡 1 suggestion(s)

1 additional finding(s) omitted (not in diff).

1 carried-forward finding(s) already raised on this PR; not re-posting as new inline comments.

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/v0/mod.rs`:
- [SUGGESTION] packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/v0/mod.rs:290-295: Stop telling rejected callers that OFFSET costs nothing
This user-facing validation error still says that `OFFSET` costs nothing, although the implementation guarantees bounded, offset-independent work rather than zero work. Offset zero retains the sequential fast path, a positive in-range offset performs a counted tree descent, and a past-end offset reads the root aggregate. The same inaccurate shorthand remains in the ranked module heading at `drive_document_ranked_query/mod.rs:55`, the test comment at `drive_document_ranked_query/tests.rs:163`, and the routing comment at `document_query/v1/routing.rs:282-284`. Describe the actual invariant consistently: positive skip work is bounded by tree depth and does not grow with the caller-controlled offset.
In `packages/rs-drive/Cargo.toml`:
- [BLOCKING] packages/rs-drive/Cargo.toml:55-60: GroveDB pin leaves pool-paid shielded writes undercharged
(existing thread: https://github.com/dashpay/platform/pull/4382#discussion_r3835794228)
The `753a11f1` pin stabilizes commitment-tree accounting, but it does not make the current flat fee sufficient. `compute_minimum_shielded_fee_v0` explicitly separates 100,000,000 credits of proof verification and 22,000,000 credits per action of compute-only processing from the storage allowance, which is only `344 × 27,400 = 9,425,600` credits per action. At this exact head, instrumenting the existing 2048-append measurement reports an average one-action GroveDB cost of 17,882,707 credits, including 14,337,000 storage credits. The independent one-action resource floor is therefore `100,000,000 + 22,000,000 + 17,882,707 = 139,882,707`, while the transition charges only 131,425,600 credits. The assertion at `shielded_transfer_transition.rs:188-205` passes because it compares the complete flat fee with GroveDB cost, effectively spending the 122,000,000-credit cryptographic budget on database work a second time. The shortfall scales to 135,313,712 credits for a 16-action bundle. `PaidFromShieldedPool` performs no metered affordability check and books at most the carved flat fee, so execution does not recover the difference. Add a new platform-versioned fee formula that preserves the compute-only terms and adds a GroveDB allowance calibrated to the fixed per-append model; assert the compute and database components independently.

[thepastaclaw]

Preliminary review — Codex only

The counted, keys-only ranked traversal is correctly wired and the prior OFFSET wording issue is fixed. The GroveDB re-pin still undercharges pool-paid shielded transitions because the fee-floor assertion lets compute-only verification fees subsidize independently metered database work, leaving one in-scope blocker.
Source: reviewers gpt-5.6-sol; final verifier claude-opus-4-6. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — security-auditor (completed), gpt-5.6-sol — rust-quality (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 1 blocking

1 carried-forward finding(s) already raised on this PR; not re-posting as new inline comments.

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-drive/Cargo.toml`:
- [BLOCKING] packages/rs-drive/Cargo.toml:55-60: GroveDB pin leaves pool-paid shielded writes undercharged
(existing thread: https://github.com/dashpay/platform/pull/4382#discussion_r3835794228)
The `753a11f1` re-pin changes the metered cost of commitment-tree writes without updating the flat shielded fee. `compute_minimum_shielded_fee_v0` reserves 100,000,000 credits for proof verification and 22,000,000 credits per action for compute-only verification, while allocating only `344 × (27,000 + 400) = 9,425,600` credits per action to database storage. The deterministic 2,048-append measurement at this head reports an average one-action GroveDB cost of 17,882,707 credits, including 14,337,000 storage credits. These are independent resources: the proof and per-action verification still execute in addition to the database operations, so the one-action floor is `100,000,000 + 22,000,000 + 17,882,707 = 139,882,707`, but only 131,425,600 is charged. Even ignoring GroveDB processing, booking the measured storage leaves only 117,088,600 processing credits, less than the 122,000,000 compute-only charge. The test at `shielded_transfer_transition.rs:188-205` misses this by comparing the complete flat fee with GroveDB cost, allowing the cryptographic budget to cover database work a second time; the shortfall grows to 135,313,712 credits at 16 actions. `PaidFromShieldedPool` skips metered affordability validation and splits no more than `fees_to_add_to_pool`, so execution cannot recover the deficit. Add a new platform-versioned fee formula for the affected protocol version that preserves the compute-only terms and adds a GroveDB allowance calibrated to the fixed per-append model, then assert the compute and database components independently.

[thepastaclaw]

Preliminary review — Codex only

The counted, keys-only ranked traversal appears correctly wired, but the broadened GroveDB pin still leaves pool-paid shielded transitions below the independently configured compute and database resource floor. The current fee-floor tests pass only because they compare the complete flat fee—including compute-only proof verification charges—against GroveDB work, so one in-scope blocking issue remains. Source: reviewer backend model gpt-5.6-sol (general, security-auditor, rust-quality); final verifier backend model claude-opus-4-6. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — security-auditor (completed), gpt-5.6-sol — rust-quality (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 1 blocking

1 carried-forward finding(s) already raised on this PR; not re-posting as new inline comments.

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-drive/Cargo.toml`:
- [BLOCKING] packages/rs-drive/Cargo.toml:55-60: GroveDB pin leaves pool-paid shielded writes undercharged
(existing thread: https://github.com/dashpay/platform/pull/4382#discussion_r3835794228)
The `753a11f1` re-pin changes commitment-tree write metering without updating the pool-paid flat shielded fee. `compute_shielded_verification_fee_v0` explicitly assigns 100,000,000 credits to proof verification and 22,000,000 credits per action to compute-only verification, while `compute_minimum_shielded_fee_v0` adds only `344 × (27,000 + 400) = 9,425,600` credits per action for database storage and processing. The deterministic 2,048-append measurement at this pin yields 17,882,707 credits of GroveDB work per one-action transfer, including 14,337,000 storage credits, so the independent one-action floor is `100,000,000 + 22,000,000 + 17,882,707 = 139,882,707`; the transition charges only 131,425,600. Even ignoring GroveDB processing, subtracting measured storage leaves 117,088,600 processing credits, below the independently configured 122,000,000 compute charge. The assertion in `shielded_transfer_transition.rs:188-205` compares the complete flat fee against GroveDB cost, allowing the compute-only allocation to subsidize database work and therefore missing the 8,457,107-credit one-action deficit, which grows to 135,313,712 credits at 16 actions. `validate_fees_of_event_v0` admits `PaidFromShieldedPool` without a metered affordability check, and `execute_event_v0` applies the writes while booking no more than `fees_to_add_to_pool`, so execution cannot recover the shortfall. Add a new platform-versioned fee formula that preserves the compute-only terms and separately includes the fixed per-append GroveDB allowance, then assert the compute and database components independently.

[QuantumExplorer]

On the remaining fee-floor blocker: the fix is #4467 (protocol-14 shielded fee rebalance — proof fee 40M reserved for compute, storage allowance 550 B/action sized to the measured GROVE_V4 footprint), which is now fully green with a no-blocker final review and lands first. Once it merges, this PR merges v4.2-dev and the fee-floor tests here switch to asserting the compute and database components independently against the measured fixed per-append cost — at those constants: storage component 15.07M ≥ 14.34M measured storage, and (flat − proof fee) 37.07M ≥ 17.89M measured total per action. Until then this blocker is expected to re-fire on every push; it is sequencing, not an open question.