Skip to content

fix(antd): public uploads count the DataMap chunk; core not-found is 404 - #270

Merged
Nic-dorman merged 1 commit into
mainfrom
nic/v2-1261-public-upload-counts
Sep 17, 2026
Merged

Nic-dorman merged 1 commit into
mainfrom
nic/v2-1261-public-upload-counts

Conversation

@Nic-dorman

Copy link
Copy Markdown
Member

Why

Found while devnet-proving @withautonomi/antd@0.1.0: for a 512 KiB public file upload antd reported chunks_stored=3 and the 3-chunk cost, while POST /v1/files/cost?is_public=true and the FFI both said 4 chunks / 4-chunk cost for the same file. The DataMap chunk was stored and paid for but not reported. Separately, fetching a public address that was never stored returned 500 instead of 404, so SDKs raised InternalError instead of NotFoundError.

What

  • Files (REST + gRPC public put): switch to ant-core's file_upload_public_with_mode, which folds the serialized DataMap into the same payment batch (verified in ant-core: spill.push(&serialized) before the chunk count is taken). Address comes from data_map_address. chunks_stored, storage_cost_atto, gas_cost_wei now include the DataMap chunk and agree with the estimate and the FFI.
  • Data (REST + gRPC public put): ant-core has no single public data upload, so the daemon still calls data_map_store itself and now reports chunks_stored + 1 via a documented helper, matching adjust_for_public_upload.
  • Not-found: AntdError::from_core gains an arm for Error::NotFoundAntdError::NotFound, which already maps to HTTP 404 / gRPC NOT_FOUND.

Tests

  • public_upload_chunks_stored counts the DataMap chunk and equals the estimate's chunk count for the same input.
  • core NotFoundAntdError::NotFoundNOT_FOUND code → HTTP 404 → gRPC NotFound; other core errors still map to Internal.
  • Locally: cargo fmt --check, cargo clippy --all-targets --all-features --locked -D warnings, cargo test --locked → 76 passed.

Not in this PR

  • The external-signer finalize path (store_data_map) and the FFI's own data_put_public share the data-path shape and are unchanged; noted on the ticket.
  • No response-shape change: same fields, corrected values. Docs/SDKs need no edits.

🤖 Generated with Claude Code

…osts; core not-found maps to 404

Public upload responses (REST + gRPC) were built only from the data-chunk
upload result, so `chunks_stored` and, for files, `storage_cost_atto` /
`gas_cost_wei` excluded the DataMap chunk that the same request stored and
paid for. `POST /v1/files/cost` with `is_public=true` and the FFI both count
it, so estimate != actual (4 vs 3 for a 512 KiB file) on every public upload.

- Files (REST `file_put_public`, gRPC `FileService.PutPublic`): use ant-core's
  `file_upload_public_with_mode`, which pushes the serialized DataMap into the
  same payment batch, and take the address from `data_map_address`. Count and
  costs now include the DataMap chunk and agree with the estimate and the FFI.
- Data (REST `data_put_public`, gRPC `DataService.PutPublic`): ant-core has no
  single public data upload, so the daemon still stores the DataMap itself;
  report `public_upload_chunks_stored(n) = n + 1` (new helper in types.rs)
  so the count matches `adjust_for_public_upload`.
- `AntdError::from_core` maps `ant_core::data::Error::NotFound` to
  `AntdError::NotFound`: a public fetch at an unknown address is now HTTP
  404 / gRPC NOT_FOUND instead of 500 / INTERNAL, so SDKs raise
  NotFoundError as documented.

Tests: helper count vs estimate; core NotFound -> NotFound -> 404 -> gRPC
NotFound; unknown core errors still Internal. fmt/clippy -D warnings/76 tests
green locally.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Nic-dorman

Copy link
Copy Markdown
Member Author

Devnet proof on dev2 (antd built from fbc6c7b, ant-devnet small preset, antd-js 0.1.0 client):

antd dataPutPublic 512 KiB -> chunksStored=4 in 518 ms
antd filePutPublic 512 KiB -> chunksStored=4 storageCostAtto=46875000000000000 (fileCost estimate: chunkCount=4 cost=46875000000000000) in 287 ms
negative control: random address -> NotFoundError: Record not found: DataMap chunk not found at … (31 ms)
GET /v1/data/public/<random> -> HTTP 404 {"code":"NOT_FOUND"}

Before this PR the same script reported chunksStored=3 storageCostAtto=35156250000000000 against the 4-chunk estimate, and the negative control was a 500 InternalError.

@dirvine dirvine left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hermes Agent Review — Approve ✅

Reviewed @ head fbc6c7b (CI green).

Verified the core ant-core contract directly (ant-client @ ant-cli-v0.3.5): file_upload_public_with_mode pushes the serialized DataMap onto the spill before chunk_count = spill.len() and returns data_map_address, so the single payment batch genuinely includes the DataMap chunk. Switching REST + gRPC public file uploads to it makes chunks_stored/costs agree with the estimate and the FFI.

  • Data public path reports chunks_stored + 1 via public_upload_chunks_stored, matching adjust_for_public_upload. Consistent.
  • Error::NotFound(String) is a real ant_core::data::Error variant; mapping to AntdError::NotFound → HTTP 404 / gRPC NOT_FOUND is correct, catch-all still maps other core errors to Internal.
  • Tests are solid (chunk counting incl. usize::MAX saturating, 404/gRPC not-found, unknown-error fallback).

No blockers. Note (non-blocking): gRPC data public path relies on the same +1 helper as REST; the file-path fix is the more thorough one (single payment batch vs two).

@Nic-dorman
Nic-dorman merged commit 247a491 into main Sep 17, 2026
4 checks passed
@Nic-dorman
Nic-dorman deleted the nic/v2-1261-public-upload-counts branch September 17, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants