Skip to content

feat(storage): speculation path set store - #501

Open
behinddwalls wants to merge 1 commit into
preetam/speculation-standardfrom
preetam/speculation-path-store
Open

feat(storage): speculation path set store#501
behinddwalls wants to merge 1 commit into
preetam/speculation-standardfrom
preetam/speculation-path-store

Conversation

@behinddwalls

@behinddwallsbehinddwalls commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

The speculate run reads every head's path set at the start of a run and persists what it funds at the end. entity.SpeculationPathSet exists, but nothing can store one.

What?

Adds storage.SpeculationPathSetStore, keyed on the head batch ID, with a MySQL implementation, schema, mocks, and contract tests.

The mutation is a conditional whole-item put — Update replaces the set, guarded on version — because a conditional put on a key is the primitive every backend offers directly; a field-level update is the one shape non-SQL backends would have to emulate with a read-modify-write. Version arguments are explicit and the entity's own Version field is ignored, per the storage README.

No secondary index: callers that want a queue's live sets enumerate the heads from the batch listing they already hold and read each set by key.

Test Plan

bazel test //submitqueue/extension/storage/... — sqlmock coverage of get/create/update and every error contract, plus a pin that Update ignores the entity's Version field.

bazel test //test/integration/submitqueue/extension/storage/mysql:go_default_test — against real MySQL: round-trip (each stored ID still hashes its stored path), missing head, duplicate create, and a compare-and-swap race where the loser must not restore the path the winner dropped.

make fmt, make gazelle, make mocks

Stack

  1. docs(rfc): explain best-first speculation generation #513
  2. feat(speculation): generator contract and bestfirst impl #446
  3. feat(speculation): allocator contract and sticky impl #450
  4. feat(speculation): standard composed speculator #451
  5. @ feat(storage): speculation path set store #501
  6. feat(storage): path-build link store for per-path builds #502
  7. feat(orchestrator): dispatch builds per speculation path #503
  8. feat(orchestrator): poll builds and stop the ones nothing wants #504
  9. fix(orchestrator): mint distinct message IDs for cancel re-publishes #505
  10. feat(orchestrator): re-plan the queue from the Speculator each run #506
  11. feat(orchestrator)!: finalize batches from their speculation paths #507
  12. feat(orchestrator): compose per-queue speculators and turn speculation on #508

@behinddwalls
behinddwalls marked this pull request as ready for review August 3, 2026 16:46
@behinddwalls
behinddwalls requested review from a team and sbalabanov as code ownersAugust 3, 2026 16:46
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-store branch 2 times, most recently from c56ac02 to e481adfCompareAugust 4, 2026 01:58
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-store branch from e481adf to c4f175eCompareAugust 4, 2026 05:11
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-store branch from c4f175e to 5483221CompareAugust 5, 2026 03:15
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-store branch from 5483221 to cdce63dCompareAugust 5, 2026 03:26
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-store branch from cdce63d to 6989aeeCompareAugust 5, 2026 03:37
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-store branch from 6989aee to 3f6c278CompareAugust 5, 2026 03:43
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-store branch from 3f6c278 to f9aa731CompareAugust 5, 2026 04:11
## Summary
### Why?
The speculate run reads every head's path set at the start of a run and persists what it funds at the end. `entity.SpeculationPathSet` exists, but nothing can store one.
### What?
Adds `storage.SpeculationPathSetStore`, keyed on the head batch ID, with a MySQL implementation, schema, mocks, and contract tests.
The mutation is a conditional whole-item put — `Update` replaces the set, guarded on version — because a conditional put on a key is the primitive every backend offers directly; a field-level update is the one shape non-SQL backends would have to emulate with a read-modify-write. Version arguments are explicit and the entity's own `Version` field is ignored, per the storage README.
No secondary index: callers that want a queue's live sets enumerate the heads from the batch listing they already hold and read each set by key.
## Test Plan
✅ `bazel test //submitqueue/extension/storage/...` — sqlmock coverage of get/create/update and every error contract, plus a pin that `Update` ignores the entity's `Version` field.
✅ `bazel test //test/integration/submitqueue/extension/storage/mysql:go_default_test` — against real MySQL: round-trip (each stored ID still hashes its stored path), missing head, duplicate create, and a compare-and-swap race where the loser must not restore the path the winner dropped.
✅ `make fmt`, `make gazelle`, `make mocks`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@behinddwalls