Skip to content

feat(storage): path-build link store for per-path builds - #502

Open
behinddwalls wants to merge 1 commit into
preetam/speculation-path-storefrom
preetam/speculation-path-build-store
Open

feat(storage): path-build link store for per-path builds#502
behinddwalls wants to merge 1 commit into
preetam/speculation-path-storefrom
preetam/speculation-path-build-store

Conversation

@behinddwalls

@behinddwallsbehinddwalls commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

Speculation needs one build per path, and something must record which build belongs to which path attempt. The runner chooses the build ID, so a caller holding a path cannot derive it; a keyed record is the key-value contract's mechanism for a reverse lookup.

What?

entity.PathBuild is new: the (pathID, attempt) -> buildID link, with storage.PathBuildStore, a MySQL implementation, schema, and mocks.

The record is write-once: created only once the runner has named the build, never updated. Absent means no build is recorded for the attempt; present names its build permanently, and a retried path is a new attempt under a different key. Because creation is the only write, the first insert also decides concurrent dispatches for the same attempt — ErrAlreadyExists tells the loser the attempt's build is someone else's. There is deliberately no reservation state and no version column: the stage that watches builds (later in this stack) stops any build whose path no longer wants it, so no reader needs to distinguish "idle" from "mid-dispatch".

entity.Build gains PathID and Attempt, and entity.SpeculationPath gains Base() — the dependencies the path assumes will succeed — so every consumer derives the base one way.

Test Plan

bazel test //submitqueue/entity/... //submitqueue/extension/storage/...

bazel test //test/integration/submitqueue/extension/storage/mysql:go_default_test — an attempt resolving to its build, ErrNotFound when undispatched, and a duplicate Create refused so the first link stands.

make fmt, make gazelle, make mocks

Stack

  1. docs(rfc): explain best-first speculation generation #513
  2. feat(speculation): generator contract and bestfirst impl #446
  3. feat(speculation): allocator contract and sticky impl #450
  4. feat(speculation): standard composed speculator #451
  5. feat(storage): speculation path set store #501
  6. @ feat(storage): path-build link store for per-path builds #502
  7. feat(orchestrator): dispatch builds per speculation path #503
  8. feat(orchestrator): poll builds and stop the ones nothing wants #504
  9. fix(orchestrator): mint distinct message IDs for cancel re-publishes #505
  10. feat(orchestrator): re-plan the queue from the Speculator each run #506
  11. feat(orchestrator)!: finalize batches from their speculation paths #507
  12. feat(orchestrator): compose per-queue speculators and turn speculation on #508

@behinddwalls
behinddwalls marked this pull request as ready for review August 3, 2026 17:11
@behinddwalls
behinddwalls requested review from a team and sbalabanov as code ownersAugust 3, 2026 17:11
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-build-store branch from 5692021 to 4443963CompareAugust 3, 2026 23:03
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-build-store branch from 4443963 to 471898dCompareAugust 4, 2026 01:58
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-build-store branch from 471898d to edc7d3aCompareAugust 4, 2026 05:11
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-build-store branch from edc7d3a to 28db660CompareAugust 5, 2026 03:15
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-build-store branch from 28db660 to b528041CompareAugust 5, 2026 03:26
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-build-store branch from b528041 to 51fe6efCompareAugust 5, 2026 03:37
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-build-store branch from 51fe6ef to 226cfe9CompareAugust 5, 2026 03:43
@behinddwalls
behinddwallsforce-pushed the preetam/speculation-path-build-store branch from 226cfe9 to 994904dCompareAugust 5, 2026 04:11
## Summary
### Why?
Speculation needs one build per *path*, and something must record which build belongs to which path attempt. The runner chooses the build ID, so a caller holding a path cannot derive it; a keyed record is the key-value contract's mechanism for a reverse lookup.
### What?
`entity.PathBuild` is new: the `(pathID, attempt) -> buildID` link, with `storage.PathBuildStore`, a MySQL implementation, schema, and mocks.
The record is write-once: created only once the runner has named the build, never updated. Absent means no build is recorded for the attempt; present names its build permanently, and a retried path is a new attempt under a different key. Because creation is the only write, the first insert also decides concurrent dispatches for the same attempt — `ErrAlreadyExists` tells the loser the attempt's build is someone else's. There is deliberately no reservation state and no version column: the stage that watches builds (later in this stack) stops any build whose path no longer wants it, so no reader needs to distinguish "idle" from "mid-dispatch".
`entity.Build` gains `PathID` and `Attempt`, and `entity.SpeculationPath` gains `Base()` — the dependencies the path assumes will succeed — so every consumer derives the base one way.
## Test Plan
✅ `bazel test //submitqueue/entity/... //submitqueue/extension/storage/...`
✅ `bazel test //test/integration/submitqueue/extension/storage/mysql:go_default_test` — an attempt resolving to its build, `ErrNotFound` when undispatched, and a duplicate `Create` refused so the first link stands.
✅ `make fmt`, `make gazelle`, `make mocks`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@behinddwalls