Skip to content

Repository files navigation

@superbuilders/issue

The one law for running work: an op issues the moment its operands are ready and a port is free. Pools, pumps, queues, and barriers are all this law, hand-rolled — and every hand-rolling picks its own default bound, its own failure routing, its own place to hide an await. This package exists so none of those decisions exist.

The framing is a CPU's out-of-order core, because the defects it kills all have hardware names:

Defect in async codeHardware nameWhat makes it unrepresentable here
await b; await c where c never needed bFalse dependencyEdges are declared data deps; a diamond runs as a diamond because you wrote a diamond
concurrency ?? items.lengthNo structural hazard trackingport is required on every op; unboundedness is written (port.open), never defaulted
Logging/fs inside the dispatch loopA store on the issue pathThe engine owns no await between an op settling and its dependents issuing; telemetry is pull
One Promise.race pump for N workersA 1-wide issue stageThere is no pump; completions issue their dependents directly
A flush promise nobody handlesImprecise exceptionsEvery outcome is a settled value: retired / faulted / squashed, with the cause chain named
"It was slow but we don't know where"No performance countersEvery settlement carries its blocked/queue/run trace; summarize reads the bottleneck back
import*asissuefrom"@superbuilders/issue"constlean=issue.port(64)constcopy=issue.op({port: lean,work: (deps,opts)=>runCopy(source,opts)})constimages=issue.op({deps: { copy },port: issue.port.open,work: ({ copy },opts)=>renderFrames(copy,opts)})consttts=issue.op({deps: { copy },port: issue.port.open,work: ({ copy },opts)=>synthesize(copy.spokenText,opts)})constplan=issue.op({deps: { images, tts },port: issue.port.open,work: ({ images, tts },opts)=>assemble(images,tts,opts)})constsettled=awaitissue.retire({ copy, images, tts, plan },opts)

images and tts run concurrently because nothing says otherwise — the dependency structure IS the schedule. Wiring the wrong producer to a consumer is a type error: deps: { copy } gives work a { copy: CopyResult } first argument by inference.

Install

pnpm add @superbuilders/issue

ESM only, runtime-neutral (no node: imports — works in the browser). Dependencies: @superbuilders/errors and @superbuilders/limiter, nothing else.

op — pure descriptions

issue.op({label?: string // defaults to the graph key at retiredeps?: {[name]: Op}// value references — cycles are unrepresentableport: Port// REQUIRED; unboundedness is issue.port.open, written visiblywork: (deps,opts)=>Promise<T>}): Op<T,TOpts>

Constructing an op runs nothing. Deps are value references, so a cycle cannot be written: a reference can only point to an already-constructed op. An op is single-use — a second retire sharing it throws ErrOpReused.

Structural, not branded

There is no brand, no phantom field, no nominal-typing emulation anywhere in this package. Op<T, TOpts> is a plain structural type whose parameters live in REAL positions — T in the work function's return type, TOpts in its opts parameter — so infer recovers them (OpValue, OpOpts), variance is native (values covariant, requirements contravariant), and any object matching the shape IS an op:

constliteral: issue.Op<string>={label: "literal",deps: {},port: issue.port.open,work: async()=>"structural",claimed: false}awaitissue.retire({ literal },opts)// retires; there is a falsifier proving it

The runtime guard (isOpShape) checks structure, not identity. Deps records are typed Op<unknown, never> — the contravariant top type — so they hold ops with any opts requirement.

The graph's context is a compile-time fact

An op CARRIES its opts requirement, requirements PROPAGATE from deps to dependents at construction (op() returns Op<T, TOpts & GraphOpts<D>>), and retire refuses an opts bag that doesn't satisfy the record:

constneedy=issue.op({port: issue.port.open,work: async(deps,opts: {signal: AbortSignal;logger: Logger})=>{}})constroot=issue.op({deps: { needy },port: issue.port.open,work: async({ needy })=>needy+1})awaitissue.retire({ root },{ signal })// @ts-expect-error — the logger requirement propagatedawaitissue.retire({ root },{ signal, logger })// compiles

One limitation, deliberately documented instead of papered over: propagation flows through TYPED constructions. Let inference carry deps types — never hand-annotate a deps record as DepsShape; that widening erases the requirements this machinery exists to carry.

port — structural hazards as values

issue.port(max)// bounded, FIFO, backed by limiter.concurrencyissue.port.open// THE explicit unbounded portissue.port.from(limiter)// adopt an existing ConcurrencyLimiter singleton — one law, one gate, no double-bounding

Bounds come only from ports, and the house posture is NO LIMITS: a numeric bound exists only where a strictly documented bottleneck forces it (a provider ceiling, N resident processes, a connection cap, cores). "Prudence" is not a bottleneck.

retire — run the closure, settle everything

constsettled=awaitissue.retire({ plan, index },opts)// { plan: Settled<Plan>, index: Settled<void> }typeSettled<T>=|{status: "retired";value: T;timing: SettledTiming}|{status: "faulted";error: Error;timing: SettledTiming}// the op's own throw, raw, never wrapped|{status: "squashed";cause: Error;timing: SettledTiming}// upstream fault (wrapped with the faulting op's label) or the abort reason

Precise exceptions. A fault squashes exactly the transitive dependents — they never issue, their ports never see them — and siblings retire undisturbed. The architectural state at failure is fully named: what retired, what faulted, what squashed and why (errors.is matches the root cause through the wrap chain). retire never rejects on op failure; it rejects only for construction misuse (ErrNotAnOp, ErrOpReused).

Abort. The limiter/retry contract, verbatim: waiting ops squash with the signal's own reason when that is an Error (else a wrap of ErrAborted); queued entries dequeue without starting; running ops see the signal through their opts and settle as their work decides.

Reading the trace

Every settlement carries timing — stamped by the engine at moments it already passes through, zero configuration, zero overhead beyond a few performance.now() calls. The three-way decomposition IS the bottleneck analysis:

FieldHazardReading
blockedMs (t0→ready)Data hazardWaiting on operands — the bottleneck is upstream; follow path
queueMs (ready→issued)Structural hazardWaiting on a port — the bound is contended, or (per the no-limits posture) wrong
runMs (issued→settled)The workThe stage itself is the cost
constsettled=awaitissue.retire(graph,opts)opts.logger.info(issue.summarize(settled),"build graph settled")// { wallMs: 241_800, workMs: 1_262_100, parallelism: 5.2,// path: [{ label: "copy", runMs: 41_200 }, { label: "images", runMs: 183_400 }, …],// byQueueMs: [{ label: "tts", queueMs: 40_100 }, …] }

criticalPath walks backward from the latest-settling op through each op's latest-settling dep — the chain that WAS the wall clock. summarize adds parallelism (workMs / wallMs — ≈1 means your graph is a chain in practice) and the full run/queue rankings (no silent top-N; sub-millisecond queues are filtered as scheduling noise). Buffer flush settlements reuse the same vocabulary: blockedMs is the coalescing wait, runMs the flush. Telemetry stays pull — the trace is data in the value you already receive; live pressure while running is scoreboard.

all — the pool killer

constresults=awaitissue.all(items,{ port,work: (item,index,opts)=>f(item,opts)},opts)// Settled<R>[] — same length, same order as items

Implemented on op/retire — there is no second scheduler. There is no fail-fast mode and therefore no orphaned-worker state (the classic pool defect where the first rejection returns to the caller while surviving workers keep consuming the iterator): every item runs to settlement or squashes on abort, and the caller narrows.

buffer — the store buffer

constbuf=issue.buffer({ label,maxItems?,maxWaitMs?,flush: (items,opts)=>Promise<R>},opts)buf.push(item)// synchronous — stores never sit on the issue pathbuf.state// { pushed, flushed, flushing, pending, failures }constall=awaitbuf.drain()// terminal, idempotent; Settled<R>[] per flush, timings included

One single-flight flush, triggered by size, timer, or drain; pushes during a flush accumulate and coalesce into exactly one follow-up. Failure routing is settled: a flush rejection lands in state.failures and the drain results — an unhandled rejection is unrepresentable — and a failed flush does not poison the buffer. Whether failure latches is caller policy, written in the flush closure. opts binds at construction because flushes fire long before drain.

scoreboard — live occupancy

issue.scoreboard({ lean, dbWrite })// { lean: { active, pending, max }, … }

The pull surface for a RUNNING graph, deliberately paired with the post-hoc trace: scoreboard answers "what is contended right now", the settled timings answer "what was the bottleneck". The dispatch path carries no logger, no callbacks, no awaits — wait-time logging belongs to the gate that owns the resource (construct your limiter with its own onStart and adopt it via port.from).

Vercel Queues, Workflow, and where this package ends

issue and Vercel Queues are ORTHOGONAL layers of the same story — the out-of-order core and the durable interconnect — and the orthogonality is a feature to protect, not a gap to bridge:

@superbuilders/issueVercel Queues
Lifetimeone process rundurable; survives crash and deploy
Dispatch latencymicrotasksnetwork round trips, redelivery schedules
Data flow between stepsfull in-memory values by referencea JSON message payload
Failure modelprecise exceptions, settled map, squashat-least-once redelivery, poison-ack
Scale unitcores + provider ceilings in one processhorizontal across function instances

Duplication exists only if issue pretends to be durable (it cannot and will not) or Queues is used for in-process fan-out (a serialization boundary and tens of milliseconds per edge on graphs whose edges carry megabytes). Three composition patterns, all application-side:

1. A graph per message. A queue consumer executes one message's internal DAG in-process — Queues owns durability and redelivery AROUND the unit of work; issue owns parallelism INSIDE it:

asyncfunctionconsume(message: Job,opts: Opts){constgraph=buildGraphFor(message)// parse → fan out → join → writeconstsettled=awaitissue.retire(graph,opts)if(settled.write.status!=="retired"){throwsettled.write.status==="faulted" ? settled.write.error : settled.write.cause}// throw ⇒ the queue redelivers}

2. A graph that enqueues. A leaf op — or better, an issue.buffer docket, since topic.send is exactly a coalescible sink — publishes durable follow-up work:

constfollowUps=issue.buffer({label: "follow-ups",maxWaitMs: 5_000,flush: (msgs,opts)=>topic.send(msgs,opts)},opts)

3. The durable-DAG boundary. If the STEPS themselves must survive a crash, that is Vercel Workflow's product (durable step execution) — not Queues, and not this package. The bridge is one workflow step = one issue subgraph: the workflow owns durability and resume, issue owns intra-step parallelism. This package will not grow durability, on purpose: its edges carry arbitrary in-memory values by reference, and durability means serializing every edge — a different contract with a different cost model. Pipelines whose stages are idempotent and hash-cached re-derive more cheaply than they checkpoint.

The neutrality law. The package imports errors and limiter, nothing else, ever — which is precisely why it runs unchanged in a TUI, a worker thread, a queue consumer, and a workflow step.

The falsifier discipline

Every law above has a named test that tries to kill it, not an example that happens to pass: F1 max-of-legs; F2 no head-of-line blocking; F3 squash precision; F4 issue-path purity; F5 the abort contract; F6 single-use; F7 open-port visibility; F8 fault isolation across ports; F9 all alignment with no orphans; F10 buffer rejections settle, never escape; F11 coalescing; F12 push purity; F13 drain terminality; F14 timer single-shot; F15 the timing decomposition identities; F16 queueMs is exactly the port wait; F17/F17b criticalPath on the diamond and the chain; F18 buffer flush timing; F-structural (a raw literal is an op); F-opts (the exact opts reference reaches every work); plus the compile-time probe file issue.test-d.ts (positive, negative, propagation, empty, intersection, extraction). The suite runs with an unhandledRejection listener installed and asserts zero events across the whole run.

Design notes

  • Prior art, deliberately narrowed: Tomasulo's algorithm and the CDC 6600 scoreboard are the semantic ancestors (issue on operand readiness + functional-unit availability, precise exceptions at retire). Promise-DAG libraries are the option-laden thing this is not — this package's point is to have no options.
  • Why structural over branded: a brand fights the language's native model to forbid values that would work. The type parameters that matter live in real positions where inference and variance are free, and the runtime checks structure — so the type system tells the truth about what the engine actually requires.
  • Why no fail-fast in all: fail-fast is where orphaned workers come from. If you want to stop the world, abort the signal — that is what it is for.
  • Why retire never rejects on op failure: a rejection collapses the whole settled map into one error, which is exactly the imprecise-exception disease. The map is the answer.

License

0BSD © Bjorn Pagen

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages