Skip to content

Solver v2: LP core (HiGHS) + gesture-derived block model + IIS root-cause diagnosis #91

Description

@ApocDev

Replace the block solver's least-squares core with an LP (HiGHS, already bundled for cost-analysis and factory-solve) — and, at the same time, replace the planning model's per-item ceremony with a smaller, gesture-derived one. This is a redesign, not a port: no bug-for-bug parity with the current solver is required.

The block model

A block is four flat pieces of user-owned state, each set by a direct gesture:

  1. Goals, all , in both directions: produce ≥ X and consume ≥ X (sink blocks — e.g. "eat 100 tar/s" — are the same constraint with the sign flipped). No exact goals: under a minimizing objective a goal is self-tightening, binding at exactly X unless chemistry forces surplus. When one recipe makes A+B 1:2 and both have goals of 1/s, the solve runs at 1/s: A binds exactly, B's extra 1/s exports. That surplus is a fact about the factory, not an error.
  2. Chosen recipes (and machine/module choices), as today.
  3. A made set — item names the block has claimed in-block production for. made items get net ≥ 0 (production covers consumption; surplus exports; imports forbidden). Items not in made are free: consumption imports, incidental byproduct production offsets the import (10/s needed, 0.02/s byproduct → net import 9.98/s — the byproduct is never scaled up to cover demand), surplus exports.
  4. Pins, per recipe row, two flavors on the same rate variable:
    • Fixed count / share (=): "5 buildings, always running" or "50% of tar production (or of the remainder after fixed pins)". Supply-push — this is how byproducts get routed to in-block consumers (pin the flare stack at 100% of remaining tar → export eliminated) and how oil-style fan-outs are declared (50% to cracking, 25% to lubricant, remainder exports). Resolution order: absolute pins subtract first, shares apply to the remainder, leftover exports.
    • Built count (): a capacity ceiling — "12 assemblers exist; fit inside." Demand-driven up to the cap; when the cap binds, the diagnosis reports the shortfall in buildings. (Later: auto-populate from the bridge's live state; pairs with Live-data alerts: flag blocks running below plan #88.)

Objective: minimize machine-seconds (recipe-cost tie-break), with a tiny epsilon cost on every recipe so zero-cost synthetic recipes (burning/venting) can't create non-unique optima. Same input → identical output, always.

What this removes. The auto/import/export/balance disposition cycle and its alt-click UI are deleted, not migrated:

  • import/export overrides are today literally one solver behavior with two labels (both just unlink the item); the rare legitimate use ("I make a little steam but feed it externally") is now "item not in made".
  • balance existed to protect equations from the least-squares loop-cutter and to keep goal-unreachable recipes from being pinned to 0. Both jobs vanish: the LP never drops constraints (infeasibility gets diagnosed, not silently relaxed), and demand propagates through constraints rather than goal-reachability, so recipes run exactly as much as their consumers need with no pinning.
  • Statuses relaxed and underdetermined disappear: the LP solves exactly or fails with a diagnosis, and the objective makes every feasible block determined.
  • The alphabetical autoFreed loop-cutting is gone entirely — an unclosed recycle loop is an infeasibility with a named cause, never a silent guess.

Migration for existing blocks: import/export dispositions → item absent from made; items that were auto-balanced (produced+consumed, no override) → in made; balance overrides → in made (the strictness they wanted is now the default). One-off side-by-side report over the real project blocks during development, reviewed once for sensibility — intentional improvements accepted without ceremony; no permanent parity fixture.

How made gets set (no inference beyond demand-matching): setting a goal marks the goal item; adding a producer via an item chip marks that item; a recipe added from search marks whichever of its products the block already demands (goal or consumed) — usually exactly one; if nothing matches, it marks nothing and the recipe sits at an honest rate 0 with a "click the output you want this to supply" flag. Never guess from main_product (Py's is frequently water when you want the soil). Every mark is visible on the chip (existing link colors) and one click clears it.

Root-cause diagnosis (the headline feature)

YAFC's infeasibility handling smears: one cost-weighted slack pass, then a binary warning flag on every recipe touching the loop — no quantities, no ranking, no pointer to what the user actually did. Two properties let us do better:

  • Provenance: every LP row traces to one user gesture (a goal, a made mark, a pin, a recipe's stoichiometry). Acceptance rule: no diagnostic may name anything the user can't click.
  • Scale: blocks run to hundreds of recipes, so root-cause isolation must not be brute force over every constraint. On infeasibility, compute an IIS (irreducible infeasible subsystem — the minimal constraint set that is jointly infeasible; remove any member and it solves) in two stages: an elastic pass first (slack on every constraint, minimize total slack — the handful of slack-positive constraints are the only candidates), then deletion filtering within that candidate set only. That keeps diagnosis at a few dozen re-solves of an already-factored LP regardless of block size. Try HiGHS's native IIS first; the elastic+deletion pipeline is the guaranteed fallback. Diagnosis may also run async after the fast failure — the solve result (infeasible) returns immediately, cards follow a beat later.

Protocol: solve fails → extract IIS → present as one card naming the specific gestures in conflict, with slack magnitudes ("these can't all hold: the 1/s science goal, iron-plate's made mark, the 12-building cap on smelting — short 3.2/s ≙ 3 buildings") and one-click fixes (unmark, unpin, adjust goal, add producer). Relax that IIS, re-solve, extract the next → independent problems arrive as separate cards. The same machinery powers suggestions in feasible-but-suspicious cases ("iron imports 9.98/s but an in-block producer exists — mark it made and scale?").

Result statuses collapse to solved | infeasible (with IIS cards) | error (numerical failure is a distinct user-facing state, never a throw).

Acceptance criteria

  • Correctness by construction: rates ≥ 0 always (negative-rate detection deleted, impossible by construction); goals ; made items net ≥ 0; free items unconstrained; pins as above. An independent mass-conservation post-check (not the solver's own residual) asserts every item's flows balance on every solve in dev/tests — coefficient construction, not the solver, is the historical bug source (Productivity math treats ignored_by_productivity as all-or-nothing #93).
  • Determinism: identical input → identical output across runs and machines; a test asserts a degenerate block (two unpinned producers of one item) solves identically twice.
  • Scale invariance: test blocks at 5,000,000/s (TW-scale energy pseudo-fluid) and 0.001/s both solve with correct flows; tolerances relative, units normalized internally; no absolute-residual thresholds (v1's TOL = 1e-4 is the anti-pattern).
  • Diagnosis contract: IIS cards as above; provenance rule enforced; multiple independent causes reported separately; every card carries quantities and actionable fixes.
  • Performance: warm solver instance (WASM instantiated once, solves run per keystroke) plus basis warm-starting — a per-keystroke edit changes one coefficient or bound, and re-solving from the previous basis is near-free while a cold solve of a big block is not. Budgets: a synthetic hundreds-of-recipes block (not just the largest real one today) cold-solves ≤ 50ms and warm-resolves ≤ 10ms; full-factory resolve within current budget; IIS diagnosis on the big block ≤ 250ms.
  • Worker thread: dropped after measurement (decided 2026-07-03). The core was benchmarked at 400 recipes on a warm WASM instance: 13ms solve, 26ms IIS diagnosis — 4–10× inside the budgets the worker was specced to protect against. Sub-frame stalls on the Node event loop are harmless to concurrent requests and the UDP bridge, so the worker_threads host, basis warm-starting, and latest-wins coalescing are all deferred until a real block ever measures slow (the LP-text architecture doesn't preclude any of them). The budgets above stay as acceptance criteria; a perf regression is what would resurrect the worker.
  • Semantics preserved from v1's surrounding layers: coefficient construction (effects, research productivity, reactor layouts, self-fueling burners, energy pseudo-fluids), expected-value semantics for probabilistic/min-max products, relevance information kept as a diagnostic ("this recipe solves to 0") rather than a constraint.
  • Forward hooks, build-ready not built: integer machine counts (Whole-machine mode: optional integer building counts (MIP solve) #98) are the MIP sibling of built-count ceilings (solver chooses integer count ≤ built); good identity stays pluggable so Solver: treat fluid temperature variants as distinct goods (YAFC-style) #110's (fluid, temperature) variants and range-pooling slot in without re-plumbing; goalless blocks driven purely by fixed pins ("here are my 5 refineries — show me the flows") work, and a block with neither goals nor pins shows an empty-state prompt instead of an underdetermined shrug.
  • Solve-result persistence extends the existing cache (block_flows/block_machines/blocks.status + data_fingerprint, written atomically with the block config on save — keep that write-through invariant; only the solve path may write these): add per-recipe rates/machine counts (so what-if, coherence, and agent tools inspect block internals with zero solves), cache the IIS cards alongside status: infeasible (the sidebar and agents can say why without re-solving), and fold a solver-version stamp into the fingerprint so the v1→v2 migration (and any future solver fix) invalidates caches naturally on first touch. Determinism makes the cache a pure memo.
  • All existing unit/e2e suites pass (updated where semantics intentionally changed), and the UI renders every new shape (chips from made/free, pin badges distinguishing fixed vs built, fan-out breakdown on split items — "20 pinned · 20 (25%) · 60 export", IIS cards) with no surface left reading a removed field (relaxed, underdetermined, autoFreed, dispositions).

Blocks #98; the natural home for #110's full model. A foundation #76 (sub-blocks v2) builds on.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: plannerProduction block / planning modelenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions