Reasoning through conflicting preclinical toxicity evidence - transparently, with the human experts still making the decision.
Pfizer Digital & Technology Hackathon 2026 · Problem Statement 3, Computational Pre-Clinical Drug Development Team BU 1 - Jack He, Andres Lopez, Jose Cruz-Lopez
A preclinical safety lead deciding whether a compound advances is rarely short of predictions. They are short of a defensible way to reconcile the ones that disagree.
A QSAR model says the structure looks hepatotoxic. A cytotoxicity assay says the cells survived. A transporter assay says bile-salt export is inhibited. A rodent study says nothing happened. These are not four opinions of equal standing - some measure a mechanism, some only correlate with structure; some were run at clinically relevant exposure, some were not. Today that reconciliation happens in a scientist's head and in a meeting, and the reasoning that produced the answer is not recoverable six months later when a regulator, or a colleague, asks why.
We help preclinical safety leads reason through conflicting toxicity evidence so they can make consistent, defensible go or no-go decisions.
ARBITER takes the conflicting evidence for a compound and produces a position - advance, do not advance, or abstain - together with the argument that led there, the evidence that would change it, and a hash-chained audit log of who signed off.
The differentiator: everyone else builds tools to predict toxicity. ARBITER reasons through the conflicts between those predictions. It is not another predictor. It is the layer that adjudicates - and the human signs.
It is deliberately an internal capability, not a product to sell. Its value is avoided cost, more consistent decisions, and a stronger evidentiary position - not licensing revenue.
Not Dempster–Shafer fusion in toxicology (precedented - Park, Ogunseitan & Lejano 2014), not structured evidence integration (OECD IATA is exactly that doctrine), not read-across with inspectable justification (OECD QSAR Toolbox). The claim is narrower and survives contact with someone who knows the literature:
- The assembly does not exist as usable software a safety lead can operate and contest - rules a scientist owns and edits live, a signed tamper-evident record, determinism enforced by the build.
- The experiment planner is driven by argument structure, not generic assay informativeness. It does not ask "which assay is usually informative?" It asks "which rule is doing the defeating, and what evidence would overturn that specific rule?"
- The as-of-date prospective replay as a validation design - testing the system on a historical case using only the evidence that existed at the decision point.
Three pieces in one repo, and it matters which one is current. The engine is kept; the deliberation app is the product; the seven-tab web app is its predecessor.
Dempster–Shafer belief fusion plus defeasible argumentation over six pre-registered rules. No clock, no randomness, no I/O - lint forbids Date, Math.random, node:*, fs, crypto, dynamic imports and parent imports anywhere in src. Deterministic to a single hash across 1000 runs.
| Rule | Name | Statement |
|---|---|---|
| R1 | Human relevance | Human-cell evidence defeats animal in vivo evidence when the question is human hepatotoxicity. |
| R2 | Mechanistic proximity | Evidence that directly measures an AOP key event defeats evidence that only correlates with chemical structure. |
| R3 | Exposure relevance | A positive finding at clinically relevant exposure defeats a negative finding whose exposure margin is unstated or untested at that range. |
| R4 | Applicability domain | Evidence from a model operating outside its applicability domain is admitted with reduced weight, or excluded. |
| R5 | Study reliability | Higher-reliability studies defeat lower-reliability ones at equal mechanistic relevance. |
| R6 | Concordance | Independent sources agreeing raises confidence more than one source agreeing with itself. |
Each rule works twice: as a defeat rule in the argumentation graph, and as an evidence-quality discount on the mass that reaches fusion. R2 cites the specific AOP key event it relies on, so mechanistic provenance sits on the rule rather than in a competing graph view.
The ruleset lives in rules/ruleset-v1.0.json and is hashed to ed073a8a7f6d9a46572e6d10016c621f0e31f169bf2b7e9676c485630b5db136. The harness refuses to run if the computed hash differs. That is the whole methodological claim: no rule was tuned after seeing a result.
This is where the work happens, and where new work goes. Four case stages, in a fixed order, because the order is the point:
| Stage | What it is for |
|---|---|
| Evidence | The compound in front of you: findings, documents, what is absent |
| Your position | Your call, written before you can see anyone else's |
| Reveal & verdict | Unreachable until everyone has answered. Then the split, the disagreement analysis, the AI adjudication, and the way through to the record: anyone named on the case can open it as one printable page (#/case/:id/report, assembled by GET /api/cases/:id/report) and print or save it as a PDF from the browser |
| Record | Sign-off and the hash-chained audit log |
Blind submission is enforced server-side by not returning the data, not by asking the
client to hide it - reading someone else's call before writing your own is the exact
failure the sequence exists to prevent. The decider is an AI adjudicator behind
services/api, disclosing a position on every rule; the engine measures it rather
than being it. See docs/superpowers/specs/2026-08-09-arbiter-ai-redesign-design.md.
A multi-user client for services/api, and the product surface. Its point is that each reviewer answers privately before anyone sees anyone else's answer, so a room produces independent readings rather than one confident one.
| Route | What it is for |
|---|---|
/dashboard | Cases you own or sit on |
/library | The case library |
/case/:id | The case: roster, documents, findings, inventory |
/case/:id/position | Your blind position - recorded before the reveal |
/case/:id/reveal | Every position at once, and where the room split |
/case/:id/record | The signed, hash-chained, tamper-checked audit log |
/ask | Retrieval-backed Q&A over the case's uploaded documents |
/method | What the system does and does not claim |
Plus apps/landing (the public entry page), apps/harness (benchmark runner, Node only), services/api (auth, documents, retrieval, adjudication), and data/prep (Python ingestion of DILIrank, splits, QSAR/Tox21 streams).
The original single-user artifact,
apps/web- a seven-tab app that ran the engine in the browser and shipped as one self-containedindex.html- was deleted on 2026-08-14. The deliberation client supersedes it. Its history is in the git log.
SUPERSEDED 2026-08-09. The table below was measured against a target that counted aspirin, amoxicillin, atenolol and amlodipine as hepatotoxic - 62% of its positive class was DILIrank's Less-concern grade. Under that definition a system correctly declining to flag amlodipine scores as wrong.
Re-graded against a corrected target (
rules/ruleset-v2.0.json), ARBITER scores 0.500 with confusiontp 2 / fp 5- five of its seven commitments are approved, widely prescribed drugs - and no baseline clears 0.601. The 0.750 below is sensitivity 1.0 averaged with a 0.5 convention for a specificity that was never measured, on n=4.Do not quote the table below. See HANDOVER §13,
tools/rescore_v2.py, anddocs/superpowers/specs/2026-08-09-arbiter-ai-redesign-design.md. It is kept unedited because it is what was believed on 2026-08-06.
Read this before quoting any number. Do not restate the headline as an accuracy.
Measured on the test split only - train fitted the QSAR model, calibration set the conformal threshold, and scoring either would be leakage. 267 compounds scored, 61 in the pre-registered conflict subset.
| pipeline | balanced accuracy | coverage | n committed | confusion (tp/fp/tn/fn) | single-class |
|---|---|---|---|---|---|
| ARBITER | 0.750 | 6.6% | 4 | 4/0/0/0 | yes |
single:transporter | 0.750 | 6.6% | 4 | 4/0/0/0 | yes |
majorityVote | 0.750 | 4.9% | 3 | 3/0/0/0 | yes |
weightedAverage | 0.547 | 100% | 61 | 51/5/1/4 | no |
single:qsar | 0.500 | 98.4% | 60 | 54/6/0/0 | no |
single:transporter matches it on every column. Say so - an earlier draft omitted this and it was corrected as a flattering omission.
And then say why, because the reason is measurable and better than the bare fact: both pipelines are scoring the same four compounds. There are only 4 transporter claims in the entire scored split, and ARBITER's four commitments are exactly those four compounds - identical sets, not an approximate overlap. An exact tie between two pipelines evaluated on the same four compounds is close to expected, not a coincidence.
ARBITER abstains on 260 of 267 compounds (97.4%). Every abstention is the belief–plausibility gap rule; none is applicability-domain and none is total conflict. Three measured causes, not one:
| # | cause | measured |
|---|---|---|
| 1 | No exposure-relevant evidence. R3 discounts a negative result tested outside the clinically relevant range to 15% of stated confidence. | 118 claims |
| 2 | QSAR measures no key event. Structure correlation alone is discounted to 6%, or 1% where it carries least. | 107 claims |
| 3 | The corpus is thin. 140 of 267 compounds carry exactly one claim. | 52.4% single-claim |
Stream coverage on the scored split makes it concrete - qsar covers 267 compounds (100%), cytotox 127 (47.6%), transporter 4 (1.5%). That resolves into three groups: 140 compounds hold qsar only, 123 hold cytotox+qsar, and 4 hold all three. ARBITER adjudicates between sources, and 140 compounds have one. The engine is being asked to do its job where its job does not exist.
Sharpest form of the result: for 254 of the 260 declines, restating every live claim at full confidence 1.0 still cannot reach the mass the threshold demands. The gap rule fires before the engine reads a single evidence value.
Planner recommendation unchanged under ±50% perturbation of every expert-elicited prior: 0.992 (2000 samples/compound, seed 20260726, 61 compounds). It holds because the planner sorts on argument structure first and score second. This is the robustness claim worth leading with.
The result is honest and defensible, but it is not a win over the baseline. The temptation will be to fix that by moving a number - and abstentionGapThreshold is pre-registered precisely so it cannot be moved after an abstention rate has been seen. Measured, moving it from 0.50 to 0.80 buys six compounds. What would fix this is data, not rules.
Lead with the things that are actually true: a pre-registered hashed ruleset, a deterministic engine, golden-file CI that catches a moved number, a planner that survives ±50% prior perturbation 99.2% of the time, and an audit trail whose tamper-evidence has been tested rather than asserted.
Not style preferences. Each protects a claim the submission makes.
- Never edit
rules/ruleset-v1.0.json. It is pre-registered and hashed. If a rule looks wrong, re-read its registered statement first - twice during development a rule looked broken and was in fact correct. A genuine error is a deliberate v1.1 re-registration with a new hash and a written reason, not an edit. - The engine stays pure. No clock, no randomness, no I/O in
packages/engine/src. Lint enforces every case. A clock or a random number breaks determinism, which is what lets golden-file CI catch a moved number at all. - Language discipline - in code, comments, UI copy, commit messages, and anything a judge reads:
| Write this | Never this |
|---|---|
| review-ready evidence package | regulator-ready dossier |
| consistent, defensible | fast |
| ARBITER's position | ARBITER's decision |
| the committee decides | the system decides |
| positions / sign-off / decision owner | voting / tally / majority |
| hash-chained audit log | blockchain |
The left column is defensible; the right overclaims regulatory standing we do not have.
npm ci
npm run dev # http://localhost:5173One command, one origin. The landing page is at /, the product at /deliberation/, the API at /api. ARBITER_PORT=4173 npm run dev moves the whole group if something already holds 5173.
The demo team is five accounts whose shared password is printed in services/api/seed-demo.ts, because the fixture is the secrecy, not the check. A fresh clone has none of them - the account store is gitignored - so create them with npm run seed:demo, or set ARBITER_DEMO_SEED=1 and let the first boot do it. The banner prints the account count either way, so a forgotten demo team is visible rather than silent.
Configuration is read from .env, or from .env.share if there is no .env. The second name exists so a file prepared for somebody else works where it lands: an unread share file and no credentials at all look identical from the outside, and that ambiguity was worth a line of code to remove. The banner names the file it read, and the directory it read it from - a server started in a second checkout reads that checkout's configuration while you are looking at this one.
The thirty-five regulatory reviews the Library searches are committed, 363 MB of them, so Ask works in a fresh clone. They were excluded until 2026-08-16 on the grounds that they were retrievable by URL; no URL was recorded anywhere, so in practice a clone showed "cannot be searched" on every entry. Fourteen of the sixteen library documents are askable - tak994 has no source document, tolcapone is a scan with no extractable text, and troglitazone has no nonclinical chapter heading. Those are refusals the product makes on purpose, and they read differently from a missing file.
There is nothing to obtain and nothing to paste. cp .env.example .env if you want to
configure anything; an empty file, or no file, is a valid configuration.
| Without credentials | |
|---|---|
| Cases, positions, blind reveal, unanimity, audit, the hash-chained record | Work. Pure code, no model. |
| Adjudication | Runs against a stub. Every response carries source: "stub", so it can never be read as a model's answer. |
| Ask & summary | 503 {"error":"no_key"}. The only surfaces that genuinely need a model. |
The startup banner names which of the two you are in.
For live AI, pick one provider. It is inferred from the model name, so there is no second switch to disagree with it:
ANTHROPIC_API_KEY=sk-ant-... ARBITER_MODEL=claude-sonnet-5 # a key, and nothing else
ARBITER_GCP_PROJECT=your-project # Gemini on ADC
GEMINI_API_KEY=AQ.... # Gemini on a keyOn Gemini, choose by who is running it. Application Default Credentials
(gcloud auth application-default login against your own project) authenticate a
person, so nothing secret belongs in .env - and equally, nothing can be handed to a
teammate. GEMINI_API_KEY is the shareable form: one line, sufficient on its own, and
still a cloud credential that bills the project it belongs to.
A key also picks a host, and only one of them works here. ARBITER_GEMINI_HOST=vertex
is the default and the catalogue every committed number was measured on. =developer
(generativelanguage.googleapis.com) cannot serve this codebase: it rejects
additionalProperties: false with a 400, and every schema in services/api sets it.
Unconstrained calls succeed there and the banner reads LIVE, so that misconfiguration
looks healthy right up until the first real adjudication fails - which is exactly why
the banner prints the endpoint rather than inferring "Vertex" from the model name.
A key must also be on a project with billing linked. Without it the key is free-tier
only, capped at 20 requests/minute shared across every holder, and Vertex refuses
outright with requires billing to be enabled.
One key shared across a team is one budget shared across a team. See
ARBITER_MODEL_BUDGET below.
Two services: Supabase holds the data, one container holds everything else. There is
no separate frontend deployment - the site is built into the image and served by the same
process that answers /api, on the same port. railway.toml is the default target and
fly.toml is the same deployment expressed for another host; any host that can run the
container and reach Postgres will do.
browser ──► container (Railway) ──► Supabase Postgres (the record,
/ landing page accounts, invites,
/deliberation/ the app document metadata)
/api/* the API ──► Supabase Storage (the PDF bytes)
Python + PyMuPDF every upload ──► Gemini / Anthropic (model calls)
Deploying is three steps: apply every file in supabase/migrations/ in order to a
Supabase project and create a private documents bucket; point Railway at this repo,
which finds the Dockerfile and railway.toml on its own; set the variables listed in
railway.toml and generate a domain. There is no deploy command - it builds on push.
- State goes to Supabase, not to a disk. What used to be five files -
results/deliberation-log.jsonl(the record itself), the account store with its password hashes, the invites sidecar, the share-links sidecar, andresults/documents/- becomes Postgres rows and Storage objects. Apply the migrations first, then set
DATABASE_URL. Absent, it silently falls back to those files, which is the right default for CI and a laptop and is exactly the arrangement that loses everything on the next redeploy. A deployment missingDATABASE_URLlooks healthy until it isn't. Setting it without the two Supabase Storage variables is refused at boot rather than half-honoured: Postgres for the record and local disk for the documents is a container that keeps the log and loses the evidence it cites.
- becomes Postgres rows and Storage objects. Apply the migrations first, then set
- Use Supabase's pooler, port 6543, not 5432. A container that redeploys or scales
opens a fresh pool each time, and direct connections exhaust a project's connection
limit fast. Transaction-mode pooling is safe here specifically because the chain
append takes
pg_advisory_xact_lock, which is released at COMMIT - the unit the pooler multiplexes on. A session-scopedpg_advisory_lockwould break silently behind a pooler, so that choice is load-bearing rather than incidental. - The container is not a plain Node image, in two ways that both bite. It needs
Python 3.12 with PyMuPDF beside Node 22, because
services/apishells out todata/prep/measure_pdf.pyfor every upload; without it every upload comes back 422 unreadable, which reads as a bad document rather than a missing dependency. And it runs from TypeScript source throughtsx, which is a devDependency - sonpm ci --omit=devbuilds an image that cannot start, andNODE_ENV=productiondoes the same thing without leaving a flag in the Dockerfile to find. The result is ~1.2 GB, most of itnode_modulesand the Python runtime. - It still needs a host that can run a subprocess, and that rules out a whole class. Every upload and every Ask forks a Python interpreter. A platform that runs JavaScript and only JavaScript - Workers, or a functions runtime - cannot serve this at all, and the failure is not a slow path but a dead one. A container is the cheap honest answer.
ARBITER_HOST=0.0.0.0to accept outside traffic. It is loopback otherwise, because this process terminates no TLS - set it only behind a proxy that does. The banner warns when it is not loopback. Inside a container the default means nothing can reach it, so this is not optional there.ARBITER_MODEL_BUDGET(default 30 per account per 10 minutes, 6x that per source) caps the four endpoints that cost money. This is what makes them safe to expose: without it, a public deployment is an open proxy to whoever's model quota it holds. It is per process, so two machines are two budgets for one account - the record has no such problem, since the chain append serialises on a database lock, but the cap does.- On Google Cloud, attach a service account rather than shipping a key. The auth
library finds it as ADC, so no key material exists on disk, in git, or in an env var.
Off Google Cloud,
GOOGLE_APPLICATION_CREDENTIALS_JSONtakes the JSON as a secret. ARBITER_STATIC_DIR=apps/landing/distis what makes the deployment a website. Without it the container serves an API and no site:services/apianswers 404 to any path whose first segment is notapi, and the client makes same-origin/apicalls, so hosting the two separately gives a page that fails on its first request. Set, the API process serves that directory for everything outside/api- the landing page at/, the staged client at/deliberation/, and the public record page for a/r/<caseId>/<token>share link - and one origin needs no proxy and no CORS. Unset is still the right default, because undernpm run devapps/landing's Vite server owns those paths and two servers claiming one URL is worse than the 404. The startup banner says which of the two you are in, and warns by name if the directory does not exist.- A built client asks who you are unless the build said otherwise. Setting
VITE_AUTO_EMAILandVITE_AUTO_PASSWORDat build time makes/deliberation/sign every visitor in as that identity, which is right for a demonstration and wrong everywhere else - it is anonymous read access to every case the deployment holds, and it makes the record say that person decided whoever was at the keyboard. Leaving them unset is what you want; see the sharing section for the whole argument. GET /api/healthis the one unauthenticated route, returning{"ok":true,"service":"arbiter-api","uptimeSeconds":N}. It exists so a health check can confirm the process serves rather than that something bound the port - a process wedged before its first response passes a TCP connect. Bothrailway.tomlandfly.tomluse it; Fly's was a TCP check only because no such route existed. It discloses nothing about the configuration, because anyone who can reach the machine can call it.
Once a case has been adjudicated, its owner can publish the record from the report page
(#/case/:id/report) - a link anyone can open, with a QR code printed onto sheet 1 of
the document so a printed page carries its own way back online.
Anyone holding the link reads the whole record, without an account. The decision,
every position in full - including ones that disagreed with the adjudication - the
evidence it was decided on, and the audit chain. It is served by an unauthenticated
route, GET /api/public/report/:caseId/:token, that exists because a share link with a
session requirement behind it is not a share link.
The email address is the only thing cut. Names, seats and every position are not.
Attribution is the record - a position with no author is a rumour, not a deliberation -
so a stranger holding the link sees exactly who said what and where they sat. What they
cannot see is how to reach that person outside the product, which they have no standing
to be handed. The cut happens where the report object is built
(services/api/verdict-report.ts), not in what the page chooses to draw, because a
field present in the response and merely hidden by the UI is one devtools tab from
being disclosed - and the public route has no session to gate that with.
Revoking stops the link. It cannot reach a copy already printed or saved. The token is derived, not stored: an HMAC over the case id and a version number, recomputed on every request rather than looked up. Revoking bumps that version, so the token already handed out stops verifying - but the PDF already saved to someone's drive, or the sheet already sitting on a desk, still shows the same QR code and the same text. It just no longer resolves. A later republish mints a different token, so it cannot reactivate a code that already went out.
Rotating ARBITER_SHARE_SECRET invalidates every published link on the deployment at
once, not just the one somebody asked to revoke - the secret is the only thing that
makes the HMAC unforgeable, so a new one makes every token derived under the old one
wrong. There is no per-link rotation, only per-deployment.
Rotate it if you ever move backings — files to Postgres, or back.share_links starts
empty and nothing carries the old store's version numbers into it, so a case that was
published and then revoked on one backing is unknown on the other: the convener is
offered "Publish this record" again and the new link is minted at version 1, which under
an unchanged secret is byte-identical to the token that was killed. Every QR printed
before the revoke starts resolving again. Rotating the secret makes that impossible,
because nothing minted under the old one verifies afterwards. The alternative, if live
links cannot be invalidated, is to copy the rows across before the first publish on the
new backing — see supabase/migrations/0002_share_links.sql.
Sharing is off unless ARBITER_SHARE_SECRET is set, and the boot banner says which:
Share: on - records can be published to a tokenised URL or Share: off - ARBITER_SHARE_SECRET is unset, so records cannot be published. Publishing without it
answers 501, naming the variable, rather than a silent no-op. The value must be at
least 32 bytes - shorter, and the process refuses to start at all, naming the variable
and why: a short secret produces links that look unguessable and are not.
/r/:caseId/:token - the public PAGE - is served everywhere the API route is. Three
arrangements, one answer:
npm run deliberate:dev- the deliberation workspace's own Vite server, whose middleware rewrites/r/*ontopublic.html.npm run dev- the unified server proxies/r/to that same middleware. It used to answer with the landing page at status 200, which reads as a broken feature rather than as an unrouted path.- A built site behind
ARBITER_STATIC_DIR-serveStaticanswers a three-segment/r/<caseId>/<token>withpublic.htmlfrom the site root, andtools/stage-site.mjsputs one there with its asset references pointed at wherever the client was staged. This is the arrangement a scanned QR code actually meets, and untile2e/public-record.spec.tsnothing in the repo opened it.
Two properties of that are worth knowing, because each was the subject of a decision.
The rewrite is one rule that resolves to one constant, not a rewrite table.serveStatic
still has no SPA fallback: a missing asset 404s rather than coming back as an HTML page with
status 200. The share-link rule matches a shape and then serves a fixed filename, so
neither the case id nor the token is ever used to build a path, and a root with no
public.html answers 404 rather than falling back to whatever else is there. That fallback
is the hazard the rule is shaped around: index.html is the app shell, and "serve
index.html for any unmatched path" is the one-line change that would hand it to anyone who
mistyped a share URL by a character.
public.html's asset references are reconciled at staging time. They are root-absolute
apps/deliberation/vite.config.ts'srenderBuiltUrl, because a share URL is two real path segments deep and a relative./assets/…would resolve against/r/<caseId>/. Root-absolute was right and root was wrong: staged under/deliberation/, the document still asked for/assets/public-<hash>.js, where the landing page's own bundle lives under different names. Served that way it was 200 OK with a blank page - a document that parses, a correct content type, and nothing in any status line saying otherwise.tools/stage-site.mjsnow points those references at the directory it staged into, and fails the build if one of them does not resolve.
Auto-sign-in is a development affordance, not a build default.apps/deliberation/src/App.tsx used to carry the seeded demo lead's address and its
published password as unconditional ?? defaults. Because that file isindex.html and
index.html is served at /deliberation/ on any deployment with ARBITER_STATIC_DIR set,
every such deployment with the demo team seeded signed in whoever reached that path - as the
convener, with read access to every case it held. Nobody had to type a credential; the build
carried one. Those defaults are now scoped to import.meta.env.DEV, so:
- development is unchanged:
npm run dev,npm run deliberate:devand the test suite all still open straight into the product; - a built artifact signs nobody in and asks who you are, unless that build explicitly
set both
VITE_AUTO_EMAILandVITE_AUTO_PASSWORD- which is how a demo deployment opts in, deliberately; - an empty value counts as absent, the same reading
ARBITER_SHARE_SECRET=""gets.
The share link itself carries no session either way, which is exactly why it must not be
trimmed and followed. Two greps hold the claims this rests on, and neither is provable from
inside a test - DEV is substituted at build time, so only the built chunks can answer:
npm run deliberate:build
# no credential in any chunk of a production build - every count 0
grep -c "arbiter-demo-2026" apps/deliberation/dist/assets/*.js
# and the public bundle still carries no auth code - only the main entry may match
grep -l "AUTO_PASSWORD\|/api/auth/login" apps/deliberation/dist/assets/*.jsnpm run lint && npm run typecheck && npm test
npm run landing:build && npm run deliberate:build && npm run e2e
npm run golden:update && git diff --exit-code results/ # must produce NO diffCI runs all of it on every push. The whole block was executed on 2026-08-14, after apps/web was deleted:
| Lint / typecheck / both builds | clean |
| Vitest | 716 tests across 48 files - was 1077 across 89 before the deletion |
| Playwright | 5 tests - the one-origin arrangement, incl. the no-WebGL guard |
Pytest (data/prep) | 32 tests across 4 files - run separately, see below |
golden:update | no diff - no reported number has moved |
| Ruleset hash | ed073a8a… matches pre-registration |
On Windows, golden:update will make the golden file look modified when it is not - the script writes LF, git's autocrlf rewrites to CRLF, and git status reports a modification with an empty git diff. Confirm it is nothing before hunting:
git show HEAD:results/golden/metrics.golden.json | sha256sum
sha256sum results/golden/metrics.golden.json # identical => nothing moved
git checkout -- results/golden/metrics.golden.jsonCalled out because "did one of my numbers move?" is the most alarming question in this project, golden:update exists to answer it, and a false yes from a line ending wastes exactly the time that guard was built to save. CI runs on Linux and never sees this.
Needed for anything touching the data layer.
python -m venv .venv &&. .venv/Scripts/activate # or bin/activate outside Windows
pip install -r data/prep/requirements.txt
cd data/prep && python -m pytest32 tests across 4 files, passing as of 2026-08-06 on Python 3.12.4 from a fresh venv with the pinned requirements.txt. data/prep/README.md documents the pipeline order. rdkit is the heavy dependency and the one most likely to fight a fresh environment, though it installed clean here.
These tests do not run in CI, so that figure is a hand measurement, not a guarded one. test_qsar_leakage.py protects the strongest methodological claim in the project - that the split was fixed before any model was fitted, which is the condition under which every reported number is valid at all. A leak reintroduced into data/prep/ today would be caught by nothing automatic. Run this suite by hand after any change under data/prep/.
packages/engine/ Pure reasoning engine. Lint-enforced purity.
src/index.ts reason(), reasonVerdictOnly(), reasonCore()
src/rules.ts R1-R6, as defeat rules and evidence-quality discounts
src/fuse.ts Dempster-Shafer belief/plausibility/conflict mass
src/argue.ts Defeasible argumentation, grounded semantics, reinstatement
src/counterfactual.ts Exhaustive minimal-flip search
src/plan.ts Value-of-information planner (the 0.992 robustness number)
apps/harness/ Benchmark runner. Node only.
src/preregistration.ts THE pre-registration surface + canonicalisation. One copy.
src/main.ts Scores the test split, writes results/
src/metrics.ts The five metrics, with their honesty caveats in comments
src/coverage-report.ts The working behind the coverage finding
apps/deliberation/ THE PRODUCT. Four stages, real backend, AI decider.
src/Layout.tsx Steps() - the four stages. The order IS the product.
src/router.ts Route union; reveal is gated on the server, not here.
src/screens.tsx Position, reveal, verdict, audit - the working screens
src/pages.tsx Auth, dashboard, case creation, method
services/api/ The backend. Accounts, cases, adjudication. Node only.
server.ts Routes, plus the built site behind ARBITER_STATIC_DIR.
/api/auth/*, /api/health and /api/public/report/* are the
unauthenticated surface. Only the last one serves case data.
share.ts Published records. The token is DERIVED, never stored.
adjudicate.ts ADJUDICATOR_PROMPT_PATH - the in-force prompt version
deliberation.ts Blind submission + unanimity. Read the contracts.
gemini.ts Vertex AI. Falls back to a labelled stub without creds.
apps/landing/ The public entry page, and the one-origin front door.
vite.config.ts server.proxy mounts /deliberation and /api behind it
src/overture/ The six-chapter WebGL overture. One canvas, six scenes.
src/overture/registry.ts The chapters. Same list the rail renders from.
src/shell/ HUD chrome: rail, preloader, menu, cursor, controls.
packages/design/ The design system both frontends dress in.
apps/atmosphere/ Scene R&D. Standalone, not wired into the product.
src/core/palette.ts ALL colour. Deep goes violet, emissive goes cyan.
src/core/Atmosphere.ts Renderer, render targets, the tear between scenes.
tools/dev-all.mjs `npm run dev`: every surface behind one port
tools/stage-site.mjs `npm run site:build`: the client, where links.ts points
e2e/ Playwright. Drives the unified server, not one app.
Dockerfile Node 22 AND Python 3.12 - the upload path forks an interpreter
fly.toml One worked deployment. No volume; state is in Supabase.
supabase/migrations/ 0001_init.sql, then 0002 onward. Append a file, never edit one.
The log is append-only at the database, not just by convention.
data/prep/*.py DILIrank ingestion, splits, QSAR/Tox21 streams
rules/ruleset-v1.0.json PRE-REGISTERED AND HASHED. Do not edit.
results/ metrics.json, golden/, verdict-manifest.json (golden-filed)
docs/superpowers/ Specs and task-by-task plans. The 2026-08-09 AI redesign
spec is IN FORCE; every earlier doc carries a banner
saying what superseded it. The plans are all
already executed - history, not a queue.
HANDOVER.md is the authority - what exists, what the result actually is, what is left, and what you must not touch. Start at §0 and read through §3. §9 is the one-paragraph version. §10 and §11 carry the phase-3 and multi-case records that would otherwise have died in a gitignored ledger.
Then, in order:
docs/superpowers/specs/2026-07-26-arbiter-design.md- the master spec. §8 is the honest results section; read it before quoting any number.docs/superpowers/specs/2026-07-27-arbiter-phase2-web-app-design.md- the web app, including §9/§9a on the static build and legibility.docs/superpowers/plans/- task-by-task plans, each recording what was measured and what went wrong.
Note that .superpowers/ is gitignored, so the SDD ledger and per-task review reports did not reach you. If a commit message refers to "the ledger" or "task-N-report.md", that is why you cannot open it. Nothing load-bearing was lost - the conclusions were copied into HANDOVER §10 and §11 - but if a decision's rationale trail stops, it stopped there.
- Scrutinise every new test for whether it CAN FAIL. A test that passes against a broken implementation is worse than no test.
- Reviews judge behaviour, not conformance to the plan. The plan is not the authority; measurement is.
- Every fix goes in both the source file and the plan's code block, so the plan stays a true record.
- Commit and push after every task. Not batched.
- Record what measured false, not just what shipped. An unverified claim spends credibility even when the surrounding work is sound.
| Endpoint | Hepatotoxicity (DILI) only |
| Engine | Complete; deterministic; ruleset hash ed073a8a… unchanged. Kept as the instrument, no longer the decider (redesign §2) |
| Deliberation app | The product. Four stages, real accounts, blind submission, AI adjudication behind services/api |
Web app (apps/web) | Predecessor. Seven tabs, eight demo beats, two hero cases; ships as one self-contained index.html. Kept working, closed to new surface |
| Phases | 1 complete · 2 complete · 3 built except Surface 2 (specified, deliberately not built) · multi-case complete |
| Intake | Custom compounds - validation, advisor, and form built; CSV upload and AI extraction not (HANDOVER §12) |
| Ablation | Aggregation, prompt and resume built and tested; no live run - needs a key and a provider decision |
| Verified | 2026-08-06 - lint, typecheck, build, 623 vitest, 12 Playwright, 32 pytest, golden all green (HANDOVER §8.3) |
| Open | LLM ablation specified but unimplemented; hero case 3 specified but not built; Cmax data is the constraint on the headline |
Submission due 16 August 2026.