Long-form LLM output gets graded as one blob, or not at all. This splits it into atomic claims, retrieves evidence for each from Wikidata and Wikipedia, verifies them independently, and aggregates the result into a risk score you can act on — so "this paragraph is probably fine" becomes "these two sentences are the problem."

The hard part isn't the checking, it's making the checking reproducible: a deterministic eval harness and an append-only audit log, because a reliability score you can't reproduce is just a vibe.

flowchart LR
A["Long-form<br/>LLM output"] --> B["Claim<br/>extraction"]
B --> C["Atomic claims"]
C --> D["Evidence retrieval<br/>Wikidata · Wikipedia"]
D --> E["Per-claim<br/>verification"]
E --> F["Risk<br/>aggregation"]
F --> G["Scored audit<br/>+ append-only log"]
E -.->|"no evidence found"| H["Flagged as<br/>unverifiable"]
H --> F

FastAPI · Next.js · retrieval · deterministic eval harnessrepo →

Resume intelligence platform that surfaces hiring bias rather than quietly encoding it. The architectural point: LLM document processing is decoupled from the request path through a worker queue, so a nine-second model call never becomes a nine-second API response.

Instrumented end to end — if a worker is falling behind, the dashboard says so before a user does.

sequenceDiagram
participant U as Client
participant A as FastAPI
participant Q as Redis queue
participant W as RQ worker
U->>A: POST /analyze
A->>Q: enqueue job
A-->>U: 202 + job id
Note over A,U: request path never waits on the model
Q->>W: dequeue
W->>W: parse · LLM · bias scoring
W->>Q: store result
U->>A: GET /jobs/{id}
A-->>U: result

FastAPI · RQ workers · PostgreSQL · Redis · Prometheus + Grafanarepo →

Automates the first pass of investment-banking deal screening: financial ratio filters, NLP over news and filings, synergy detection, and backtesting to check whether the screen would actually have caught the deals that mattered.

Built as a monorepo and hardened for the boring realities — retries, structured logging, health and readiness probes.

Monorepo · Celery · PostgreSQL · observabilityrepo →

Decision-support system for pricing under demand uncertainty and capacity constraints. Runs candidate policies against simulated demand and benchmarks them on the same footing, which surfaced a 4–5% revenue lift over static pricing in simulation.

Simulation runner · policy benchmarking · Next.js dashboardrepo →

• pushed to addyvantage/axedstack-website · 7d ago
• pushed to addyvantage/addyvantage · 25d ago
• opened issue #44823 in timburgan/timburgan · 25d ago