fix(api): /api/stat returns insufficient status to match /api/citable - #626
Closed
Flotapponnier wants to merge 21 commits into
Closed
fix(api): /api/stat returns insufficient status to match /api/citable#626Flotapponnier wants to merge 21 commits into
Flotapponnier wants to merge 21 commits into
Conversation
rss: ship /rss.xml feed for benchmark releases
…ee (#593) (#594) Adds the Near Intents 1Click API provider to both bridge benchmarks. Metrics will start populating once the bridge-monitor harness is redeployed with the nearintents_bridge.go integration shipped on mobula-api side and the NEARINTENTS_API_KEY env var set on Railway. Includes Near Protocol logo (public/logos/near-intents.svg), provider registry entry with longDescription + docs link, and YAML provider entries on both bridge-quote-latency and bridge-fee with the post-#584 avg(...) wrapper on success queries. SEO cohort copy updated to include Near Intents in both benches.
Adds a 'dimensions.chain' block to bridge-quote-latency and bridge-fee with values matching the harness chain label (Solana / Base / Arbitrum / HyperCore). The site UI surfaces this as clickable tabs. Why: the aggregate cross-corridor view is misleading on bridges with asymmetric per-corridor performance. Near Intents in particular is bimodal (Arb to HyperCore ~30ms via solver-cached bid, Sol to Base ~3000ms full auction wait), and the aggregate histogram quantile produced 175ms p50 which represents neither corridor. Requires the matching mobula-api PR (chain label emission in the bridge-monitor harness).
) (#598) Each destination chain in the current route set has exactly one source chain, so we can surface the full corridor (Sol to Base, Arb to HyperCore, etc.) in the tab label without changing the underlying dimension (still chain = destination, Prom label intact). User scans the tabs and knows immediately which route each tab represents instead of having to remember the route set.
…tab (#599) (#600) Two small UX fixes on the chain dimension: 1. Add a hypercore -> hyperliquid alias in logo-manifest.ts so the 'Arb to HyperCore' tab renders with the Hyperliquid logo. HyperCore is the trading engine of Hyperliquid, same brand. No new asset needed. 2. Drop the 'All routes' default tab. The aggregated cross-corridor view is mathematically valid but conceptually meaningless on these benches (Sol to Base ~3000ms and Arb to HyperCore ~30ms produce a histogram_quantile that lands at neither corridor). Per-corridor tabs are the honest default.
…#603) The route was redirecting to /api/llm-context, which is fine for permissive clients but breaks: - llms.txt validators (llmstxt-checker fails on non-200) - stricter AI crawlers (Perplexity, some Bing AI variants) that mark non-standard paths returning a redirect as no-content and skip - canonical attribution (AI engines citing the body now point at the internal /api/llm-context path instead of the public /llms-full.txt) Re-exports the GET handler from /api/llm-context so there is a single source of truth for the markdown body. No content change.
…#605) * fix(sitemap): pin <lastmod> to deploy time, not request time (#604) Sitemap runs on force-dynamic (to bypass Next's 2 MB Data Cache limit on the rendered output), which means `new Date()` at module init re-evaluates on every Google crawl. Effect: every URL had a freshly updated <lastmod> each visit, Google flagged the signal as unreliable sitewide and stopped using lastmod for recrawl prioritisation (confirmed behaviour, documented by Gary Illyes publicly). Fix: bake a build-time timestamp via next.config env injection, read it from process.env.NEXT_PUBLIC_BUILD_TIME in sitemap.ts. Stable per deploy, fresh per deploy. Per-bench <lastmod> still uses live bench.lastRunAt as before — that path was already correct. * release: hold solana-tx-landing-latency off main Bench and its dependent /alternatives + /answers pages held off main while the harness data quality / methodology is iterated on dev. Matches the existing release-hold pattern (see 12bb8a7). Files stay on dev untouched; dev to main merges will need to re-delete or skip. Removed: - benchmarks/solana-tx-landing-latency.yml - alternatives/jito.yml (benchmark: solana-tx-landing-latency) - alternatives/helius.yml (benchmark: solana-tx-landing-latency) - answers/which-solana-rpc-lands-the-most-transactions.yml
Fixes /api/citable returning all benchmarks as draft when /api/stat/<slug> for the same bench returns live data. Root cause: loadAllBenchmarksCached returned an all-draft set whenever per-bench fetches threw at cold start, and the cache then served that poisoned set for the rest of the 60s revalidate window. Restores the throw-on-all-draft behaviour at the cache boundary so unstable_cache keeps the previous good value during a Prom blackout. Adds an AllBenchmarksDraftError sentinel and a getBenchmarksSafe wrapper. Call sites split: - API endpoints and feeds (citable, llm-context, llms.txt, rss.xml) catch the sentinel and return 503 with retry-after, so downstream consumers do not treat the placeholder set as ground truth. - Pages (home, benchmarks index, bench slug, category, mcp page) and shared loaders (chains, providers) use getBenchmarksSafe which catches the sentinel and renders draft placeholders so the build and the page render still succeed. - Sitemap already uses safeLoad which catches and falls back. Cache key bumped from v12 to v13 to flush any poisoned snapshot already stored in Upstash KV. Adds a regression test that asserts the aggregator never returns a stable all-draft list. Extracted aggregateBenchmarks as a pure function so the test does not need Prometheus or unstable_cache.
…) (#617) * feat(rate-limit): bypass per-IP throttle for known AI crawler user agents * rate-limit: expand AI bot allowlist to match robots.txt Adds Meta-ExternalAgent, Meta-ExternalFetcher, cohere-ai, Perplexity-User (all already in robots.txt), plus AI2Bot, Ai2Bot-Dolma, Kagibot, FacebookBot, MistralAI-User, TimpiBot, Webzio-Extended for completeness. Comment now states the regex must stay in sync with src/app/robots.ts. Western LLM coverage is at ~95 percent after this. Chinese models other than ByteDance (Bytespider) do not publish a documented bot UA, so they fetch via generic browser UAs and stay throttled by design.
…#621) (#622) PR #613 left valid ad-hoc /compare/<a>-vs-<b> URLs serving HTTP 200 + a loading skeleton + noindex meta. With loading.tsx in place, Next 16 streams the skeleton with 200 before the page body's notFound() can demote the status. Plus generateMetadata only checked that both providers exist, missing the no-shared-bench case. This hotfix lands the same fix as PR #621 (dev) but inline on main's older file structure (no extracted compare-compute helper module yet). hasSharedBenches() is a pure set arithmetic helper on appearances, no Prom fan out. loading.tsx removal lets notFound() ship a real 404. Verified locally: /compare/alchemy-vs-quicknode -> 404 (quicknode missing) /compare/mobula-vs-coingecko -> 308 then 404 (coingecko missing) /compare/alchemy-vs-helius -> 404 (no shared bench) /compare/quicknode-vs-alchemy -> 308 to canonical /compare/binance-vs-bybit -> 200 real h1 (valid ad-hoc) /compare/ethereum-vs-solana -> 200 real h1 (curated)
Flotapponnier
commented
Jun 22, 2026
CollaboratorAuthor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/api/stat was claiming live data with a value and a leader for
benchmarks where the underlying harness has no usable sample (network-fees,
token-deployment-cost, oracle-deviation 3 samples). /api/citable
for the same bench correctly returned status draft with null values.
LLMs and agents that fetch /api/stat directly were seeing the
misleading assertion. This change adds an isInsufficient predicate
that both endpoints now share, and propagates the same status
through headlineSentence so the /answers page and /api/llm-context
also degrade to "insufficient data" instead of asserting a winner.
No code in the harness changed. The signal that was missing was
the consumer-side guard.
Predicate
Surfaces touched
Backward compat
status is now "live" | "draft" | "insufficient". Existing consumers
keep the same shape; downstream that hard-typed status to ("live" | "draft")
should treat "insufficient" as not-live.