Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 0 additions & 163 deletions docs/methodology/solana-tx-landing-active.md

This file was deleted.

7 changes: 4 additions & 3 deletions harnesses/solana-tx-landing/README.md
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
# solana-tx-landing harness

Source for two OpenChainBench benches that share a single binary:
Source for the OpenChainBench bench:

- [`solana-tx-landing`](https://openchainbench.com/benchmarks/solana-tx-landing) — observational market-share view of Solana transaction landing services (Jito, Helius Sender, Nozomi, Astralane, 0slot, etc.) measured via on-chain tip-wallet attribution.
- [`solana-tx-landing-latency`](https://openchainbench.com/benchmarks/solana-tx-landing-latency) — active probing: a synthetic 1-lamport self-transfer submitted through each service, timing the slot delta to confirmation. See [`docs/methodology/solana-tx-landing-active.md`](../../docs/methodology/solana-tx-landing-active.md) for the pre-registered methodology.

The binary also embeds an opt-in active prober (slot-delta latency), kept around for re-enablement but not currently wired to a public bench page on OCB.

Exposes Prometheus metrics on `:2112/metrics` (OCB Railway convention).

Expand DownExpand Up@@ -56,7 +57,7 @@ curl localhost:2112/metrics | grep solana_landing
| `LOGS_TOKEN` | (unset) | Optional, gates `/logs?tail=N` |
| `SLACK_WEBHOOK_URL` | (unset) | Optional, posts probe failures + low-balance alerts |

See [`docs/methodology/solana-tx-landing-active.md`](../../docs/methodology/solana-tx-landing-active.md) for the exact probe payload, tip floors per service, and statistical thresholds.
The probe payload, tip floors per service, and statistical thresholds are documented inline in `cmd/script/prober.go` and `cmd/script/senders.go`.

## Reproducibility

Expand Down
5 changes: 2 additions & 3 deletions harnesses/solana-tx-landing/cmd/script/active_metrics.go
Original file line numberDiff line numberDiff line change
Expand Up@@ -16,9 +16,8 @@ import (
// • landing_rate = success_total / (success_total + dropped{reason=timeout})
// • p50 / p99 derived in Prom from the *_histogram series
//
// Methodology pinned at docs/methodology/solana-tx-landing-active.md
// (OpenChainBench repo). Any label / metric change is a methodology PR
// with the 14-day comment window.
// Label / metric shape is stable; any change should ship as a public PR
// with a 14-day comment window before redeploy.

var (
// One increment per landed (confirmed) probe. Headline numerator of
Expand Down
1 change: 0 additions & 1 deletion harnesses/solana-tx-landing/cmd/script/main.go
Original file line numberDiff line numberDiff line change
Expand Up@@ -53,7 +53,6 @@ func main() {
go runSubscriber(ctx, wsURL)

// Active prober (opt-in: requires SOLANA_PROBE_KEYPAIR_BASE58).
// Methodology: docs/methodology/solana-tx-landing-active.md
go runProber(ctx)

sig := make(chan os.Signal, 1)
Expand Down
3 changes: 1 addition & 2 deletions harnesses/solana-tx-landing/cmd/script/prober.go
Original file line numberDiff line numberDiff line change
Expand Up@@ -22,8 +22,7 @@ import (

// Active prober — submits a synthetic mainnet tx through each landing
// service every cycle, then polls confirmation. Headline metrics =
// landing_rate + p50/p99 latency per service. Methodology pinned at
// OpenChainBench/docs/methodology/solana-tx-landing-active.md.
// landing_rate + p50/p99 latency per service.
//
// The prober is OPT-IN. It only runs when SOLANA_PROBE_KEYPAIR_BASE58
// is set; absence keeps the harness in pure observational mode.
Expand Down
8 changes: 4 additions & 4 deletions src/app/answers/[slug]/page.tsx
Original file line numberDiff line numberDiff line change
Expand Up@@ -43,10 +43,10 @@ export async function generateMetadata({
const url = `${SITE.url}/answers/${ans.slug}`;
const title = ans.seo_title ?? ans.question;
const descSource = ans.seo_description ?? ans.short_answer;
// Clean leftover tokens AFTER renderTemplate so a draft bench
// (e.g. solana-tx-landing-latency mid-soak) never leaks a literal
// `{{best_name}}` into the meta description, og:description or
// twitter:description, all of which feed the SERP and social previews.
// Clean leftover tokens AFTER renderTemplate so a draft bench never
// leaks a literal `{{best_name}}` into the meta description,
// og:description or twitter:description, all of which feed the SERP
// and social previews.
const description = capDescription(
cleanLeftoverTokens(renderTemplate(descSource, ans.bench)),
158,
Expand Down
7 changes: 3 additions & 4 deletions src/app/answers/page.tsx
Original file line numberDiff line numberDiff line change
Expand Up@@ -28,10 +28,9 @@ export default async function AnswersHubPage() {
// before the JSX touches the string.
//
// Tokens that renderTemplate can't resolve get a neutral fallback so
// a draft / awaiting-data bench (e.g. solana-tx-landing-latency mid-soak
// with every provider's p50 still at 0) never surfaces raw `{{best_name}}`
// to the SERP. Same pattern as resolveLeftoverPlaceholders on the
// per-chain bench page.
// a draft / awaiting-data bench (every provider's p50 still at 0)
// never surfaces raw `{{best_name}}` to the SERP. Same pattern as
// resolveLeftoverPlaceholders on the per-chain bench page.
const rendered = await Promise.all(
answers.map(async (a) => {
const bench = await loadBenchmark(a.benchmark, { chain: a.chain });
Expand Down
4 changes: 2 additions & 2 deletions src/app/benchmarks/page.tsx
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
import type { Metadata } from "next";
import { getBenchmarksSafe } from "@/data/benchmarks";
import { getBenchmarksSafe, toBenchmarkCardData } from "@/data/benchmarks";
import { BenchmarkGrid } from "@/components/benchmark-grid";
import { safeJsonLd } from "@/lib/jsonld";

Expand DownExpand Up@@ -86,7 +86,7 @@ export default async function BenchmarksPage() {
{DESCRIPTION}
</p>
</header>
<BenchmarkGrid benchmarks={benchmarks} />
<BenchmarkGrid benchmarks={benchmarks.map(toBenchmarkCardData)} />
</article>
);
}
8 changes: 6 additions & 2 deletions src/components/benchmark-card.tsx
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
import Link from "next/link";
import type { Benchmark } from "@/types/benchmark";
import type { BenchmarkCardData } from "@/data/benchmarks";
import { Hint } from "@/components/hint";
import { MiniChart } from "@/components/mini-chart";
import { CATEGORY_COLOR } from "@/lib/category-colors";
Expand All@@ -16,8 +16,12 @@ import { fmtValue, unitSuffix } from "@/lib/format";
* - Insufficient samples: published but the harness has no usable p50
* yet. "Insufficient samples" pill, greyed-out value, chart skipped.
* - Live: standard rendering, leader p50 in display style.
*
* Prop is the slim `BenchmarkCardData` projection, not the full
* Benchmark. Keeps the RSC payload for `/benchmarks` to the fields the
* card actually reads.
*/
export function BenchmarkCard({ benchmark }: { benchmark: Benchmark }) {
export function BenchmarkCard({ benchmark }: { benchmark: BenchmarkCardData }) {
const b = benchmark;
const isDraft = b.status === "draft";
const insufficient = !isDraft && isInsufficient(b);
Expand Down
4 changes: 2 additions & 2 deletions src/components/benchmark-grid.tsx
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,7 +2,7 @@

import { useMemo, useState } from "react";
import { LayoutGrid, List, Search } from "lucide-react";
import type { Benchmark } from "@/types/benchmark";
import type { BenchmarkCardData } from "@/data/benchmarks";
import { BenchmarkCard } from "@/components/benchmark-card";

/**
Expand All@@ -11,7 +11,7 @@ import { BenchmarkCard } from "@/components/benchmark-card";
* the only fully-implemented mode here - list view degrades to a single
* column) and a search input with a ⌘K affordance.
*/
export function BenchmarkGrid({ benchmarks }: { benchmarks: Benchmark[] }) {
export function BenchmarkGrid({ benchmarks }: { benchmarks: BenchmarkCardData[] }) {
const [query, setQuery] = useState("");
const [activeCategory, setActiveCategory] = useState<string | null>(null);
const [view, setView] = useState<"grid" | "list">("grid");
Expand Down
14 changes: 12 additions & 2 deletions src/components/mini-chart.tsx
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,26 @@
import { useMemo } from "react";
import type { Benchmark } from "@/types/benchmark";
import { buildProviderColors } from "@/lib/series-colors";

/**
* Compact multi-line chart for bench cards. Same visual language as the
* full TimeSeriesChart on the bench detail page. every provider gets a
* line in their signature color. but stripped of axes, hover and tabs.
* Reads `series24h` from the benchmark's `extras` payload.
*
* Structural prop type: accepts the full `Benchmark` (home table) and
* the slim `BenchmarkCardData` projection (hub grid) so the hub doesn't
* need to ship the full Benchmark shape (series7d, series30d,
* metricPanels, editorial copy, ...) per card in the RSC payload.
*/

type MiniChartBenchmark = {
results: { slug: string; name: string; ms: { p50: number } }[];
higherIsBetter: boolean;
extras: { series24h: Record<string, number[]> };
};

type Props = {
benchmark: Benchmark;
benchmark: MiniChartBenchmark;
/** Internal viewBox width used for path math. Visual width is 100% of parent. */
viewBoxWidth?: number;
height?: number;
Expand Down
1 change: 0 additions & 1 deletion src/data/bench-published.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,7 +13,6 @@
"perp-fees": "2026-05-07T14:44:23.000Z",
"pm-data-freshness": "2026-06-04T17:50:08.000Z",
"rpc-capabilities": "2026-05-20T18:43:56.000Z",
"solana-tx-landing-latency": "2026-05-21T17:54:38.000Z",
"stablecoin-peg": "2026-05-20T18:43:56.000Z",
"stablecoin-peg-usdt-anchored": "2026-05-20T21:20:02.000Z",
"token-deployment-cost": "2026-06-09T15:34:12.000Z",
Expand Down
61 changes: 61 additions & 0 deletions src/data/benchmarks.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,67 @@ export type {
Series24h,
} from "@/types/benchmark";

/**
* Card-shaped projection used by the hub grid and category pages. The
* full Benchmark object carries ~30 fields per bench (extras.series7d,
* series30d, seriesByRegion*, metricPanels, methodology, abstract, faq,
* findings, perChainExplainer, seoIntro, bestPerChain, worstPerChain,
* cellRanks, providersPerChain, ...) that the card never reads. Passing
* the full shape to `<BenchmarkGrid />` (a client component) baked the
* whole thing into the RSC payload — 3.2 MB of HTML for 37 cards, which
* is what made `/benchmarks` feel like an 8 s page even though the
* server TTFB is sub-second.
*
* Project once at the page boundary so the wire payload only carries
* what the card + the grid's search/filter use.
*/
export type BenchmarkCardData = {
slug: string;
title: string;
subtitle: string;
category: Benchmark["category"];
status: Benchmark["status"];
editorialStatus: Benchmark["editorialStatus"];
unit: Benchmark["unit"];
higherIsBetter: boolean;
sampleSize: number;
lastRunAt: string;
metric: string;
results: {
slug: string;
name: string;
availability?: "live" | "unavailable";
ms: { p50: number };
}[];
extras: { series24h: Record<string, number[]> };
};

/** Strip a Benchmark to the fields the hub card actually renders. Keeps
* `results` in its original sort order so the card's own sort
* (best-leader heuristic) stays correct. */
export function toBenchmarkCardData(b: Benchmark): BenchmarkCardData {
return {
slug: b.slug,
title: b.title,
subtitle: b.subtitle,
category: b.category,
status: b.status,
editorialStatus: b.editorialStatus,
unit: b.unit,
higherIsBetter: b.higherIsBetter,
sampleSize: b.sampleSize,
lastRunAt: b.lastRunAt,
metric: b.metric,
results: b.results.map((r) => ({
slug: r.slug,
name: r.name,
availability: r.availability,
ms: { p50: r.ms.p50 },
})),
extras: { series24h: b.extras?.series24h ?? {} },
};
}

/**
* Strict loader. Throws AllBenchmarksDraftError when every bench has
* collapsed to draft (Prom blackout, cold start with no KV snapshot).
Expand Down
9 changes: 0 additions & 9 deletions src/data/compare-pairs.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -157,15 +157,6 @@ export const COMPARE_PAIRS: ComparePair[] = [
providerB: "hyperliquid",
publishedAt: "2026-06-17",
},
{
// solana-tx-landing-latency exposes the Helius RPC sender under the
// slug `helius-sender`, not bare `helius`, so the pair canonical
// slug uses helius-sender to match the bench provider id.
slug: "helius-sender-vs-jito",
providerA: "helius-sender",
providerB: "jito",
publishedAt: "2026-06-09",
},
{
slug: "hyperliquid-vs-lighter",
providerA: "hyperliquid",
Expand Down
16 changes: 14 additions & 2 deletions src/lib/citation.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -38,7 +38,17 @@ import { fmtUnit } from "@/lib/format";
* - Use this predicate BEFORE deriving leader / fieldValue / headline
* for any externally-visible surface.
*/
export function isInsufficient(b: Benchmark): boolean {
/** Structural subset accepted by `isInsufficient`. Lets the hub card
* call this with the slim BenchmarkCardData shape (which omits the
* full Benchmark fields the predicate does not read) without breaking
* the existing full-Benchmark call sites. */
export type InsufficientCheckInput = {
editorialStatus: Benchmark["editorialStatus"];
status: Benchmark["status"];
results: { ms: { p50: number }; availability?: "live" | "unavailable" }[];
};

export function isInsufficient(b: InsufficientCheckInput): boolean {
if (b.editorialStatus !== "live") return true;
if (b.status !== "live") return true;
// Note: do NOT key on b.sampleSize === 0. The aggregator loader can
Expand All@@ -47,7 +57,9 @@ export function isInsufficient(b: Benchmark): boolean {
// benches as insufficient on /api/citable while /api/stat returned
// live values for the same slug. The liveResults length and p50
// finiteness checks below already catch the genuine empty case.
const live = liveResults(b.results);
const live = b.results.filter(
(r) => r.availability !== "unavailable" && r.ms.p50 > 0,
);
if (live.length === 0) return true;
return live.every((r) => !Number.isFinite(r.ms.p50) || r.ms.p50 <= 0);
}
Expand Down
Loading