diff --git a/benchmarks/buyback-audit.md b/benchmarks/buyback-audit.md new file mode 100644 index 00000000..500605c0 --- /dev/null +++ b/benchmarks/buyback-audit.md @@ -0,0 +1,53 @@ +# buyback-audit methodology + +Companion document to [`buyback-audit.yml`](./buyback-audit.yml). Mirrors the harness README in the [mobula-monorepo `miniapps/buyback-audit/`](https://github.com/MobulaFi/mobula-monorepo/tree/main/miniapps/buyback-audit) folder so both repos stay in sync. + +Bench № 018 - measures `executed_USD / promised_USD` for live on-chain token buyback programs over rolling 7-day and 30-day windows. No transactions are sent. The harness polls DeFiLlama for the promised side, and the destination wallet's actual token inflows (via the Hyperliquid `info` API for HYPE and Etherscan v2 for SKY) for the executed side, then publishes both as Prometheus gauges along with the ratio. + +## Why this bench is honest + +Every protocol with a buyback program publishes a fee share ("97% of fees buy back HYPE", "100% of surplus buys SKY"). Those numbers are commitments, not measurements. This bench takes the commitment as input and asks: given the protocol's own reported fees over the last 7 / 30 days, how much of the implied USD value actually arrived at the destination wallet. The ratio is the answer; the two windows surface execution cadence vs sustained delivery. + +## Coverage in v1 + +| Protocol | Destination | Promise | Promised source | Executed source | +| --- | --- | --- | --- | --- | +| **Hyperliquid** | Assistance Fund `0xfefe…fefe` | 97% of HyperCore fees | DeFiLlama `hyperliquid` | Hyperliquid `info` API user fills, priced at `metaAndAssetCtxs` on-chain oracle | +| **Sky / Maker** | SBE receiver `0xBE8E…98FB` | 100% of protocol surplus | DeFiLlama `makerdao` | Etherscan v2 (`chainid=1`) SKY ERC-20 inflows, priced at CoinGecko `sky` | + +## Excluded in v1 + +| Protocol | Why | +| --- | --- | +| **GMX** | V2 does not run a single on-market buyback wallet. Fees flow to GLP / GM pool LPs in ETH and stables, and to GMX stakers via esGMX reward distributors. The original treasury `0x68863dDE…dea6A` has been dormant since 2022-08 (verified via Etherscan v2). An audit of `executed_usd / promised_usd` shape has nothing to point at, so reporting a number would be dishonest. | + +v2 will add Jupiter Litterbox Trust (50% of Jupiter fees → on-market JUP buyback on Solana) and Aave AFC once their executor addresses are confirmed. + +## Honest reading of a low ratio + +A ratio well below 1.0 is not automatically under-delivery. Two regimes produce it: + +1. **Batched execution.** Sky's SBE accumulates surplus inside the Maker / Sky protocol and concentrates on-market SKY buys into irregular batches via Uniswap. A 7-day window taken mid-accumulation reads low; the long-run accrual to the destination address tracks closer to 1.0. +2. **Sustained under-funding.** The promised share overstates what is actually being routed to the buyback program. This shows up as a 30-day ratio that stays well below 1.0 across multiple snapshots — the cadence signal in (1) damps out, the structural signal does not. + +The 7d vs 30d split is the cheapest tell for which regime is in play. Continuous executors (Hyperliquid AF) show 7d ≈ 30d; batched executors (Sky SBE) show 7d < 30d; structurally under-funded programs would show both windows low. + +## Operational requirement + +The Sky leg requires `ETHERSCAN_API_KEY` set on the harness host. Without it, the executed-side query returns 0 and the Sky ratio reads 0 regardless of what SBE actually did on-chain. The Etherscan v2 single-key unified endpoint (5 req/s, 100k req/day on the free tier) is more than enough for the one paginated call per protocol per 5-minute scrape. The Hyperliquid leg has no upstream key requirement. + +## Metrics emitted + +``` +ocb_buyback_executed_usd{protocol, window} gauge (USD) +ocb_buyback_promised_usd{protocol, window} gauge (USD) +ocb_buyback_ratio{protocol, window} gauge (executed / promised, 0..1+) +ocb_buyback_scrape_errors_total{protocol, source} counter +ocb_buyback_last_scrape_timestamp_seconds{protocol} gauge +``` + +`window` is `7d` or `30d`. `protocol` is `hyperliquid` or `sky`. The leaderboard YAML consumes the 7d window by default; readers comparing cadence vs sustained delivery can query the 30d series directly against the same metric. + +## Source + +Harness lives in the (private) [mobula-monorepo](https://github.com/MobulaFi/mobula-monorepo) under `miniapps/buyback-audit/`. Spec + this doc live here in the public OpenChainBench repo. Both stay in sync via PR. diff --git a/benchmarks/buyback-audit.yml b/benchmarks/buyback-audit.yml new file mode 100644 index 00000000..4f6f3a47 --- /dev/null +++ b/benchmarks/buyback-audit.yml @@ -0,0 +1,148 @@ +# OpenChainBench. Bench № 018 + +slug: buyback-audit +number: "018" +title: Token buyback execution audit +seo_title: "Token buyback execution audit 2026: Hyperliquid HYPE, Sky SKY live executed-vs-promised ratio (7d, 30d)" +seo_description: "Live audit of on-chain token buyback programs. Executed USD over promised USD, rolling 7-day and 30-day windows, measured on-chain for Hyperliquid Assistance Fund (97% of fees, HYPE spot buyback) and Sky Smart Burn Engine (100% of surplus, SKY via Uniswap). Promised side comes from DeFiLlama protocol fees, executed side from the destination wallet's actual token inflows priced at CoinGecko." +subtitle: Executed USD over promised USD for live on-chain buyback programs, computed against destination-wallet inflows over rolling 7-day and 30-day windows. +category: Trading +status: live +metric: Execution ratio +unit: pct +higher_is_better: true + +disclaimer: | + Two caveats. (1) A low ratio is not a protocol failure. Some programs (Sky SBE) batch executions, so the executed_usd line is lumpy and routinely sits well below promised_usd inside a rolling window even though the long-run ratio converges to ~1. (2) The Sky leg requires an ETHERSCAN_API_KEY on the harness host. Without it the executed side returns 0 and the ratio reads 0; the live deployment has the key set. + +seo_intro: | + This benchmark answers the question every long-term token holder + asks but almost no one measures. when a protocol says "we use 97% + of our fees to buy back the token", does that actually show up + on-chain, and over what window. The industry standard answer is + "yes" plus a marketing blog post. The on-chain reality is a ratio: + USD value of tokens the destination wallet actually received, + divided by USD value the promise implies given the protocol's + reported revenue over the same window. We compute that ratio for + two of the largest live programs in DeFi. Hyperliquid's Assistance + Fund (the multisig at `0xfefe…fefe` that takes 97% of HyperCore + fees and spot-buys HYPE on the native order book) and Sky's + Smart Burn Engine (the `0xBE8E…98FB` SBE contract that pulls the + Maker / Sky protocol surplus and market-buys SKY through Uniswap). + The promised side is reconstructed from DeFiLlama's per-protocol + fees endpoint, scaled by the protocol's stated buyback share. The + executed side is the actual ERC-20 (or native HYPE) inflow into + the destination wallet over the same window, priced at CoinGecko. + The leaderboard surfaces the live ratio over both 7-day and 30-day + rolling windows so readers can see both the short-term execution + rhythm (where batched programs look low) and the long-run accrual + (where programs that actually run the trade converge near 1.0). + +abstract: | + We measure live token buyback execution ratio by independently + reconstructing the two sides of the equation. Promised USD comes + from DeFiLlama's protocol fees endpoint, summed over a rolling + window and multiplied by the protocol's publicly stated buyback + share (Hyperliquid 97%, Sky SBE 100% of the surplus stream). + Executed USD comes from the destination wallet's actual token + inflows over the same window — HYPE spot fills credited to the + Assistance Fund on HyperCore for Hyperliquid, SKY ERC-20 transfers + into the SBE address on Ethereum mainnet for Sky — priced at the + CoinGecko per-token spot at scrape time. The ratio + `executed_usd / promised_usd` is the headline number. Two windows + (7d, 30d) are tracked so readers can separate execution cadence + (a batched program looks low on 7d, closer to long-run on 30d) + from sustained delivery. GMX V2 is intentionally excluded because + it does not run a single auditable on-market buyback wallet — fees + flow to GLP / GM LPs in ETH and stables and to esGMX reward + distributors, so the on-chain footprint of "buyback" is structurally + unmeasurable with the methodology used here. + +methodology: + - "Promised USD - DeFiLlama `summary/fees/` endpoint, summed over the rolling window, multiplied by the protocol's documented buyback share. Hyperliquid: `hyperliquid` slug, share 0.97 (HyperCore fees → Assistance Fund). Sky: `makerdao` slug, share 1.00 (Maker / Sky protocol surplus → SBE)." + - "Executed USD, Hyperliquid - HYPE buys credited to the Assistance Fund multisig `0xfefefefefefefefefefefefefefefefefefefefe` on HyperCore, retrieved from the Hyperliquid `info` API user fills endpoint. Quantity is converted to USD using the on-chain oracle price returned by `metaAndAssetCtxs` because that is the price the AF actually pays at execution, not a CEX index." + - "Executed USD, Sky - SKY ERC-20 inflows into the SBE receiver `0xBE8E3e3618f7474F8cB1d074A26afFef007E98FB` on Ethereum mainnet, paginated through Etherscan v2 (`chainid=1`, single API key for the whole multichain unified endpoint). Quantity is decoded with the SKY contract's 18 decimals and priced at the CoinGecko `sky` spot at scrape time." + - "Windows - rolling 7d and 30d. The harness recomputes the full window's sums each scrape rather than maintaining a delta, so historical revisions of DeFiLlama's fees series propagate automatically and no state leaks across restarts." + - "Cadence - 5 minutes per protocol. The numbers are gauges, not counters; sub-5-minute resolution is meaningless because both promised and executed sides are bounded by minutes-to-hours-grain upstream data (DeFiLlama daily aggregation, on-chain block time)." + - "Excluded - GMX. V2 does not run a single on-market buyback wallet whose inflows we can audit. Fees flow to GLP / GM pool LPs in ETH and stables and to GMX stakers via esGMX reward distributors. The original treasury `0x68863dDE…dea6A` has been dormant since 2022-08 (verified via Etherscan v2), so there is no executor address that would let an audit of this shape resolve. v2 of this bench will add Jupiter Litterbox Trust (50% fees → JUP buyback on Solana) and Aave AFC as cleaner replacements." + - "Pricing - CoinGecko free tier (`hyperliquid`, `sky`) for the executed-side USD valuation. For Hyperliquid the on-chain oracle from `metaAndAssetCtxs` is preferred when available because that is the AF's actual fill reference, not a delayed third-party median. Aggregator drift between CoinGecko and the on-chain oracle is typically < 50 bps and is dwarfed by the ratio's own measurement scale." + - "Honesty boundary - the ratio measures execution against a documented promise. It does not measure whether the promise itself is generous or stingy, whether the bought-back token is held vs burned vs redistributed, or whether the buy was timed adversarially. Those are separate benchmarks that this one explicitly does not try to answer." + +findings: + - "{{best_name}} currently leads at {{best_p50}} (execution ratio, 7d) across {{count}} measured buyback programs. The ratio is the USD value of tokens actually credited to the destination wallet over the window, divided by the USD value the published promise implies given DeFiLlama's reported fee revenue over the same window." + - "{{name:hyperliquid}} returns {{p50:hyperliquid}} (execution ratio, 7d). The Assistance Fund executes HYPE spot buys on HyperCore on a near-continuous cadence as HyperCore fees accrue, so the 7-day window stays close to the long-run delivery. Persistent gap from 1.0 reflects timing between fee accrual and AF execution plus a small CoinGecko-vs-oracle pricing drift on the executed side." + - "{{name:sky}} sits at {{p50:sky}} (execution ratio, 7d). This is not under-delivery — the Smart Burn Engine pulls surplus and market-buys SKY through Uniswap on a batched schedule, so the executed line is lumpy by design. Across a 7-day window the SBE often spends most of the period accumulating surplus inside the Maker protocol and concentrates the actual on-market buys into shorter windows, which is why the rolling ratio reads low even when long-run accrual to the SBE address converges much closer to 1.0." + - "The 7d vs 30d split is where this bench earns its keep. A program that delivers on long-run promises but executes lumpily (Sky) shows a 7d well below 30d; a program that executes near-continuously (Hyperliquid) shows the two windows close to each other. Comparing the two columns is the cheapest tell for which protocol runs the trade on a tick basis vs which one batches." + - "GMX is absent by design. The V2 design does not pay fees into a single on-market buyback wallet — they flow to GLP / GM pool LPs in ETH and stables and to GMX stakers via esGMX reward distributors. An audit of `executed_usd / promised_usd` shape would have nothing on-chain to point at, so reporting a number would be dishonest. v2 of this bench will add Jupiter Litterbox Trust and Aave AFC." + +faq: + - q: "What is a token buyback execution ratio?" + a: "It is the USD value of tokens that the protocol's destination buyback wallet actually received over a rolling window, divided by the USD value the protocol's published promise implies given the protocol's own fee revenue over the same window. A ratio of 1.0 means the program delivered exactly what the promise scaled to. Above 1.0 means the program executed more than the promise (rare, usually a window-edge effect or a one-off top-up). Below 1.0 means either the program is batched and the executed side is mid-cycle, or the executor address fell behind, or the documented share overstates what the program actually pays out. The benchmark reports the ratio at 7d and 30d so readers can distinguish 'mid-batch' from 'under-delivering'." + - q: "Why is Sky's ratio so much lower than Hyperliquid's?" + a: "Cadence, not generosity. Hyperliquid's Assistance Fund executes HYPE buys on HyperCore as the protocol's fee stream accrues, so the executed line is roughly continuous and a 7-day window reads close to the long-run delivery. Sky's Smart Burn Engine pulls surplus DAI from the Maker / Sky protocol and market-buys SKY through Uniswap on a batched schedule — the SBE can sit for days accumulating surplus and then concentrate the on-market buys into shorter windows, so a 7-day snapshot taken mid-accumulation shows a low ratio. The 30-day window damps the effect but does not eliminate it because the batch frequency is irregular. Long-run delivery against the 100% surplus promise tracks closer to 1.0 — the rolling ratio is a cadence signal, not a delivery verdict." + - q: "Why isn't GMX in this benchmark?" + a: "GMX V2 does not run a single on-chain buyback wallet whose inflows can be audited the way Hyperliquid's Assistance Fund or Sky's SBE can. V2 fees flow to GLP / GM pool LPs (paid out in ETH and stables) and to GMX stakers via the esGMX reward distributor system. There is no executor address that aggregates 'bought back GMX' on a measurable cadence. The original GMX treasury at `0x68863dDE…dea6A` has been dormant since August 2022 (verified via Etherscan v2). Reporting a ratio for GMX with this methodology would either be zero by construction or require imputing the buyback equivalent from staker-side distributions, which is a different benchmark. v2 of this bench will add Jupiter Litterbox Trust (50% of Jupiter fees → JUP buyback on Solana) and Aave's AFC once the executor addresses are confirmed." + - q: "How accurate is the on-chain measurement?" + a: "The executed-side measurement is exact at the chain level — the SBE address holds the SKY tokens that Etherscan reports, the AF address holds the HYPE that the Hyperliquid `info` API reports, both are deterministic. Three sources of measurement noise sit on top of that. (1) Pricing — the executed quantity is USD-valued at scrape time, not at fill time, so a token that pumped between fill and scrape over-prices the executed side relative to the moment-of-execution USD; the drift is typically < 1% over a 5-minute scrape window. (2) Promised side — DeFiLlama's fees series occasionally revises historical days as upstream data fills in, and the harness recomputes the full window each scrape rather than locking historical sums, so the ratio can move on prior-day revisions independently of fresh execution. (3) Pagination — the Etherscan v2 query for the Sky leg paginates through transfers and any missing page would understate the executed side; the harness logs page completion and the `ocb_buyback_scrape_errors_total` counter surfaces failures." + - q: "Where does the 97% / 100% promise share come from?" + a: "Hyperliquid's 97% comes from the protocol's published fee allocation: 97% of HyperCore fees route to the Assistance Fund for HYPE buybacks, with the remainder funding HLP and other on-chain mechanisms. Sky's 100% reflects the Smart Burn Engine's design — the SBE is supposed to consume the entire Maker / Sky protocol surplus stream and convert it into on-market SKY purchases via Uniswap. Both numbers are the protocols' own published commitments; if either changes the harness config updates the share constant and the ratio rebases automatically against the new promise. The benchmark does not opine on whether 97% vs 100% is the 'right' share for a buyback program — it only measures whether the share that was promised showed up." + - q: "How is this bench different from a simple 'tokens burned per week' chart?" + a: "A burn chart shows what the destination wallet did with the tokens it received. This bench shows whether the destination wallet received what the protocol promised it would. Those are different questions and answer different concerns. A burn chart with healthy burns tells you the supply mechanic is working; a low execution ratio with healthy burns can still mean the protocol's promise is being under-funded relative to fees, just that the small amount that does arrive gets cleanly burned. The reverse is also true — a high execution ratio with no burns means the destination wallet is accumulating, which can be by design (the SBE holds SKY before burning in batches) or a warning sign (a treasury that promises buyback but never actually retires supply). The two numbers complement each other; OCB tracks the upstream one because it is the harder one to fake." + - q: "How often does this bench update?" + a: "Every 5 minutes per protocol. The numbers are gauges over rolling 7-day and 30-day windows, so sub-5-minute resolution would not surface anything new — both the DeFiLlama promised side and the destination-wallet executed side are bounded by minutes-to-hours upstream cadences. The full window is recomputed on each scrape rather than carrying a running delta, so the ratio reacts immediately to DeFiLlama historical revisions or to a freshly indexed Etherscan transfer without needing a backfill pass." + +source: https://github.com/MobulaFi/mobula-monorepo/tree/main/miniapps/buyback-audit + +prometheus: + window: 7d + +# Real metrics emitted by the buyback-audit harness: +# ocb_buyback_executed_usd{protocol, window} gauge (USD) +# ocb_buyback_promised_usd{protocol, window} gauge (USD) +# ocb_buyback_ratio{protocol, window} gauge (executed / promised, 0..1+) +# ocb_buyback_scrape_errors_total{protocol, source} counter +# ocb_buyback_last_scrape_timestamp_seconds{protocol} gauge +# +# Each "provider" below is one buyback program. The leaderboard ranks +# programs by their 7d execution ratio (higher = closer to delivering +# the promised buyback). The 30d ratio is exposed on the same metric +# with `window="30d"` for readers who want the longer-cadence view; +# the live page defaults to 7d to match the prometheus.window above. +# p50/p90/p99 columns are filled with the same scalar gauge because +# the ratio is a single observation per (protocol, window), not a +# statistical distribution — the OCB renderer expects all four to +# resolve regardless. + +providers: + - slug: hyperliquid + name: Hyperliquid + tag: Assistance Fund 0xfefe…fefe, 97% of HyperCore fees → on-market HYPE spot buyback + queries: + p50: ocb_buyback_ratio{protocol="hyperliquid",window="7d"} + p90: ocb_buyback_ratio{protocol="hyperliquid",window="7d"} + p99: ocb_buyback_ratio{protocol="hyperliquid",window="7d"} + mean: ocb_buyback_ratio{protocol="hyperliquid",window="7d"} + success: clamp_max(1 - (sum(rate(ocb_buyback_scrape_errors_total{protocol="hyperliquid"}[24h])) / 1), 1) + sample_size: ocb_buyback_executed_usd{protocol="hyperliquid",window="7d"} + series: ocb_buyback_ratio{protocol="hyperliquid",window="7d"} + + - slug: sky + name: Sky (Maker) + tag: Smart Burn Engine 0xBE8E…98FB, 100% of protocol surplus → on-market SKY buyback via Uniswap + queries: + p50: ocb_buyback_ratio{protocol="sky",window="7d"} + p90: ocb_buyback_ratio{protocol="sky",window="7d"} + p99: ocb_buyback_ratio{protocol="sky",window="7d"} + mean: ocb_buyback_ratio{protocol="sky",window="7d"} + success: clamp_max(1 - (sum(rate(ocb_buyback_scrape_errors_total{protocol="sky"}[24h])) / 1), 1) + sample_size: ocb_buyback_executed_usd{protocol="sky",window="7d"} + series: ocb_buyback_ratio{protocol="sky",window="7d"} + +# GMX is intentionally excluded from v1. V2 does not run a single +# on-market buyback wallet whose inflows we can audit — fees flow +# to GLP / GM pool LPs (paid in ETH and stables) and to GMX stakers +# via esGMX reward distributors. The original treasury +# 0x68863dDE…dea6A has been dormant since 2022-08 (verified on +# Etherscan v2). v2 of this bench will add Jupiter Litterbox Trust +# (50% of Jupiter fees → on-market JUP buyback on Solana) and +# Aave AFC once the executor addresses are confirmed. diff --git a/benchmarks/oracle-deviation.yml b/benchmarks/oracle-deviation.yml new file mode 100644 index 00000000..99a9e591 --- /dev/null +++ b/benchmarks/oracle-deviation.yml @@ -0,0 +1,245 @@ +# OpenChainBench. Bench № 025 + +slug: oracle-deviation +number: "025" +title: Cross-oracle price deviation across 10 USD pairs +seo_title: "Oracle deviation benchmark 2026: Chainlink, Pyth, Binance, Coinbase live cross-source price disagreement on BTC, ETH, SOL and 7 more" +seo_description: "Live cross-oracle deviation leaderboard. Maximum pairwise price disagreement in basis points across Chainlink AggregatorV3, Pyth Hermes, Binance and Coinbase tickers on 10 USD pairs (BTC, ETH, SOL, BNB, XRP, ADA, DOGE, AVAX, LINK, POL), polled every 30 s, p99 over the last 24 h. Includes Chainlink on-chain round-age signal." +subtitle: Maximum pairwise price disagreement in basis points across Chainlink, Pyth, Binance and Coinbase, polled every 30 s. +category: Trading +status: live +metric: Max cross-oracle deviation +unit: bps +higher_is_better: false + +disclaimer: | + Three caveats. (1) Lower is better, but a non-zero floor (~5-20 bps) is normal — exchange ticker last-traded prices have natural micro-jitter and Chainlink's on-chain feed only updates on deviation or heartbeat, so even healthy minutes show single-digit bps disagreement. (2) Three pairs (XRP, ADA, DOGE) are measured on 3 sources because Chainlink deprecated those USD feed contracts on Ethereum mainnet — the tag flags it explicitly. (3) "POL/USD" is the post-migration symbol for what was MATIC; Binance still trades MATICUSDT as a separate (frozen) pair, so we point the bench at POLUSDT instead. + +seo_intro: | + This benchmark answers the question every protocol designer + picking an oracle asks. how aligned are the major price feeds in + reality, not in marketing, on the exact pair I'm settling at this + exact second. Most "oracle comparison" articles cite vendor + whitepapers ("99.9% accurate", "sub-second latency") without + publishing the diff. We poll four oracles every 30 seconds for ten + USD-quoted blue chips, compute the pairwise deviation + `|a-b|/avg * 100` for every source pair, and publish the per-asset + maximum as a Prometheus gauge in basis points. Coverage. Chainlink + AggregatorV3 (eth_call against the canonical Ethereum mainnet + contracts), Pyth Hermes REST (batched latest_price_feeds), Binance + REST ticker (USDT-quoted, treated as ≈ USD), Coinbase REST ticker + (USD-quoted). Pairs. BTC, ETH, SOL, BNB, AVAX, LINK, POL (4 + oracles each) and XRP, ADA, DOGE (3 oracles — Chainlink mainnet + feeds for these pairs are deprecated, documented per provider). + Honest about the SOL case. SOL/USD currently shows the widest + disagreement (~1.2% live) not because the bench is broken but + because the Chainlink on-chain SOL/USD feed is updated on a 0.5% + deviation trigger that frequently lags the CEX/Pyth quote during + fast moves — exactly the kind of oracle-health signal a protocol + team needs to see before settling perp liquidations on it. + +abstract: | + We measure live oracle disagreement by polling four price oracles + every 30 seconds for ten USD-quoted blue chips, then computing the + full pairwise deviation matrix per pair. Deviation between two + sources is `|price_a - price_b| / ((price_a + price_b) / 2) * 100` + in percent; the bench surfaces it both per source-pair + (`ocb_oracle_deviation_pct{source_a, source_b}`) and as the per-pair + maximum across all source pairs (`ocb_oracle_max_deviation_pct`). + The headline leaderboard ranks the 10 pairs by p99 of the max + deviation over 24 h (lower = better cross-oracle alignment, i.e. + more agreement on what the asset actually trades at). A stale-price + guard skips any sample older than `2 * pollInterval` (60 s) so a + dead poller doesn't artificially flatten the deviation. Three pairs + (XRP, ADA, DOGE) are measured on 3 sources only because Chainlink + deprecated those USD feed contracts on Ethereum mainnet; the bench + exposes them honestly with the disclaimer in the provider tag and + still computes the deviation across the remaining trio. The + `ocb_oracle_last_round_age_seconds{source="chainlink"}` companion + gauge surfaces Chainlink's on-chain update gap separately, so a + high deviation can be attributed to the right cause (a lagging + feed vs. a CEX outlier) without confusing the reader. + +methodology: + - "Sources. Chainlink AggregatorV3 via `eth_call(latestRoundData())` on Ethereum mainnet (no API key, public RPC with fallback), Pyth Network via Hermes REST `/api/latest_price_feeds` batch endpoint (no key), Binance REST `/api/v3/ticker/price` on USDT-quoted symbols (treated as ≈ USD), Coinbase REST `/products/

/ticker` on USD-quoted products. All four are free, no-auth, public endpoints." + - "Pairs (10). BTC, ETH, SOL, BNB, AVAX, LINK and POL are tracked on all 4 sources. XRP, ADA, DOGE are tracked on 3 sources (Pyth + Binance + Coinbase) because the Chainlink mainnet AggregatorV3 contracts for these USD feeds are deprecated and no longer updated by node operators — the provider tag flags it." + - "Cadence. 30 second polling interval per (source, pair). 4 sources × 10 pairs × 2/minute = 80 requests/minute total, comfortably under every free-tier ceiling (Binance 1200/min, Coinbase 10/s public, Pyth ~30/s soft, public RPC courteous). HTTP timeout is 8 s per request to match the gas-estimation harness." + - "Deviation formula. For every unordered pair of sources `(a, b)` that both have a fresh sample for the same pair: `deviation_pct = |price_a - price_b| / ((price_a + price_b) / 2) * 100`. With 4 sources we compute 6 pairwise deviations per asset; with 3 sources we compute 3. Per-pair headline is the max across all source-pairs (`ocb_oracle_max_deviation_pct`)." + - "Stale-price guard. A source's sample is considered stale and excluded from the deviation calculation if its last successful update is older than `2 * pollInterval` (60 s). Prevents a dead Coinbase ticker from registering as a 0% deviation against itself by leaving the price gauge fossilized." + - "Chainlink round age. The on-chain `updatedAt` timestamp from `latestRoundData()` is published separately as `ocb_oracle_last_round_age_seconds{source=\"chainlink\", pair}`. Chainlink updates only on deviation (typically 0.25-0.5% for blue chips) or heartbeat (~1 h), so a 30-minute age on a quiet ETH minute is normal; a 2-hour age on SOL during a volatile minute is the actual signal." + - "USDT ≈ USD assumption. Binance only quotes USDT pairs (BTCUSDT, etc.). We treat USDT as ≈ USD ± 10 bps drift, which is acceptable for a bench whose alert floor is ≥ 10 bps. A real USDT depeg would surface as Binance drifting from the other three sources for *every* pair simultaneously — exactly what we want this bench to flag, not hide." + - "MATIC → POL migration. Polygon migrated MATIC → POL 1:1 on Sep 4 2024. The Chainlink mainnet feed contract is still named MATIC/USD but on-chain `description()` confirms it tracks the POL token; Pyth renamed the feed to POL/USD; Coinbase delisted MATIC-USD and only lists POL-USD; Binance kept MATICUSDT as a frozen historical pair AND lists POLUSDT. We point Binance at POLUSDT so all four sources track the same underlying asset. Bench label is kept as `pair=\"MATIC/USD\"` for query continuity." + - "Excluded by design. Redstone (push-pull, on-demand, no continuous gauge to scrape without an integration contract), Uniswap V3 TWAP (would need per-pool integration and is a derivation of the same CEX prints rather than an independent feed), DIA (smaller deployment footprint than the four kept), and any aggregator-of-aggregators (CoinGecko, CoinMarketCap, DefiLlama) for the same reason as the stablecoin-peg bench — re-aggregating already-aggregated data adds latency and hides the per-source disagreement we're trying to surface." + +findings: + - "{{best_name}} currently leads at {{best_p50}} (p99 max cross-oracle deviation, 24 h) across {{count}} measured pairs. The leaderboard ranks by how tightly the available sources agree on the asset's price — lower = better — so the leader is the asset for which every source you might integrate would settle within fractions of a basis point of every other." + - "{{name:btc-usd}} clocks {{p50:btc-usd}} on 4 sources. BTC's deep liquidity on every CEX plus the most-watched Chainlink feed on mainnet (heartbeats updated within minutes by competing node operators) means the 4-way disagreement is structurally near the floor; if you can't get oracle agreement on BTC, you have a poller bug, not a market signal." + - "{{name:sol-usd}} runs noticeably wider at {{p50:sol-usd}} on 4 sources. This is a real signal, not a bench bug. Chainlink's SOL/USD mainnet feed updates only on a 0.5% deviation trigger or 1 h heartbeat, while Pyth/Binance/Coinbase publish continuously — during fast SOL moves the on-chain Chainlink price visibly lags the CEX print by minutes (the companion `ocb_oracle_last_round_age_seconds{pair=\"SOL/USD\"}` gauge frequently shows multi-thousand-second ages versus ~1000-1500 s for blue chips). Any perp DEX settling SOL liquidations off Chainlink alone inherits this lag as slippage on the trader." + - "XRP, ADA and DOGE are measured on 3 sources, not 4 — the Chainlink mainnet AggregatorV3 contracts for these USD pairs are deprecated and no longer updated by node operators, so the bench drops them from the deviation matrix rather than averaging in stale data. The 3-source deviation is honest (Pyth + Binance + Coinbase, all live), and the provider tag flags the asymmetry. Protocols settling these assets on Chainlink mainnet should confirm directly with the contract's `updatedAt`." + - "{{name:matic-usd}} sits at {{p50:matic-usd}} — the post-migration POL token. The Chainlink mainnet contract still answers to the legacy MATIC/USD name but on-chain `description()` confirms it tracks POL 1:1 (verified against Pyth's renamed POL/USD feed and Coinbase's POL-USD product). If you're building on Polygon and integrating the legacy contract address, you're already on POL — no migration needed." + - "{{worst_name}} trails at {{worst_p50}} on the active filter. The gap between leader and laggard is dominated by two factors: how fast the asset moves (volatile minutes widen the deviation regardless of oracle quality) and how aggressive each oracle's update trigger is (Chainlink's deviation threshold is the binding constraint on most pairs; Pyth/Binance/Coinbase are within bps of each other most of the time)." + +faq: + - q: "What is oracle deviation and why does it matter?" + a: "Oracle deviation is the basis-point gap between two oracles' reported price for the same asset at the same moment. It matters because every on-chain product that settles in fiat (perp DEXes, lending markets, derivatives, stablecoin issuers) picks one oracle and inherits its drift. If your lending market uses Chainlink ETH/USD and Chainlink lags the CEX print by 30 bps during a fast move, every liquidation triggered in that window over- or under-pays the keeper by 30 bps — which is the difference between a healthy liquidation engine and a bad-debt spiral. The bench measures this gap live so you can size the risk before you ship the integration." + - q: "Why is SOL deviation so much higher than BTC?" + a: "Two reasons, both structural and both informative. (1) Chainlink's SOL/USD mainnet feed is configured with a 0.5% deviation trigger and a 1-hour heartbeat — so during fast SOL moves the on-chain price legitimately lags the live CEX print until the trigger fires. (2) SOL is more volatile per unit time than BTC/ETH, so even within the trigger window there's more price travel between updates. The companion gauge `ocb_oracle_last_round_age_seconds{pair=\"SOL/USD\"}` regularly shows ages in the multiple thousands of seconds (vs ~1000-1500 s for ETH/BTC/AVAX/LINK), which is the on-chain confirmation that the deviation is Chainlink lag, not a CEX outlier. The bench surfaces this honestly — it's exactly the signal a protocol team needs to see before settling SOL perps off a single oracle." + - q: "What does 'Chainlink lag' mean in practice — does the oracle break?" + a: "Chainlink doesn't break, but it intentionally doesn't push every tick. On-chain updates cost gas; Chainlink node operators only post a new round when either (a) the off-chain aggregated price has drifted by a configured threshold from the last on-chain value (the 'deviation trigger', typically 0.25-0.5% for blue chips, 1% for long-tail) or (b) the configured heartbeat interval has elapsed since the last update (typically 1 h for blue chips, longer for stale-tolerant assets). Between updates, the on-chain value is whatever was last pushed — so during a fast move the on-chain price can lag the off-chain market by 1× the deviation threshold for as long as it takes the next round to land. That's by design: it trades latency for gas cost. The bench's `ocb_oracle_last_round_age_seconds` gauge surfaces the gap directly so the lag is never invisible." + - q: "Why are XRP, ADA and DOGE measured on only 3 sources?" + a: "The Chainlink mainnet AggregatorV3 contracts for XRP/USD, ADA/USD and DOGE/USD are deprecated — node operators no longer post fresh rounds to them, so the on-chain `updatedAt` is days or weeks stale. Rather than averaging in a fossilized value (which would either falsely deflate the deviation by anchoring it to an outdated number or falsely inflate it once the live market moves away), the bench excludes Chainlink from the deviation matrix for these three pairs and computes the disagreement across Pyth + Binance + Coinbase only. The provider tag for each of these pairs flags '3 sources' explicitly. Protocols settling XRP, ADA or DOGE on Ethereum mainnet through Chainlink should confirm the feed's status directly — these are tier-2 assets on Chainlink's Ethereum L1 deployment and the production feeds for them live on other chains (BSC, Solana for ADA/DOGE)." + - q: "MATIC vs POL — what changed and which one is this bench measuring?" + a: "Polygon migrated its native token from MATIC to POL on September 4 2024, 1:1. POL is the new token; MATIC is the legacy ticker. The four oracles handled it differently: Pyth renamed the feed to POL/USD; Coinbase delisted MATIC-USD and lists only POL-USD; Binance kept MATICUSDT as a frozen historical pair (don't use it — it detaches from spot) AND lists POLUSDT; Chainlink left the mainnet feed contract at its old address with the legacy `description()` of 'MATIC / USD' but it tracks the POL token. The bench resolves all four to POL and keeps the label `pair=\"MATIC/USD\"` for query continuity. The reported price is POL/USD and all four sources agree on the same underlying asset." + - q: "Why these four oracles and not Redstone, Uniswap TWAP or DIA?" + a: "Two filters: 'free no-auth public endpoint' and 'continuous gauge'. Chainlink, Pyth Hermes, Binance and Coinbase all expose continuously-updating price feeds via REST or eth_call with no API key, which lets the bench scrape them at 30 s cadence indefinitely at zero cost. Redstone is push-pull/on-demand — the on-chain price is only refreshed when a transaction includes the signed off-chain quote in calldata, so there is no continuous gauge to scrape without deploying an integration contract that pulls on a timer. Uniswap V3 TWAP is a derivation of the same CEX prints we already poll directly, plus it needs per-pool integration code. DIA has smaller deployment footprint than the four kept. We may add Redstone and a pure-on-chain reference (TWAP) in a follow-up bench that runs on-chain itself; for the off-chain comparison this is the honest set." + - q: "How often is each oracle polled and why 30 s?" + a: "Every (source, pair) is polled every 30 seconds. The cadence is bounded below by Chainlink's update mechanics — the on-chain feed only refreshes on deviation or heartbeat, so polling faster than the feed updates returns the same value and wastes RPC calls. 30 s is fast enough to catch every Chainlink round within ~half a heartbeat of its on-chain landing while keeping total request volume at 80 req/min across all 4 sources × 10 pairs (well under every free-tier ceiling). Pyth Hermes, Binance and Coinbase tickers refresh continuously so 30 s polling captures them at full freshness." + +source: https://github.com/OpenChainBench/OpenChainBench/tree/main/harnesses/oracle-deviation + +prometheus: + window: 24h + +# Real metrics emitted by the oracle-deviation harness: +# ocb_oracle_price{source, pair} gauge (USD) +# ocb_oracle_deviation_pct{pair, source_a, source_b} gauge (percent) +# ocb_oracle_max_deviation_pct{pair} gauge (percent, max across all source pairs) +# ocb_oracle_update_latency_seconds{source, pair} gauge +# ocb_oracle_last_round_age_seconds{source="chainlink", pair} gauge (Chainlink only — on-chain updatedAt gap) +# ocb_oracle_scrape_errors_total{source, pair} counter +# +# sources are "chainlink" | "pyth" | "binance" | "coinbase". +# +# Each "provider" below is one USD pair. The leaderboard ranks +# pairs by p99 of the per-pair max cross-oracle deviation over +# 24 h (lower = tighter cross-oracle agreement). The harness +# publishes the metric as a percent; we multiply by 100 in every +# query to surface it in basis points (bps), which is the right +# granularity for oracle-grade reporting (1 bps = 0.01%). +# +# XRP, ADA and DOGE are 3-source pairs because the Chainlink +# mainnet AggregatorV3 contracts for those USD feeds are +# deprecated (the harness drops Chainlink from the deviation +# matrix for these pairs rather than averaging in a stale value). +# The provider tag flags it explicitly. + +providers: + - slug: btc-usd + name: BTC/USD + tag: Bitcoin spot, 4 sources (Chainlink, Pyth, Binance, Coinbase) + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="BTC/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="BTC/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="BTC/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="BTC/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="BTC/USD"}[24h])) / (4 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="BTC/USD"})) + series: ocb_oracle_max_deviation_pct{pair="BTC/USD"} * 100 + + - slug: eth-usd + name: ETH/USD + tag: Ether spot, 4 sources (Chainlink, Pyth, Binance, Coinbase) + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="ETH/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="ETH/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="ETH/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="ETH/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="ETH/USD"}[24h])) / (4 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="ETH/USD"})) + series: ocb_oracle_max_deviation_pct{pair="ETH/USD"} * 100 + + - slug: sol-usd + name: SOL/USD + tag: Solana spot, 4 sources — Chainlink mainnet feed lags on fast moves + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="SOL/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="SOL/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="SOL/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="SOL/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="SOL/USD"}[24h])) / (4 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="SOL/USD"})) + series: ocb_oracle_max_deviation_pct{pair="SOL/USD"} * 100 + + - slug: bnb-usd + name: BNB/USD + tag: Binance Coin spot, 4 sources (Chainlink, Pyth, Binance, Coinbase) + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="BNB/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="BNB/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="BNB/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="BNB/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="BNB/USD"}[24h])) / (4 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="BNB/USD"})) + series: ocb_oracle_max_deviation_pct{pair="BNB/USD"} * 100 + + - slug: avax-usd + name: AVAX/USD + tag: Avalanche spot, 4 sources (Chainlink, Pyth, Binance, Coinbase) + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="AVAX/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="AVAX/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="AVAX/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="AVAX/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="AVAX/USD"}[24h])) / (4 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="AVAX/USD"})) + series: ocb_oracle_max_deviation_pct{pair="AVAX/USD"} * 100 + + - slug: link-usd + name: LINK/USD + tag: Chainlink token spot, 4 sources (Chainlink, Pyth, Binance, Coinbase) + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="LINK/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="LINK/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="LINK/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="LINK/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="LINK/USD"}[24h])) / (4 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="LINK/USD"})) + series: ocb_oracle_max_deviation_pct{pair="LINK/USD"} * 100 + + - slug: matic-usd + name: POL/USD + tag: Polygon POL spot (legacy MATIC ticker on Chainlink feed), 4 sources + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="MATIC/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="MATIC/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="MATIC/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="MATIC/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="MATIC/USD"}[24h])) / (4 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="MATIC/USD"})) + series: ocb_oracle_max_deviation_pct{pair="MATIC/USD"} * 100 + + - slug: xrp-usd + name: XRP/USD + tag: Ripple spot, 3 sources (Pyth, Binance, Coinbase — Chainlink mainnet feed deprecated) + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="XRP/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="XRP/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="XRP/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="XRP/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="XRP/USD"}[24h])) / (3 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="XRP/USD"})) + series: ocb_oracle_max_deviation_pct{pair="XRP/USD"} * 100 + + - slug: ada-usd + name: ADA/USD + tag: Cardano spot, 3 sources (Pyth, Binance, Coinbase — Chainlink mainnet feed deprecated) + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="ADA/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="ADA/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="ADA/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="ADA/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="ADA/USD"}[24h])) / (3 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="ADA/USD"})) + series: ocb_oracle_max_deviation_pct{pair="ADA/USD"} * 100 + + - slug: doge-usd + name: DOGE/USD + tag: Dogecoin spot, 3 sources (Pyth, Binance, Coinbase — Chainlink mainnet feed deprecated) + queries: + p50: quantile_over_time(0.50, ocb_oracle_max_deviation_pct{pair="DOGE/USD"}[24h]) * 100 + p90: quantile_over_time(0.90, ocb_oracle_max_deviation_pct{pair="DOGE/USD"}[24h]) * 100 + p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="DOGE/USD"}[24h]) * 100 + mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="DOGE/USD"}[24h]) * 100 + success: 1 - (sum(rate(ocb_oracle_scrape_errors_total{pair="DOGE/USD"}[24h])) / (3 * (1/30))) + sample_size: count(count by (source) (ocb_oracle_price{pair="DOGE/USD"})) + series: ocb_oracle_max_deviation_pct{pair="DOGE/USD"} * 100 diff --git a/benchmarks/solana-tx-landing-latency.yml b/benchmarks/solana-tx-landing-latency.yml new file mode 100644 index 00000000..7eb3f2ff --- /dev/null +++ b/benchmarks/solana-tx-landing-latency.yml @@ -0,0 +1,192 @@ +# OpenChainBench. Bench № 027 + +slug: solana-tx-landing-latency +number: "027" +title: Solana transaction landing latency +seo_title: "Solana tx landing latency 2026: Jito vs Helius vs Nozomi vs Astralane vs 0slot time-to-land + landing rate" +seo_description: "Live active benchmark of Solana transaction landing services. p50 / p99 time-to-land in slots and milliseconds, landing rate %, drop classification. Identical signed mainnet probes submitted to each service from us-east, every hour. Methodology pre-registered." +subtitle: How fast does each landing service get a signed mainnet tx confirmed. Active probing — we send real synthetic transactions every hour from us-east and measure landing latency. +category: Trading +status: live +metric: p50 time-to-land (7-day window) +unit: ms +higher_is_better: false + +disclaimer: | + Five caveats. (1) us-east only — sgp / eu-west arrive in V2. (2) One pre-registered tip per service. (3) Synthetic payload (1-lamport self-transfer + memo); real swaps may land differently. (4) Helius / Astralane / Nozomi fan out to Jito internally; we run a Jito control probe each cycle. Helius is probed in `swqos_only=true` mode. (5) Confirmation = `confirmed`; rare `processed`-only landings undercounted. Pair with the observational view at /benchmarks/solana-tx-landing. + +seo_intro: | + This benchmark answers the only question that actually matters + to a Solana trader picking a landing service. how fast does + your signed mainnet transaction reach the confirmed state on + the wire? Every hour from a us-east probe, the harness submits + an identical signed tx through each of 5 services in parallel, + measures the slot delta and wall-clock milliseconds from + sendTransaction acknowledgement to first non-null + confirmationStatus on the public RPC, and increments + per-service Prometheus histograms. Headline numbers shown are + p50 and p99 over a rolling 7-day window — long enough to be + stable, short enough to reflect current chain conditions. + Coverage. 5 services probed in V0-Lean. Jito Block Engine (the + control / baseline because Helius, Astralane, Nozomi all + internally route some flow through it). Helius Sender in + `swqos_only=true` mode (isolates the Helius own-path from the + Jito leg). Astralane Iris (tip-refund mechanism). Nozomi by + Temporal Labs (premium tier, hard 1M lamport tip floor). + 0slot.trade (premium tier). NextBlock, bloXroute and + SolanaVibeStation arrive in the next tier (V1) once the first + sponsors land. Companion bench. /benchmarks/solana-tx-landing + measures market share via on-chain tip-wallet attribution — + who carries the flow today, regardless of speed. The two + benches are complementary. fast service with no volume vs + slow service that everyone uses is a real product question. + +abstract: | + We probe 5 Solana transaction landing services from a single + Railway us-east region, once per hour, by submitting an + identical signed mainnet transaction to each. The payload is + the minimal valid Solana tx — compute-budget instructions + (50k CU limit, 50k micro-lamport/CU price), a 1-lamport + self-transfer, the per-service tip transfer to the service's + documented tip wallet, and an OCB-prefixed memo for forensic + traceability. All five services are submitted in parallel + within a single cycle so they sample the same chain congestion + window. We capture submit_slot via getSlot(processed) before + send and land_slot from getSignatureStatuses polling at 1 s + intervals on the public RPC. A 60 s no-confirmation deadline + classifies the probe as dropped{reason=timeout}; structured + RPC errors classify as invalid; transport failures as + network_error; HTTP 419 / 429 / "rate limit" errors classify + as rate_limited (a separate label so quota issues don't bias + the bench against the throttled service). Cost. ~$159/mo at + SOL=$86, 86 % of which goes to the four ≥1M-lamport-floor + services (Nozomi, 0slot, bloXroute, NextBlock — only two of + these in V0-Lean). Sponsor SOL credits covering a service's + own probes are explicitly allowed per the sponsor-proof + framework. Limitations. (a) Single us-east region — sgp / + eu-west arrive in V2 once sponsors fund geographic-edge + story. (b) 1-hour cadence — 168 probes per service per 7-day + window, enough for stable p50 / p99 over the publication + window, not enough for intra-hour resolution (V0.5 / V1 + upgrade if needed). (c) Fan-out — Helius probed in + `swqos_only` mode only in V0-Lean to keep wire shape simple; + dual-mode arrives in v1.0.1 methodology PR. + +methodology: + - "Source endpoints (all base64 sendTransaction, all submitted from a Railway us-east node) — Jito: `https://ny.mainnet.block-engine.jito.wtf/api/v1/transactions`. Helius Sender: `http://ewr-sender.helius-rpc.com/fast?swqos_only=true` (mandatory `skipPreflight=true`, `maxRetries=0`). Nozomi: `https://ewr.nozomi.temporal.xyz/?c=`. Astralane: `https://ny.gateway.astralane.io/iris?api-key=` (non-canonical 3-element params with `mevProtect=true`). 0slot: `https://ny.0slot.trade?api-key=`." + - "Probe payload. 5 instructions in this exact order: SetComputeUnitLimit(50,000) + SetComputeUnitPrice(50,000 micro-lamports/CU) + SystemProgram.Transfer(payer→payer, 1 lamport) + SystemProgram.Transfer(payer→service tip wallet, floor lamports) + Memo(`ocb---`). cycle_id is an 8-byte random hex shared across the five parallel probes of one cycle, so the on-chain memos correlate." + - "Tip floors (pre-registered, methodology PR + 14-day window to change). Jito 10,000 lamports. Helius Sender 10,000. Astralane 500,000 net of refunds. Nozomi 1,000,000. 0slot 1,000,000." + - "Submission flow. One getLatestBlockhash(processed) shared across all five probes of a cycle. One getSlot(processed) captured before send — used as submit_slot. The five probes fire in parallel goroutines, sign, POST. Each returns a signature; we poll getSignatureStatuses on the public RPC every 1 s. First non-null confirmationStatus == confirmed or finalized counts as landed; land_slot is read from the response context." + - "Drop classification. timeout = no confirmation within 60 s. invalid = RPC error, on-chain Err, or BlockhashNotFound. network_error = transport-level (timeout, DNS, EOF, connection refused). rate_limited = HTTP 419 / 429 / 'rate limit' / 'too many requests'. landing_rate is published as success / (success + timeout) — rate_limited and network_error are excluded so quota / transport issues don't bias the bench against a throttled service." + - "Jito control probe. Helius (default mode), Astralane, Nozomi route a portion of flow through Jito internally. Without controlling for this, their measured latency conflates 'this service's own path' with 'Jito caught it via this service'. Jito is in the V0-Lean probe set so the Jito control fires in the same cycle as the suspect services — comparing land_slot delta between Jito and a suspect service reveals whether the service is adding value beyond a Jito wrapper." + - "Reproducibility. The full harness source is at github.com/MobulaFi/mobula-monorepo/tree/main/miniapps/solana-tx-landing. Anyone with a funded Solana keypair (~1 SOL) can clone, set SOLANA_PROBE_KEYPAIR_BASE58, run the binary, and reproduce these metrics. The bench does not rely on any internal Mobula service for measurement — the only RPC dependency is the public `api.mainnet-beta.solana.com`." + - "Methodology v1.1 pre-registered at github.com/OpenChainBench/OpenChainBench/blob/main/docs/methodology/solana-tx-landing-active.md. Any change (tip floor, probe payload, cadence, region, metric definitions) ships as a public PR with a 14-day comment window. Major version bumps run a 30-day shadow period publishing old and new metrics in parallel." + +findings: + - "{{best_name}} is the fastest landing service in the V0-Lean probe set at p50 = {{best_p50}} ms over the rolling 7-day window. The gap between fastest and slowest is the operational signal — every service claims '99 %+ landing rate' in its marketing copy, but the chain doesn't lie about latency." + - "{{name:jito}} is the baseline / control. Because Helius default mode, Astralane, and Nozomi all internally fan out to Jito, the Jito p50 is the floor any service must beat to justify its premium. {{name:jito}} sits at {{p50:jito}} ms." + - "{{name:helius-sender}} in `swqos_only` mode tells you what Helius's own routing path delivers without the Jito leg. p50 = {{p50:helius-sender}} ms. Helius's default mode (with Jito fan-out) lands at the Jito floor or better — a v1.0.1 methodology update will publish both series side-by-side." + - "{{name:nozomi}} at {{p50:nozomi}} ms — Nozomi's premium pricing (1M lamport hard floor, ~10 × Jito's competitive level) only makes economic sense if the latency advantage is large enough to recover the tip differential. The 7-day p50 gap vs Jito quantifies that bet." + - "{{worst_name}} trails at {{worst_p50}} ms p50. The worst latency in the V0-Lean set is not necessarily a bad service — it may be a service whose strength is in dimensions this bench doesn't measure (anti-MEV protection, durable nonce support, fee-refund mechanics). Latency is one variable, not the whole product." + +faq: + - q: "Why an active bench when /benchmarks/solana-tx-landing already exists?" + a: "/benchmarks/solana-tx-landing is observational — it watches the chain and counts who carries the flow. It cannot answer 'how fast does my tx land if I send it now', because it doesn't send anything. This bench (active probing) answers that — at the cost of running 24 / 7 with real SOL ($159 / month at the V0-Lean cadence). The two benches answer different product questions. Read both." + - q: "Why only 5 services, not the 8 you measure observationally?" + a: "NextBlock, bloXroute Trader, and SolanaVibeStation all require paid plans or sales-call onboarding before they issue an API key. We're shipping V0-Lean today — the 5 services that have a clear self-serve or contact-based path — and adding the other 3 as soon as a sponsor signs (V1 escalation). The observational bench at /benchmarks/solana-tx-landing already covers all 8 because it doesn't need API keys." + - q: "Is this biased toward services that sponsor OCB?" + a: "The sponsor-proof framework forbids it. (1) Methodology is pre-registered on GitHub before any sponsor contract is signed. (2) Inclusion is automatic for any reachable public landing endpoint — sponsors don't gain leaderboard entry through cash. (3) Probe parameters are identical across services and frozen per methodology version. (4) Sponsor contracts include a non-suppression clause; OCB retains unilateral right to publish all results, including ones unfavourable to the sponsor. (5) Funding is publicly itemised on /funding quarterly. (6) Sponsors see numbers when the public does. See docs/methodology/solana-tx-landing-active.md §11 for full text." + - q: "Why us-east only?" + a: "V0-Lean. us-east is the de-facto Solana baseline (Jito, NextBlock, bloXroute, Helius all anchor their best-connected POPs there) and Mobula already runs Railway in us-east. Adding eu-west and sgp triples the bench cost and answers a different question ('does the ranking change by geography?'). That story unlocks in V2 when 2-3 sponsors are signed." + - q: "What's the probe payload?" + a: "Five instructions in this exact order, locked by methodology §3. (1) SetComputeUnitLimit(50,000). (2) SetComputeUnitPrice(50,000 micro-lamports/CU) — together a 2,500-lamport priority fee. (3) SystemProgram.Transfer of 1 lamport from the prober keypair to itself — the minimal valid state-touching tx. (4) SystemProgram.Transfer to the service's documented tip wallet at the pre-registered floor. (5) Memo program write with the cycle ID, service name, and probe mode. Total weight: ~600 bytes, well under the 1,232-byte tx limit." + - q: "How is fan-out handled?" + a: "Helius (default mode), Astralane, and Nozomi route a portion of flow through Jito internally. The Jito control probe — Jito is part of the V0-Lean probe set — fires in the same cycle as the suspect services with the same blockhash and a comparable tip. If Nozomi's land_slot matches Jito's in a cycle, the inference is that Jito caught it on Nozomi's behalf. If Nozomi lands 2+ slots before Jito, Nozomi's own path added measurable value. Helius is additionally probed in `?swqos_only=true` mode to isolate its own routing path." + - q: "Can a service detect and prioritise our probes?" + a: "Yes, in principle. The memo prefix `ocb-` is deterministic and the keypair is constant per region. Anti-fingerprinting (memo randomisation, sub-account rotation, tip jitter within the floor band) ships in v1.0.2 methodology PR. We disclose this risk openly; the trade-off is that announcing the bench publicly to providers gives them a chance to fix real performance issues before we publish, which is a good outcome. We do NOT accept private deals to alter the probe surface for any specific service." + - q: "Why is sample_size on the dashboard ~168 per service?" + a: "V0-Lean cadence = 1 probe per service per hour from 1 region. 168 = 24 hours × 7 days. The 7-day publication window is the trade-off between statistical resolution (sample size grows with window) and freshness (shorter window reflects current chain conditions). At ~168 samples per cell, p50 is stable to within ±5 % and p99 to within ±15 %. Lower confidence intervals are unlocked at V0.5 cadence (1 / 10 min, ~$760 / mo) and above." + +source: https://github.com/MobulaFi/mobula-monorepo/tree/main/miniapps/solana-tx-landing + +methodology_doc: https://github.com/OpenChainBench/OpenChainBench/blob/main/docs/methodology/solana-tx-landing-active.md + +related_benches: + - slug: solana-tx-landing + relation: "Observational counterpart — measures market share via on-chain tip-wallet attribution (who carries the flow). Read both." + +prometheus: + window: 7d + +# Real metrics emitted by the active prober in solana-tx-landing harness: +# solana_landing_probe_success_total{service, mode, region} counter +# solana_landing_probe_dropped_total{service, mode, region, reason} counter +# solana_landing_probe_latency_slots_histogram{service, mode, region} histogram +# solana_landing_probe_latency_ms_histogram{service, mode, region} histogram +# solana_landing_probe_keypair_balance_sol{region} gauge +# solana_landing_probe_cycle_total{region} counter +# solana_landing_probe_enabled{region} gauge +# +# Headline metric = p50 ms time-to-land. lower_is_better. +# Mode label is `swqos_only` for helius-sender, `default` for the rest. + +providers: + - slug: jito + name: Jito Block Engine + tag: Baseline + control probe; atomic bundles + tip auction since 2022 + queries: + p50: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="jito",region="us-east"}[7d]))) + p90: histogram_quantile(0.9, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="jito",region="us-east"}[7d]))) + p99: histogram_quantile(0.99, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="jito",region="us-east"}[7d]))) + mean: avg_over_time(solana_landing_probe_latency_ms{service="jito",region="us-east"}[7d]) + success: sum(rate(solana_landing_probe_success_total{service="jito",region="us-east"}[7d])) / (sum(rate(solana_landing_probe_success_total{service="jito",region="us-east"}[7d])) + sum(rate(solana_landing_probe_dropped_total{service="jito",region="us-east",reason="timeout"}[7d]))) + sample_size: sum(increase(solana_landing_probe_success_total{service="jito",region="us-east"}[7d])) + series: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="jito",region="us-east"}[1h]))) + + - slug: helius-sender + name: Helius Sender (swqos_only) + tag: Isolated Helius own-path (no Jito fan-out); anycast + 7 POPs + queries: + p50: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="helius-sender",mode="swqos_only",region="us-east"}[7d]))) + p90: histogram_quantile(0.9, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="helius-sender",mode="swqos_only",region="us-east"}[7d]))) + p99: histogram_quantile(0.99, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="helius-sender",mode="swqos_only",region="us-east"}[7d]))) + mean: avg_over_time(solana_landing_probe_latency_ms{service="helius-sender",mode="swqos_only",region="us-east"}[7d]) + success: sum(rate(solana_landing_probe_success_total{service="helius-sender",mode="swqos_only",region="us-east"}[7d])) / (sum(rate(solana_landing_probe_success_total{service="helius-sender",mode="swqos_only",region="us-east"}[7d])) + sum(rate(solana_landing_probe_dropped_total{service="helius-sender",mode="swqos_only",region="us-east",reason="timeout"}[7d]))) + sample_size: sum(increase(solana_landing_probe_success_total{service="helius-sender",mode="swqos_only",region="us-east"}[7d])) + series: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="helius-sender",mode="swqos_only",region="us-east"}[1h]))) + + - slug: astralane + name: Astralane Iris + tag: Tip-refund mechanism, sendBundle / sendIdeal modes, FRA + NY POPs + queries: + p50: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="astralane",region="us-east"}[7d]))) + p90: histogram_quantile(0.9, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="astralane",region="us-east"}[7d]))) + p99: histogram_quantile(0.99, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="astralane",region="us-east"}[7d]))) + mean: avg_over_time(solana_landing_probe_latency_ms{service="astralane",region="us-east"}[7d]) + success: sum(rate(solana_landing_probe_success_total{service="astralane",region="us-east"}[7d])) / (sum(rate(solana_landing_probe_success_total{service="astralane",region="us-east"}[7d])) + sum(rate(solana_landing_probe_dropped_total{service="astralane",region="us-east",reason="timeout"}[7d]))) + sample_size: sum(increase(solana_landing_probe_success_total{service="astralane",region="us-east"}[7d])) + series: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="astralane",region="us-east"}[1h]))) + + - slug: nozomi + name: Nozomi + tag: Temporal Labs, direct-to-leader, premium 1M-lamport hard floor + queries: + p50: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="nozomi",region="us-east"}[7d]))) + p90: histogram_quantile(0.9, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="nozomi",region="us-east"}[7d]))) + p99: histogram_quantile(0.99, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="nozomi",region="us-east"}[7d]))) + mean: avg_over_time(solana_landing_probe_latency_ms{service="nozomi",region="us-east"}[7d]) + success: sum(rate(solana_landing_probe_success_total{service="nozomi",region="us-east"}[7d])) / (sum(rate(solana_landing_probe_success_total{service="nozomi",region="us-east"}[7d])) + sum(rate(solana_landing_probe_dropped_total{service="nozomi",region="us-east",reason="timeout"}[7d]))) + sample_size: sum(increase(solana_landing_probe_success_total{service="nozomi",region="us-east"}[7d])) + series: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="nozomi",region="us-east"}[1h]))) + + - slug: 0slot + name: 0slot.trade + tag: Premium 1M-lamport floor (100k on Advanced tier), anti-MEV + durable-nonce + queries: + p50: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="0slot",region="us-east"}[7d]))) + p90: histogram_quantile(0.9, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="0slot",region="us-east"}[7d]))) + p99: histogram_quantile(0.99, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="0slot",region="us-east"}[7d]))) + mean: avg_over_time(solana_landing_probe_latency_ms{service="0slot",region="us-east"}[7d]) + success: sum(rate(solana_landing_probe_success_total{service="0slot",region="us-east"}[7d])) / (sum(rate(solana_landing_probe_success_total{service="0slot",region="us-east"}[7d])) + sum(rate(solana_landing_probe_dropped_total{service="0slot",region="us-east",reason="timeout"}[7d]))) + sample_size: sum(increase(solana_landing_probe_success_total{service="0slot",region="us-east"}[7d])) + series: histogram_quantile(0.5, sum by (le)(rate(solana_landing_probe_latency_ms_histogram_bucket{service="0slot",region="us-east"}[1h]))) diff --git a/benchmarks/token-unlocks.yml b/benchmarks/token-unlocks.yml new file mode 100644 index 00000000..6b9c3bda --- /dev/null +++ b/benchmarks/token-unlocks.yml @@ -0,0 +1,260 @@ +# OpenChainBench. Bench № 022 + +slug: token-unlocks +number: "022" +title: Token unlocks pressure (90-day horizon) +seo_title: "Token unlocks 2026: ARB, OP, SUI, JTO, EIGEN, ENA, WLD, APT, TIA, NEAR live unlock calendar with conditional price impact" +seo_description: "Live token unlocks leaderboard. % circulating supply impacted over the next 90 days for 53 tracked tokens (curated whitelist + CryptoRank hasVesting=true top-200), plus a conditional median price-impact prediction (J+30) built from the last 200 historical comparable unlocks bucketed by unlock size." +subtitle: Upcoming unlock pressure (% circulating supply) over a J+0 to J+90 horizon, with a conditional median price-impact prediction at J+30 built from historical bucket-matched comparables. +category: Trading +status: live +metric: Expected impact (J+30) +unit: pct +higher_is_better: true + +disclaimer: | + Four caveats. (1) Predictions are conditional medians, not point forecasts — they describe the historical centre-of-mass for bucket-matched unlocks, not "the price will move X%". (2) ARB has a known v1 gap — the upcoming-supply rows surface correctly but the expected-impact gauge is silently dropped on collection; filed for v2. (3) The 3-5% and >10% buckets currently hold zero mature samples, so unlocks falling in those size classes do not emit an expected-impact value (fail-closed by design, the gate requires ≥5 samples). (4) Source is CryptoRank's public Next.js routes — when CR rotates its buildId on deploy, the calendar lags by up to one scrape cycle until the harness re-parses __NEXT_DATA__. + +seo_intro: | + This benchmark answers the question every token holder, market + maker and protocol treasury asks before the next vesting event. + how much supply lands in the next 90 days, on which date, and + what does the historical comparable set say about the price + reaction. Most "token unlock tracker" sites stop at the calendar + view (date + nominal token count + dollar notional at spot). + That ignores the only number that actually drives the trade. + what % of circulating supply does the unlock represent, and + what is the conditional median price move for unlocks of that + size bucket over the next 30 days. We pull the calendar and + per-token vesting schedules from CryptoRank's public Next.js + data routes (no API key, ~1 req/s with backoff), enrich with + DeFiLlama daily prices for the historical backfill, and pool + the last 200 historical unlocks into 5 size buckets (<1%, + 1-3%, 3-5%, 5-10%, >10% of circulating supply at the unlock + date). For each upcoming unlock we publish the % supply, the + dollar notional at current spot, the matched-bucket median of + `(price_J+30 / price_J-7 − 1) × 100` across all historical + pooled unlocks (gated to require ≥5 mature samples, otherwise + no gauge emitted), and a 22-token curated whitelist (ARB, OP, + SUI, STRK, JTO, EIGEN, ENA, WLD, APT, MANTA, NEAR, W, ZK, DYM, + TIA, ONDO, PYTH, AVAX, FET, TON, HYPE, OMNI) unioned with the + CryptoRank top-200 filtered by `hasVesting=true` so the page + covers every token that has both a public schedule and enough + market cap to matter. + +abstract: | + We compute upcoming-unlock pressure by parsing CryptoRank's + Next.js `_next/data//...` token-unlock.json (global + calendar) and per-token vesting.json (full schedule per slug), + refreshed hourly. For each token in the priority list (curated + whitelist of 22 majors unioned with CR's top-200 hasVesting=true + coins, capped at 200 total), we expose every unlock in J+0..J+90 + as `ocb_unlock_upcoming_pct_supply{token, unlock_date, allocation_type}` + in % circulating supply, plus `ocb_unlock_upcoming_value_usd` at + current spot. The conditional impact model runs separately. for + each mature historical unlock (J+30 already elapsed) we compute + `(price_J+30 / price_J-7 − 1) × 100` using DeFiLlama daily + prices, bucket by % supply (<1%, 1-3%, 3-5%, 5-10%, >10%), and + expose the per-bucket median as the prediction for every upcoming + unlock that falls into the same bucket. Median over mean for + robustness — a handful of post-unlock 2× rallies (catalyst + coinciding with vesting) would swamp the arithmetic mean. The + prediction gauge is gated to require ≥5 mature samples in the + matching bucket; thinner buckets fail closed (no gauge emitted) + rather than publish a misleading median computed from 1-2 + outliers. CryptoRank's buildId rotates on every CR deploy; + the harness lazily re-parses `