model.tracker

English | 简体中文

Automatically tracks model releases, pricing, and performance across the world's major AI vendors. Updated daily.

Coverage

14 vendors:

International: OpenAI · Anthropic · Google · Meta · Mistral · xAI · Cohere
China: DeepSeek · Qwen (Alibaba) · Zhipu GLM · Doubao (ByteDance) · Kimi (Moonshot) · Baichuan · Hunyuan (Tencent)

3 kinds of performance data sources:

LMSYS Chatbot Arena (human blind-test Elo)
Artificial Analysis (independent third-party speed + quality benchmarks)
Academic benchmarks (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Architecture

 ┌─────────────────────┐
│ GitHub Actions │ daily 02:00 UTC cron
│ scrapers/run.py │
└──────────┬──────────┘
│
① discovery + auto-promote ▼
┌──────────────────────────────────────────┐
│ discovery: vendor Models APIs (trusted) │→ unseen models auto-registered
│ + benchmark boards (noise→log) │
└───────────────┬──────────────────────────┘
│ register_extra → unified registry (catalog ∪ auto-discovered)
② scrape ▼
┌──────────────────────────────────────────┐
│ vendor scrapers benchmarks LLM fallback │
│ (price/specs) (Elo/scores) (Claude Haiku)│
└───────────────┬──────────────────────────┘
│ normalized exact matching (model_registry) — never mismatch
③ validation gate▼
┌──────────────────────────────────────────┐
│ validation: price >3× / ELO >100 jumps │ quarantined, no dirty writes
│ benchmark coverage drop alert │
└───────────────┬──────────────────────────┘
▼
┌────────────────┐
│ Supabase │ Postgres + RLS
│ models/prices │ benchmark_scores · daily_snapshots
│ discovery_* │ pending_changes · scrape_errors
└────────┬───────┘
▼
┌────────────────┐
│ Next.js ISR │ apps/web (incl. /health data-health page)
│ on Vercel CDN │ push→git auto-deploy; data refreshes via ISR
└────────────────┘

Robustness design (see the "robustness overhaul" section in IMPLEMENTATION_CHECKLIST.md):

Never miss a new model: vendor official Models APIs are the trusted signal — any unseen model is auto-registered (sparse metadata; price/Elo get filled in later by the scrape layer, never fabricated). Arena/AA leaderboard names are noise: never auto-registered, never nagging, only optionally viewable on /health.
Never mismatch: a single identity registry (core/model_registry.py) derived from catalog ∪ auto-discovered models, with normalized exact matching (controlled stripping of -thinking/date-style suffixes only — size/version is never stripped). CI enforces zero alias collisions. Each model is defined in exactly one place.
No dirty values: an anomaly gate quarantines wild jumps (the root cause of prices once flip-flopping); a value is only confirmed after appearing 2 consecutive times.
No silent failures: an unmatched name is logged as a discovery candidate + drift alert — everything surfaces on /health.

Directory layout

apps/web/ Next.js frontend (incl. /health data-health page)
scrapers/ Python scrapers
vendors/ one module per vendor (catalog-driven)
benchmarks/ LMSYS / Artificial Analysis / academic
discovery/ discovery layer: trusted vendor Models APIs etc.
core/
model_registry.py single identity registry (catalog ∪ auto-discovered, exact matching)
discovery.py discovery filtering + vendor inference (pure logic)
promotion.py auto-promotion: trusted source → model record (pure logic)
validation.py price/ELO anomaly gate (pure logic)
extractor / db / differ / registry
tests/ registry / discovery / validation / promotion unit tests (CI)
alert_candidates.py data-health alerts (quarantined values) → GitHub issue
supabase/migrations/ Postgres schema (0001–0007)
.github/workflows/ scrape-daily (cron) + test (pytest on PR)

Local development

Frontend

cd apps/web
npm install
cp ../../.env.example .env.local # fill in SUPABASE_URL + SUPABASE_ANON_KEY
npm run dev

Scrapers

python3 -m venv .venv
source .venv/bin/activate
pip install -r scrapers/requirements.txt
playwright install chromium
cp .env.example .env # fill in all KEYs
python -m scrapers.run --dry-run # full run without writing to the DB
python -m scrapers.run # real run
python -m scrapers.run --vendor openai # single vendor only
python -m scrapers.run --skip-discovery # skip discovery/auto-promotion
pytest scrapers/tests/ # unit tests (registry/discovery/validation/promotion)

Supabase

# Create a project at https://supabase.com/dashboard, grab the URL + Service Key.# Run all migrations in order in the SQL Editor, then load the seed:# supabase/migrations/0001_initial.sql initial 6 tables + views + RLS# supabase/migrations/0002_dedupe_benchmarks.sql# supabase/migrations/0003_add_model_license.sql# supabase/migrations/0004_discovery_candidates.sql discovery candidates table# supabase/migrations/0005_pending_changes.sql anomaly quarantine table# supabase/migrations/0006_data_health_views.sql sanitized error views# supabase/migrations/0007_auto_discovered.sql models.auto_discovered# supabase/seed.sql

Data schema

Table / view	Description
`vendors`	Vendor master data (OpenAI / Anthropic / ...)
`models`	Model master data, one row per model; `auto_discovered` marks API auto-promotion
`prices`	Price snapshots, append-only, full history retained
`benchmark_scores`	Benchmark snapshots, append-only
`daily_snapshots`	Daily summary + that day's changes as JSON (incl. discovery candidates)
`discovery_candidates`	Model names surfaced by discovery but not yet registered (proposals only, tiered by vendor)
`pending_changes`	Suspicious values quarantined by the anomaly gate (auto-confirmed after 2 consecutive identical readings)
`scrape_errors`	Scraper error log (incl. drift alerts)
`models_overview`(view)	Models + current price + Arena Elo aggregate, read by the frontend
`recent_scrape_issues`(view)	Sanitized projection of `scrape_errors` (no traceback/url), for /health

Data sources & attribution

This project aggregates publicly available information from:

Vendor pricing pages — listed in supabase/seed.sql per vendor (pricing_url)
LMSYS Chatbot Arena — Elo leaderboard
Artificial Analysis — independent third-party benchmarks
Official vendor announcements — for academic benchmark numbers (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Every prices and benchmark_scores row carries a source_url pointing back to the upstream.

Disclaimer

Prices are best-effort. Always confirm against the vendor's official pricing page before making a procurement decision. Scrapers can miss page changes and the fallback_prices baked into vendor files may be stale.
Benchmark numbers are reported figures, not independently re-run. Where vendors and third parties disagree, both are shown when available; trust your own evaluation.
No affiliation. This project is not affiliated with or endorsed by any vendor listed.

License

MIT — see LICENSE file. Contributions welcome — see CONTRIBUTING.md. Security issues — see SECURITY.md.

model.tracker

English | 简体中文

Automatically tracks model releases, pricing, and performance across the world's major AI vendors. Updated daily.

Coverage

14 vendors:

International: OpenAI · Anthropic · Google · Meta · Mistral · xAI · Cohere
China: DeepSeek · Qwen (Alibaba) · Zhipu GLM · Doubao (ByteDance) · Kimi (Moonshot) · Baichuan · Hunyuan (Tencent)

3 kinds of performance data sources:

LMSYS Chatbot Arena (human blind-test Elo)
Artificial Analysis (independent third-party speed + quality benchmarks)
Academic benchmarks (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Architecture

 ┌─────────────────────┐
│ GitHub Actions │ daily 02:00 UTC cron
│ scrapers/run.py │
└──────────┬──────────┘
│
① discovery + auto-promote ▼
┌──────────────────────────────────────────┐
│ discovery: vendor Models APIs (trusted) │→ unseen models auto-registered
│ + benchmark boards (noise→log) │
└───────────────┬──────────────────────────┘
│ register_extra → unified registry (catalog ∪ auto-discovered)
② scrape ▼
┌──────────────────────────────────────────┐
│ vendor scrapers benchmarks LLM fallback │
│ (price/specs) (Elo/scores) (Claude Haiku)│
└───────────────┬──────────────────────────┘
│ normalized exact matching (model_registry) — never mismatch
③ validation gate▼
┌──────────────────────────────────────────┐
│ validation: price >3× / ELO >100 jumps │ quarantined, no dirty writes
│ benchmark coverage drop alert │
└───────────────┬──────────────────────────┘
▼
┌────────────────┐
│ Supabase │ Postgres + RLS
│ models/prices │ benchmark_scores · daily_snapshots
│ discovery_* │ pending_changes · scrape_errors
└────────┬───────┘
▼
┌────────────────┐
│ Next.js ISR │ apps/web (incl. /health data-health page)
│ on Vercel CDN │ push→git auto-deploy; data refreshes via ISR
└────────────────┘

Robustness design (see the "robustness overhaul" section in IMPLEMENTATION_CHECKLIST.md):

Never miss a new model: vendor official Models APIs are the trusted signal — any unseen model is auto-registered (sparse metadata; price/Elo get filled in later by the scrape layer, never fabricated). Arena/AA leaderboard names are noise: never auto-registered, never nagging, only optionally viewable on /health.
Never mismatch: a single identity registry (core/model_registry.py) derived from catalog ∪ auto-discovered models, with normalized exact matching (controlled stripping of -thinking/date-style suffixes only — size/version is never stripped). CI enforces zero alias collisions. Each model is defined in exactly one place.
No dirty values: an anomaly gate quarantines wild jumps (the root cause of prices once flip-flopping); a value is only confirmed after appearing 2 consecutive times.
No silent failures: an unmatched name is logged as a discovery candidate + drift alert — everything surfaces on /health.

Directory layout

apps/web/ Next.js frontend (incl. /health data-health page)
scrapers/ Python scrapers
vendors/ one module per vendor (catalog-driven)
benchmarks/ LMSYS / Artificial Analysis / academic
discovery/ discovery layer: trusted vendor Models APIs etc.
core/
model_registry.py single identity registry (catalog ∪ auto-discovered, exact matching)
discovery.py discovery filtering + vendor inference (pure logic)
promotion.py auto-promotion: trusted source → model record (pure logic)
validation.py price/ELO anomaly gate (pure logic)
extractor / db / differ / registry
tests/ registry / discovery / validation / promotion unit tests (CI)
alert_candidates.py data-health alerts (quarantined values) → GitHub issue
supabase/migrations/ Postgres schema (0001–0007)
.github/workflows/ scrape-daily (cron) + test (pytest on PR)

Local development

Frontend

cd apps/web
npm install
cp ../../.env.example .env.local # fill in SUPABASE_URL + SUPABASE_ANON_KEY
npm run dev

Scrapers

python3 -m venv .venv
source .venv/bin/activate
pip install -r scrapers/requirements.txt
playwright install chromium
cp .env.example .env # fill in all KEYs
python -m scrapers.run --dry-run # full run without writing to the DB
python -m scrapers.run # real run
python -m scrapers.run --vendor openai # single vendor only
python -m scrapers.run --skip-discovery # skip discovery/auto-promotion
pytest scrapers/tests/ # unit tests (registry/discovery/validation/promotion)

Supabase

# Create a project at https://supabase.com/dashboard, grab the URL + Service Key.# Run all migrations in order in the SQL Editor, then load the seed:# supabase/migrations/0001_initial.sql initial 6 tables + views + RLS# supabase/migrations/0002_dedupe_benchmarks.sql# supabase/migrations/0003_add_model_license.sql# supabase/migrations/0004_discovery_candidates.sql discovery candidates table# supabase/migrations/0005_pending_changes.sql anomaly quarantine table# supabase/migrations/0006_data_health_views.sql sanitized error views# supabase/migrations/0007_auto_discovered.sql models.auto_discovered# supabase/seed.sql

Data schema

Table / view	Description
`vendors`	Vendor master data (OpenAI / Anthropic / ...)
`models`	Model master data, one row per model; `auto_discovered` marks API auto-promotion
`prices`	Price snapshots, append-only, full history retained
`benchmark_scores`	Benchmark snapshots, append-only
`daily_snapshots`	Daily summary + that day's changes as JSON (incl. discovery candidates)
`discovery_candidates`	Model names surfaced by discovery but not yet registered (proposals only, tiered by vendor)
`pending_changes`	Suspicious values quarantined by the anomaly gate (auto-confirmed after 2 consecutive identical readings)
`scrape_errors`	Scraper error log (incl. drift alerts)
`models_overview`(view)	Models + current price + Arena Elo aggregate, read by the frontend
`recent_scrape_issues`(view)	Sanitized projection of `scrape_errors` (no traceback/url), for /health

Data sources & attribution

This project aggregates publicly available information from:

Vendor pricing pages — listed in supabase/seed.sql per vendor (pricing_url)
LMSYS Chatbot Arena — Elo leaderboard
Artificial Analysis — independent third-party benchmarks
Official vendor announcements — for academic benchmark numbers (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Every prices and benchmark_scores row carries a source_url pointing back to the upstream.

Disclaimer

Prices are best-effort. Always confirm against the vendor's official pricing page before making a procurement decision. Scrapers can miss page changes and the fallback_prices baked into vendor files may be stale.
Benchmark numbers are reported figures, not independently re-run. Where vendors and third parties disagree, both are shown when available; trust your own evaluation.
No affiliation. This project is not affiliated with or endorsed by any vendor listed.

License

MIT — see LICENSE file. Contributions welcome — see CONTRIBUTING.md. Security issues — see SECURITY.md.

model.tracker

English | 简体中文

Automatically tracks model releases, pricing, and performance across the world's major AI vendors. Updated daily.

Coverage

14 vendors:

International: OpenAI · Anthropic · Google · Meta · Mistral · xAI · Cohere
China: DeepSeek · Qwen (Alibaba) · Zhipu GLM · Doubao (ByteDance) · Kimi (Moonshot) · Baichuan · Hunyuan (Tencent)

3 kinds of performance data sources:

LMSYS Chatbot Arena (human blind-test Elo)
Artificial Analysis (independent third-party speed + quality benchmarks)
Academic benchmarks (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Architecture

 ┌─────────────────────┐
│ GitHub Actions │ daily 02:00 UTC cron
│ scrapers/run.py │
└──────────┬──────────┘
│
① discovery + auto-promote ▼
┌──────────────────────────────────────────┐
│ discovery: vendor Models APIs (trusted) │→ unseen models auto-registered
│ + benchmark boards (noise→log) │
└───────────────┬──────────────────────────┘
│ register_extra → unified registry (catalog ∪ auto-discovered)
② scrape ▼
┌──────────────────────────────────────────┐
│ vendor scrapers benchmarks LLM fallback │
│ (price/specs) (Elo/scores) (Claude Haiku)│
└───────────────┬──────────────────────────┘
│ normalized exact matching (model_registry) — never mismatch
③ validation gate▼
┌──────────────────────────────────────────┐
│ validation: price >3× / ELO >100 jumps │ quarantined, no dirty writes
│ benchmark coverage drop alert │
└───────────────┬──────────────────────────┘
▼
┌────────────────┐
│ Supabase │ Postgres + RLS
│ models/prices │ benchmark_scores · daily_snapshots
│ discovery_* │ pending_changes · scrape_errors
└────────┬───────┘
▼
┌────────────────┐
│ Next.js ISR │ apps/web (incl. /health data-health page)
│ on Vercel CDN │ push→git auto-deploy; data refreshes via ISR
└────────────────┘

Robustness design (see the "robustness overhaul" section in IMPLEMENTATION_CHECKLIST.md):

Never miss a new model: vendor official Models APIs are the trusted signal — any unseen model is auto-registered (sparse metadata; price/Elo get filled in later by the scrape layer, never fabricated). Arena/AA leaderboard names are noise: never auto-registered, never nagging, only optionally viewable on /health.
Never mismatch: a single identity registry (core/model_registry.py) derived from catalog ∪ auto-discovered models, with normalized exact matching (controlled stripping of -thinking/date-style suffixes only — size/version is never stripped). CI enforces zero alias collisions. Each model is defined in exactly one place.
No dirty values: an anomaly gate quarantines wild jumps (the root cause of prices once flip-flopping); a value is only confirmed after appearing 2 consecutive times.
No silent failures: an unmatched name is logged as a discovery candidate + drift alert — everything surfaces on /health.

Directory layout

apps/web/ Next.js frontend (incl. /health data-health page)
scrapers/ Python scrapers
vendors/ one module per vendor (catalog-driven)
benchmarks/ LMSYS / Artificial Analysis / academic
discovery/ discovery layer: trusted vendor Models APIs etc.
core/
model_registry.py single identity registry (catalog ∪ auto-discovered, exact matching)
discovery.py discovery filtering + vendor inference (pure logic)
promotion.py auto-promotion: trusted source → model record (pure logic)
validation.py price/ELO anomaly gate (pure logic)
extractor / db / differ / registry
tests/ registry / discovery / validation / promotion unit tests (CI)
alert_candidates.py data-health alerts (quarantined values) → GitHub issue
supabase/migrations/ Postgres schema (0001–0007)
.github/workflows/ scrape-daily (cron) + test (pytest on PR)

Local development

Frontend

cd apps/web
npm install
cp ../../.env.example .env.local # fill in SUPABASE_URL + SUPABASE_ANON_KEY
npm run dev

Scrapers

python3 -m venv .venv
source .venv/bin/activate
pip install -r scrapers/requirements.txt
playwright install chromium
cp .env.example .env # fill in all KEYs
python -m scrapers.run --dry-run # full run without writing to the DB
python -m scrapers.run # real run
python -m scrapers.run --vendor openai # single vendor only
python -m scrapers.run --skip-discovery # skip discovery/auto-promotion
pytest scrapers/tests/ # unit tests (registry/discovery/validation/promotion)

Supabase

# Create a project at https://supabase.com/dashboard, grab the URL + Service Key.# Run all migrations in order in the SQL Editor, then load the seed:# supabase/migrations/0001_initial.sql initial 6 tables + views + RLS# supabase/migrations/0002_dedupe_benchmarks.sql# supabase/migrations/0003_add_model_license.sql# supabase/migrations/0004_discovery_candidates.sql discovery candidates table# supabase/migrations/0005_pending_changes.sql anomaly quarantine table# supabase/migrations/0006_data_health_views.sql sanitized error views# supabase/migrations/0007_auto_discovered.sql models.auto_discovered# supabase/seed.sql

Data schema

Table / view	Description
`vendors`	Vendor master data (OpenAI / Anthropic / ...)
`models`	Model master data, one row per model; `auto_discovered` marks API auto-promotion
`prices`	Price snapshots, append-only, full history retained
`benchmark_scores`	Benchmark snapshots, append-only
`daily_snapshots`	Daily summary + that day's changes as JSON (incl. discovery candidates)
`discovery_candidates`	Model names surfaced by discovery but not yet registered (proposals only, tiered by vendor)
`pending_changes`	Suspicious values quarantined by the anomaly gate (auto-confirmed after 2 consecutive identical readings)
`scrape_errors`	Scraper error log (incl. drift alerts)
`models_overview`(view)	Models + current price + Arena Elo aggregate, read by the frontend
`recent_scrape_issues`(view)	Sanitized projection of `scrape_errors` (no traceback/url), for /health

Data sources & attribution

This project aggregates publicly available information from:

Vendor pricing pages — listed in supabase/seed.sql per vendor (pricing_url)
LMSYS Chatbot Arena — Elo leaderboard
Artificial Analysis — independent third-party benchmarks
Official vendor announcements — for academic benchmark numbers (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Every prices and benchmark_scores row carries a source_url pointing back to the upstream.

Disclaimer

Prices are best-effort. Always confirm against the vendor's official pricing page before making a procurement decision. Scrapers can miss page changes and the fallback_prices baked into vendor files may be stale.
Benchmark numbers are reported figures, not independently re-run. Where vendors and third parties disagree, both are shown when available; trust your own evaluation.
No affiliation. This project is not affiliated with or endorsed by any vendor listed.

License

MIT — see LICENSE file. Contributions welcome — see CONTRIBUTING.md. Security issues — see SECURITY.md.

model.tracker

English | 简体中文

Automatically tracks model releases, pricing, and performance across the world's major AI vendors. Updated daily.

Coverage

14 vendors:

International: OpenAI · Anthropic · Google · Meta · Mistral · xAI · Cohere
China: DeepSeek · Qwen (Alibaba) · Zhipu GLM · Doubao (ByteDance) · Kimi (Moonshot) · Baichuan · Hunyuan (Tencent)

3 kinds of performance data sources:

LMSYS Chatbot Arena (human blind-test Elo)
Artificial Analysis (independent third-party speed + quality benchmarks)
Academic benchmarks (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Architecture

 ┌─────────────────────┐
│ GitHub Actions │ daily 02:00 UTC cron
│ scrapers/run.py │
└──────────┬──────────┘
│
① discovery + auto-promote ▼
┌──────────────────────────────────────────┐
│ discovery: vendor Models APIs (trusted) │→ unseen models auto-registered
│ + benchmark boards (noise→log) │
└───────────────┬──────────────────────────┘
│ register_extra → unified registry (catalog ∪ auto-discovered)
② scrape ▼
┌──────────────────────────────────────────┐
│ vendor scrapers benchmarks LLM fallback │
│ (price/specs) (Elo/scores) (Claude Haiku)│
└───────────────┬──────────────────────────┘
│ normalized exact matching (model_registry) — never mismatch
③ validation gate▼
┌──────────────────────────────────────────┐
│ validation: price >3× / ELO >100 jumps │ quarantined, no dirty writes
│ benchmark coverage drop alert │
└───────────────┬──────────────────────────┘
▼
┌────────────────┐
│ Supabase │ Postgres + RLS
│ models/prices │ benchmark_scores · daily_snapshots
│ discovery_* │ pending_changes · scrape_errors
└────────┬───────┘
▼
┌────────────────┐
│ Next.js ISR │ apps/web (incl. /health data-health page)
│ on Vercel CDN │ push→git auto-deploy; data refreshes via ISR
└────────────────┘

Robustness design (see the "robustness overhaul" section in IMPLEMENTATION_CHECKLIST.md):

Never miss a new model: vendor official Models APIs are the trusted signal — any unseen model is auto-registered (sparse metadata; price/Elo get filled in later by the scrape layer, never fabricated). Arena/AA leaderboard names are noise: never auto-registered, never nagging, only optionally viewable on /health.
Never mismatch: a single identity registry (core/model_registry.py) derived from catalog ∪ auto-discovered models, with normalized exact matching (controlled stripping of -thinking/date-style suffixes only — size/version is never stripped). CI enforces zero alias collisions. Each model is defined in exactly one place.
No dirty values: an anomaly gate quarantines wild jumps (the root cause of prices once flip-flopping); a value is only confirmed after appearing 2 consecutive times.
No silent failures: an unmatched name is logged as a discovery candidate + drift alert — everything surfaces on /health.

Directory layout

apps/web/ Next.js frontend (incl. /health data-health page)
scrapers/ Python scrapers
vendors/ one module per vendor (catalog-driven)
benchmarks/ LMSYS / Artificial Analysis / academic
discovery/ discovery layer: trusted vendor Models APIs etc.
core/
model_registry.py single identity registry (catalog ∪ auto-discovered, exact matching)
discovery.py discovery filtering + vendor inference (pure logic)
promotion.py auto-promotion: trusted source → model record (pure logic)
validation.py price/ELO anomaly gate (pure logic)
extractor / db / differ / registry
tests/ registry / discovery / validation / promotion unit tests (CI)
alert_candidates.py data-health alerts (quarantined values) → GitHub issue
supabase/migrations/ Postgres schema (0001–0007)
.github/workflows/ scrape-daily (cron) + test (pytest on PR)

Local development

Frontend

cd apps/web
npm install
cp ../../.env.example .env.local # fill in SUPABASE_URL + SUPABASE_ANON_KEY
npm run dev

Scrapers

python3 -m venv .venv
source .venv/bin/activate
pip install -r scrapers/requirements.txt
playwright install chromium
cp .env.example .env # fill in all KEYs
python -m scrapers.run --dry-run # full run without writing to the DB
python -m scrapers.run # real run
python -m scrapers.run --vendor openai # single vendor only
python -m scrapers.run --skip-discovery # skip discovery/auto-promotion
pytest scrapers/tests/ # unit tests (registry/discovery/validation/promotion)

Supabase

# Create a project at https://supabase.com/dashboard, grab the URL + Service Key.# Run all migrations in order in the SQL Editor, then load the seed:# supabase/migrations/0001_initial.sql initial 6 tables + views + RLS# supabase/migrations/0002_dedupe_benchmarks.sql# supabase/migrations/0003_add_model_license.sql# supabase/migrations/0004_discovery_candidates.sql discovery candidates table# supabase/migrations/0005_pending_changes.sql anomaly quarantine table# supabase/migrations/0006_data_health_views.sql sanitized error views# supabase/migrations/0007_auto_discovered.sql models.auto_discovered# supabase/seed.sql

Data schema

Table / view	Description
`vendors`	Vendor master data (OpenAI / Anthropic / ...)
`models`	Model master data, one row per model; `auto_discovered` marks API auto-promotion
`prices`	Price snapshots, append-only, full history retained
`benchmark_scores`	Benchmark snapshots, append-only
`daily_snapshots`	Daily summary + that day's changes as JSON (incl. discovery candidates)
`discovery_candidates`	Model names surfaced by discovery but not yet registered (proposals only, tiered by vendor)
`pending_changes`	Suspicious values quarantined by the anomaly gate (auto-confirmed after 2 consecutive identical readings)
`scrape_errors`	Scraper error log (incl. drift alerts)
`models_overview`(view)	Models + current price + Arena Elo aggregate, read by the frontend
`recent_scrape_issues`(view)	Sanitized projection of `scrape_errors` (no traceback/url), for /health

Data sources & attribution

This project aggregates publicly available information from:

Vendor pricing pages — listed in supabase/seed.sql per vendor (pricing_url)
LMSYS Chatbot Arena — Elo leaderboard
Artificial Analysis — independent third-party benchmarks
Official vendor announcements — for academic benchmark numbers (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Every prices and benchmark_scores row carries a source_url pointing back to the upstream.

Disclaimer

Prices are best-effort. Always confirm against the vendor's official pricing page before making a procurement decision. Scrapers can miss page changes and the fallback_prices baked into vendor files may be stale.
Benchmark numbers are reported figures, not independently re-run. Where vendors and third parties disagree, both are shown when available; trust your own evaluation.
No affiliation. This project is not affiliated with or endorsed by any vendor listed.

License

MIT — see LICENSE file. Contributions welcome — see CONTRIBUTING.md. Security issues — see SECURITY.md.

model.tracker

English | 简体中文

Automatically tracks model releases, pricing, and performance across the world's major AI vendors. Updated daily.

Coverage

14 vendors:

International: OpenAI · Anthropic · Google · Meta · Mistral · xAI · Cohere
China: DeepSeek · Qwen (Alibaba) · Zhipu GLM · Doubao (ByteDance) · Kimi (Moonshot) · Baichuan · Hunyuan (Tencent)

3 kinds of performance data sources:

LMSYS Chatbot Arena (human blind-test Elo)
Artificial Analysis (independent third-party speed + quality benchmarks)
Academic benchmarks (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Architecture

 ┌─────────────────────┐
│ GitHub Actions │ daily 02:00 UTC cron
│ scrapers/run.py │
└──────────┬──────────┘
│
① discovery + auto-promote ▼
┌──────────────────────────────────────────┐
│ discovery: vendor Models APIs (trusted) │→ unseen models auto-registered
│ + benchmark boards (noise→log) │
└───────────────┬──────────────────────────┘
│ register_extra → unified registry (catalog ∪ auto-discovered)
② scrape ▼
┌──────────────────────────────────────────┐
│ vendor scrapers benchmarks LLM fallback │
│ (price/specs) (Elo/scores) (Claude Haiku)│
└───────────────┬──────────────────────────┘
│ normalized exact matching (model_registry) — never mismatch
③ validation gate▼
┌──────────────────────────────────────────┐
│ validation: price >3× / ELO >100 jumps │ quarantined, no dirty writes
│ benchmark coverage drop alert │
└───────────────┬──────────────────────────┘
▼
┌────────────────┐
│ Supabase │ Postgres + RLS
│ models/prices │ benchmark_scores · daily_snapshots
│ discovery_* │ pending_changes · scrape_errors
└────────┬───────┘
▼
┌────────────────┐
│ Next.js ISR │ apps/web (incl. /health data-health page)
│ on Vercel CDN │ push→git auto-deploy; data refreshes via ISR
└────────────────┘

Robustness design (see the "robustness overhaul" section in IMPLEMENTATION_CHECKLIST.md):

Never miss a new model: vendor official Models APIs are the trusted signal — any unseen model is auto-registered (sparse metadata; price/Elo get filled in later by the scrape layer, never fabricated). Arena/AA leaderboard names are noise: never auto-registered, never nagging, only optionally viewable on /health.
Never mismatch: a single identity registry (core/model_registry.py) derived from catalog ∪ auto-discovered models, with normalized exact matching (controlled stripping of -thinking/date-style suffixes only — size/version is never stripped). CI enforces zero alias collisions. Each model is defined in exactly one place.
No dirty values: an anomaly gate quarantines wild jumps (the root cause of prices once flip-flopping); a value is only confirmed after appearing 2 consecutive times.
No silent failures: an unmatched name is logged as a discovery candidate + drift alert — everything surfaces on /health.

Directory layout

apps/web/ Next.js frontend (incl. /health data-health page)
scrapers/ Python scrapers
vendors/ one module per vendor (catalog-driven)
benchmarks/ LMSYS / Artificial Analysis / academic
discovery/ discovery layer: trusted vendor Models APIs etc.
core/
model_registry.py single identity registry (catalog ∪ auto-discovered, exact matching)
discovery.py discovery filtering + vendor inference (pure logic)
promotion.py auto-promotion: trusted source → model record (pure logic)
validation.py price/ELO anomaly gate (pure logic)
extractor / db / differ / registry
tests/ registry / discovery / validation / promotion unit tests (CI)
alert_candidates.py data-health alerts (quarantined values) → GitHub issue
supabase/migrations/ Postgres schema (0001–0007)
.github/workflows/ scrape-daily (cron) + test (pytest on PR)

Local development

Frontend

cd apps/web
npm install
cp ../../.env.example .env.local # fill in SUPABASE_URL + SUPABASE_ANON_KEY
npm run dev

Scrapers

python3 -m venv .venv
source .venv/bin/activate
pip install -r scrapers/requirements.txt
playwright install chromium
cp .env.example .env # fill in all KEYs
python -m scrapers.run --dry-run # full run without writing to the DB
python -m scrapers.run # real run
python -m scrapers.run --vendor openai # single vendor only
python -m scrapers.run --skip-discovery # skip discovery/auto-promotion
pytest scrapers/tests/ # unit tests (registry/discovery/validation/promotion)

Supabase

# Create a project at https://supabase.com/dashboard, grab the URL + Service Key.# Run all migrations in order in the SQL Editor, then load the seed:# supabase/migrations/0001_initial.sql initial 6 tables + views + RLS# supabase/migrations/0002_dedupe_benchmarks.sql# supabase/migrations/0003_add_model_license.sql# supabase/migrations/0004_discovery_candidates.sql discovery candidates table# supabase/migrations/0005_pending_changes.sql anomaly quarantine table# supabase/migrations/0006_data_health_views.sql sanitized error views# supabase/migrations/0007_auto_discovered.sql models.auto_discovered# supabase/seed.sql

Data schema

Table / view	Description
`vendors`	Vendor master data (OpenAI / Anthropic / ...)
`models`	Model master data, one row per model; `auto_discovered` marks API auto-promotion
`prices`	Price snapshots, append-only, full history retained
`benchmark_scores`	Benchmark snapshots, append-only
`daily_snapshots`	Daily summary + that day's changes as JSON (incl. discovery candidates)
`discovery_candidates`	Model names surfaced by discovery but not yet registered (proposals only, tiered by vendor)
`pending_changes`	Suspicious values quarantined by the anomaly gate (auto-confirmed after 2 consecutive identical readings)
`scrape_errors`	Scraper error log (incl. drift alerts)
`models_overview`(view)	Models + current price + Arena Elo aggregate, read by the frontend
`recent_scrape_issues`(view)	Sanitized projection of `scrape_errors` (no traceback/url), for /health

Data sources & attribution

This project aggregates publicly available information from:

Vendor pricing pages — listed in supabase/seed.sql per vendor (pricing_url)
LMSYS Chatbot Arena — Elo leaderboard
Artificial Analysis — independent third-party benchmarks
Official vendor announcements — for academic benchmark numbers (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Every prices and benchmark_scores row carries a source_url pointing back to the upstream.

Disclaimer

Prices are best-effort. Always confirm against the vendor's official pricing page before making a procurement decision. Scrapers can miss page changes and the fallback_prices baked into vendor files may be stale.
Benchmark numbers are reported figures, not independently re-run. Where vendors and third parties disagree, both are shown when available; trust your own evaluation.
No affiliation. This project is not affiliated with or endorsed by any vendor listed.

License

MIT — see LICENSE file. Contributions welcome — see CONTRIBUTING.md. Security issues — see SECURITY.md.

model.tracker

English | 简体中文

Automatically tracks model releases, pricing, and performance across the world's major AI vendors. Updated daily.

Coverage

14 vendors:

International: OpenAI · Anthropic · Google · Meta · Mistral · xAI · Cohere
China: DeepSeek · Qwen (Alibaba) · Zhipu GLM · Doubao (ByteDance) · Kimi (Moonshot) · Baichuan · Hunyuan (Tencent)

3 kinds of performance data sources:

LMSYS Chatbot Arena (human blind-test Elo)
Artificial Analysis (independent third-party speed + quality benchmarks)
Academic benchmarks (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Architecture

 ┌─────────────────────┐
│ GitHub Actions │ daily 02:00 UTC cron
│ scrapers/run.py │
└──────────┬──────────┘
│
① discovery + auto-promote ▼
┌──────────────────────────────────────────┐
│ discovery: vendor Models APIs (trusted) │→ unseen models auto-registered
│ + benchmark boards (noise→log) │
└───────────────┬──────────────────────────┘
│ register_extra → unified registry (catalog ∪ auto-discovered)
② scrape ▼
┌──────────────────────────────────────────┐
│ vendor scrapers benchmarks LLM fallback │
│ (price/specs) (Elo/scores) (Claude Haiku)│
└───────────────┬──────────────────────────┘
│ normalized exact matching (model_registry) — never mismatch
③ validation gate▼
┌──────────────────────────────────────────┐
│ validation: price >3× / ELO >100 jumps │ quarantined, no dirty writes
│ benchmark coverage drop alert │
└───────────────┬──────────────────────────┘
▼
┌────────────────┐
│ Supabase │ Postgres + RLS
│ models/prices │ benchmark_scores · daily_snapshots
│ discovery_* │ pending_changes · scrape_errors
└────────┬───────┘
▼
┌────────────────┐
│ Next.js ISR │ apps/web (incl. /health data-health page)
│ on Vercel CDN │ push→git auto-deploy; data refreshes via ISR
└────────────────┘

Robustness design (see the "robustness overhaul" section in IMPLEMENTATION_CHECKLIST.md):

Never miss a new model: vendor official Models APIs are the trusted signal — any unseen model is auto-registered (sparse metadata; price/Elo get filled in later by the scrape layer, never fabricated). Arena/AA leaderboard names are noise: never auto-registered, never nagging, only optionally viewable on /health.
Never mismatch: a single identity registry (core/model_registry.py) derived from catalog ∪ auto-discovered models, with normalized exact matching (controlled stripping of -thinking/date-style suffixes only — size/version is never stripped). CI enforces zero alias collisions. Each model is defined in exactly one place.
No dirty values: an anomaly gate quarantines wild jumps (the root cause of prices once flip-flopping); a value is only confirmed after appearing 2 consecutive times.
No silent failures: an unmatched name is logged as a discovery candidate + drift alert — everything surfaces on /health.

Directory layout

apps/web/ Next.js frontend (incl. /health data-health page)
scrapers/ Python scrapers
vendors/ one module per vendor (catalog-driven)
benchmarks/ LMSYS / Artificial Analysis / academic
discovery/ discovery layer: trusted vendor Models APIs etc.
core/
model_registry.py single identity registry (catalog ∪ auto-discovered, exact matching)
discovery.py discovery filtering + vendor inference (pure logic)
promotion.py auto-promotion: trusted source → model record (pure logic)
validation.py price/ELO anomaly gate (pure logic)
extractor / db / differ / registry
tests/ registry / discovery / validation / promotion unit tests (CI)
alert_candidates.py data-health alerts (quarantined values) → GitHub issue
supabase/migrations/ Postgres schema (0001–0007)
.github/workflows/ scrape-daily (cron) + test (pytest on PR)

Local development

Frontend

cd apps/web
npm install
cp ../../.env.example .env.local # fill in SUPABASE_URL + SUPABASE_ANON_KEY
npm run dev

Scrapers

python3 -m venv .venv
source .venv/bin/activate
pip install -r scrapers/requirements.txt
playwright install chromium
cp .env.example .env # fill in all KEYs
python -m scrapers.run --dry-run # full run without writing to the DB
python -m scrapers.run # real run
python -m scrapers.run --vendor openai # single vendor only
python -m scrapers.run --skip-discovery # skip discovery/auto-promotion
pytest scrapers/tests/ # unit tests (registry/discovery/validation/promotion)

Supabase

# Create a project at https://supabase.com/dashboard, grab the URL + Service Key.# Run all migrations in order in the SQL Editor, then load the seed:# supabase/migrations/0001_initial.sql initial 6 tables + views + RLS# supabase/migrations/0002_dedupe_benchmarks.sql# supabase/migrations/0003_add_model_license.sql# supabase/migrations/0004_discovery_candidates.sql discovery candidates table# supabase/migrations/0005_pending_changes.sql anomaly quarantine table# supabase/migrations/0006_data_health_views.sql sanitized error views# supabase/migrations/0007_auto_discovered.sql models.auto_discovered# supabase/seed.sql

Data schema

Table / view	Description
`vendors`	Vendor master data (OpenAI / Anthropic / ...)
`models`	Model master data, one row per model; `auto_discovered` marks API auto-promotion
`prices`	Price snapshots, append-only, full history retained
`benchmark_scores`	Benchmark snapshots, append-only
`daily_snapshots`	Daily summary + that day's changes as JSON (incl. discovery candidates)
`discovery_candidates`	Model names surfaced by discovery but not yet registered (proposals only, tiered by vendor)
`pending_changes`	Suspicious values quarantined by the anomaly gate (auto-confirmed after 2 consecutive identical readings)
`scrape_errors`	Scraper error log (incl. drift alerts)
`models_overview`(view)	Models + current price + Arena Elo aggregate, read by the frontend
`recent_scrape_issues`(view)	Sanitized projection of `scrape_errors` (no traceback/url), for /health

Data sources & attribution

This project aggregates publicly available information from:

Vendor pricing pages — listed in supabase/seed.sql per vendor (pricing_url)
LMSYS Chatbot Arena — Elo leaderboard
Artificial Analysis — independent third-party benchmarks
Official vendor announcements — for academic benchmark numbers (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Every prices and benchmark_scores row carries a source_url pointing back to the upstream.

Disclaimer

Prices are best-effort. Always confirm against the vendor's official pricing page before making a procurement decision. Scrapers can miss page changes and the fallback_prices baked into vendor files may be stale.
Benchmark numbers are reported figures, not independently re-run. Where vendors and third parties disagree, both are shown when available; trust your own evaluation.
No affiliation. This project is not affiliated with or endorsed by any vendor listed.

License

MIT — see LICENSE file. Contributions welcome — see CONTRIBUTING.md. Security issues — see SECURITY.md.

model.tracker

English | 简体中文

Automatically tracks model releases, pricing, and performance across the world's major AI vendors. Updated daily.

Coverage

14 vendors:

International: OpenAI · Anthropic · Google · Meta · Mistral · xAI · Cohere
China: DeepSeek · Qwen (Alibaba) · Zhipu GLM · Doubao (ByteDance) · Kimi (Moonshot) · Baichuan · Hunyuan (Tencent)

3 kinds of performance data sources:

LMSYS Chatbot Arena (human blind-test Elo)
Artificial Analysis (independent third-party speed + quality benchmarks)
Academic benchmarks (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Architecture

 ┌─────────────────────┐
│ GitHub Actions │ daily 02:00 UTC cron
│ scrapers/run.py │
└──────────┬──────────┘
│
① discovery + auto-promote ▼
┌──────────────────────────────────────────┐
│ discovery: vendor Models APIs (trusted) │→ unseen models auto-registered
│ + benchmark boards (noise→log) │
└───────────────┬──────────────────────────┘
│ register_extra → unified registry (catalog ∪ auto-discovered)
② scrape ▼
┌──────────────────────────────────────────┐
│ vendor scrapers benchmarks LLM fallback │
│ (price/specs) (Elo/scores) (Claude Haiku)│
└───────────────┬──────────────────────────┘
│ normalized exact matching (model_registry) — never mismatch
③ validation gate▼
┌──────────────────────────────────────────┐
│ validation: price >3× / ELO >100 jumps │ quarantined, no dirty writes
│ benchmark coverage drop alert │
└───────────────┬──────────────────────────┘
▼
┌────────────────┐
│ Supabase │ Postgres + RLS
│ models/prices │ benchmark_scores · daily_snapshots
│ discovery_* │ pending_changes · scrape_errors
└────────┬───────┘
▼
┌────────────────┐
│ Next.js ISR │ apps/web (incl. /health data-health page)
│ on Vercel CDN │ push→git auto-deploy; data refreshes via ISR
└────────────────┘

Robustness design (see the "robustness overhaul" section in IMPLEMENTATION_CHECKLIST.md):

Never miss a new model: vendor official Models APIs are the trusted signal — any unseen model is auto-registered (sparse metadata; price/Elo get filled in later by the scrape layer, never fabricated). Arena/AA leaderboard names are noise: never auto-registered, never nagging, only optionally viewable on /health.
Never mismatch: a single identity registry (core/model_registry.py) derived from catalog ∪ auto-discovered models, with normalized exact matching (controlled stripping of -thinking/date-style suffixes only — size/version is never stripped). CI enforces zero alias collisions. Each model is defined in exactly one place.
No dirty values: an anomaly gate quarantines wild jumps (the root cause of prices once flip-flopping); a value is only confirmed after appearing 2 consecutive times.
No silent failures: an unmatched name is logged as a discovery candidate + drift alert — everything surfaces on /health.

Directory layout

apps/web/ Next.js frontend (incl. /health data-health page)
scrapers/ Python scrapers
vendors/ one module per vendor (catalog-driven)
benchmarks/ LMSYS / Artificial Analysis / academic
discovery/ discovery layer: trusted vendor Models APIs etc.
core/
model_registry.py single identity registry (catalog ∪ auto-discovered, exact matching)
discovery.py discovery filtering + vendor inference (pure logic)
promotion.py auto-promotion: trusted source → model record (pure logic)
validation.py price/ELO anomaly gate (pure logic)
extractor / db / differ / registry
tests/ registry / discovery / validation / promotion unit tests (CI)
alert_candidates.py data-health alerts (quarantined values) → GitHub issue
supabase/migrations/ Postgres schema (0001–0007)
.github/workflows/ scrape-daily (cron) + test (pytest on PR)

Local development

Frontend

cd apps/web
npm install
cp ../../.env.example .env.local # fill in SUPABASE_URL + SUPABASE_ANON_KEY
npm run dev

Scrapers

python3 -m venv .venv
source .venv/bin/activate
pip install -r scrapers/requirements.txt
playwright install chromium
cp .env.example .env # fill in all KEYs
python -m scrapers.run --dry-run # full run without writing to the DB
python -m scrapers.run # real run
python -m scrapers.run --vendor openai # single vendor only
python -m scrapers.run --skip-discovery # skip discovery/auto-promotion
pytest scrapers/tests/ # unit tests (registry/discovery/validation/promotion)

Supabase

# Create a project at https://supabase.com/dashboard, grab the URL + Service Key.# Run all migrations in order in the SQL Editor, then load the seed:# supabase/migrations/0001_initial.sql initial 6 tables + views + RLS# supabase/migrations/0002_dedupe_benchmarks.sql# supabase/migrations/0003_add_model_license.sql# supabase/migrations/0004_discovery_candidates.sql discovery candidates table# supabase/migrations/0005_pending_changes.sql anomaly quarantine table# supabase/migrations/0006_data_health_views.sql sanitized error views# supabase/migrations/0007_auto_discovered.sql models.auto_discovered# supabase/seed.sql

Data schema

Table / view	Description
`vendors`	Vendor master data (OpenAI / Anthropic / ...)
`models`	Model master data, one row per model; `auto_discovered` marks API auto-promotion
`prices`	Price snapshots, append-only, full history retained
`benchmark_scores`	Benchmark snapshots, append-only
`daily_snapshots`	Daily summary + that day's changes as JSON (incl. discovery candidates)
`discovery_candidates`	Model names surfaced by discovery but not yet registered (proposals only, tiered by vendor)
`pending_changes`	Suspicious values quarantined by the anomaly gate (auto-confirmed after 2 consecutive identical readings)
`scrape_errors`	Scraper error log (incl. drift alerts)
`models_overview`(view)	Models + current price + Arena Elo aggregate, read by the frontend
`recent_scrape_issues`(view)	Sanitized projection of `scrape_errors` (no traceback/url), for /health

Data sources & attribution

This project aggregates publicly available information from:

Vendor pricing pages — listed in supabase/seed.sql per vendor (pricing_url)
LMSYS Chatbot Arena — Elo leaderboard
Artificial Analysis — independent third-party benchmarks
Official vendor announcements — for academic benchmark numbers (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Every prices and benchmark_scores row carries a source_url pointing back to the upstream.

Disclaimer

Prices are best-effort. Always confirm against the vendor's official pricing page before making a procurement decision. Scrapers can miss page changes and the fallback_prices baked into vendor files may be stale.
Benchmark numbers are reported figures, not independently re-run. Where vendors and third parties disagree, both are shown when available; trust your own evaluation.
No affiliation. This project is not affiliated with or endorsed by any vendor listed.

License

MIT — see LICENSE file. Contributions welcome — see CONTRIBUTING.md. Security issues — see SECURITY.md.

model.tracker

English | 简体中文

Automatically tracks model releases, pricing, and performance across the world's major AI vendors. Updated daily.

Coverage

14 vendors:

International: OpenAI · Anthropic · Google · Meta · Mistral · xAI · Cohere
China: DeepSeek · Qwen (Alibaba) · Zhipu GLM · Doubao (ByteDance) · Kimi (Moonshot) · Baichuan · Hunyuan (Tencent)

3 kinds of performance data sources:

LMSYS Chatbot Arena (human blind-test Elo)
Artificial Analysis (independent third-party speed + quality benchmarks)
Academic benchmarks (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Architecture

 ┌─────────────────────┐
│ GitHub Actions │ daily 02:00 UTC cron
│ scrapers/run.py │
└──────────┬──────────┘
│
① discovery + auto-promote ▼
┌──────────────────────────────────────────┐
│ discovery: vendor Models APIs (trusted) │→ unseen models auto-registered
│ + benchmark boards (noise→log) │
└───────────────┬──────────────────────────┘
│ register_extra → unified registry (catalog ∪ auto-discovered)
② scrape ▼
┌──────────────────────────────────────────┐
│ vendor scrapers benchmarks LLM fallback │
│ (price/specs) (Elo/scores) (Claude Haiku)│
└───────────────┬──────────────────────────┘
│ normalized exact matching (model_registry) — never mismatch
③ validation gate▼
┌──────────────────────────────────────────┐
│ validation: price >3× / ELO >100 jumps │ quarantined, no dirty writes
│ benchmark coverage drop alert │
└───────────────┬──────────────────────────┘
▼
┌────────────────┐
│ Supabase │ Postgres + RLS
│ models/prices │ benchmark_scores · daily_snapshots
│ discovery_* │ pending_changes · scrape_errors
└────────┬───────┘
▼
┌────────────────┐
│ Next.js ISR │ apps/web (incl. /health data-health page)
│ on Vercel CDN │ push→git auto-deploy; data refreshes via ISR
└────────────────┘

Robustness design (see the "robustness overhaul" section in IMPLEMENTATION_CHECKLIST.md):

Never miss a new model: vendor official Models APIs are the trusted signal — any unseen model is auto-registered (sparse metadata; price/Elo get filled in later by the scrape layer, never fabricated). Arena/AA leaderboard names are noise: never auto-registered, never nagging, only optionally viewable on /health.
Never mismatch: a single identity registry (core/model_registry.py) derived from catalog ∪ auto-discovered models, with normalized exact matching (controlled stripping of -thinking/date-style suffixes only — size/version is never stripped). CI enforces zero alias collisions. Each model is defined in exactly one place.
No dirty values: an anomaly gate quarantines wild jumps (the root cause of prices once flip-flopping); a value is only confirmed after appearing 2 consecutive times.
No silent failures: an unmatched name is logged as a discovery candidate + drift alert — everything surfaces on /health.

Directory layout

apps/web/ Next.js frontend (incl. /health data-health page)
scrapers/ Python scrapers
vendors/ one module per vendor (catalog-driven)
benchmarks/ LMSYS / Artificial Analysis / academic
discovery/ discovery layer: trusted vendor Models APIs etc.
core/
model_registry.py single identity registry (catalog ∪ auto-discovered, exact matching)
discovery.py discovery filtering + vendor inference (pure logic)
promotion.py auto-promotion: trusted source → model record (pure logic)
validation.py price/ELO anomaly gate (pure logic)
extractor / db / differ / registry
tests/ registry / discovery / validation / promotion unit tests (CI)
alert_candidates.py data-health alerts (quarantined values) → GitHub issue
supabase/migrations/ Postgres schema (0001–0007)
.github/workflows/ scrape-daily (cron) + test (pytest on PR)

Local development

Frontend

cd apps/web
npm install
cp ../../.env.example .env.local # fill in SUPABASE_URL + SUPABASE_ANON_KEY
npm run dev

Scrapers

python3 -m venv .venv
source .venv/bin/activate
pip install -r scrapers/requirements.txt
playwright install chromium
cp .env.example .env # fill in all KEYs
python -m scrapers.run --dry-run # full run without writing to the DB
python -m scrapers.run # real run
python -m scrapers.run --vendor openai # single vendor only
python -m scrapers.run --skip-discovery # skip discovery/auto-promotion
pytest scrapers/tests/ # unit tests (registry/discovery/validation/promotion)

Supabase

# Create a project at https://supabase.com/dashboard, grab the URL + Service Key.# Run all migrations in order in the SQL Editor, then load the seed:# supabase/migrations/0001_initial.sql initial 6 tables + views + RLS# supabase/migrations/0002_dedupe_benchmarks.sql# supabase/migrations/0003_add_model_license.sql# supabase/migrations/0004_discovery_candidates.sql discovery candidates table# supabase/migrations/0005_pending_changes.sql anomaly quarantine table# supabase/migrations/0006_data_health_views.sql sanitized error views# supabase/migrations/0007_auto_discovered.sql models.auto_discovered# supabase/seed.sql

Data schema

Table / view	Description
`vendors`	Vendor master data (OpenAI / Anthropic / ...)
`models`	Model master data, one row per model; `auto_discovered` marks API auto-promotion
`prices`	Price snapshots, append-only, full history retained
`benchmark_scores`	Benchmark snapshots, append-only
`daily_snapshots`	Daily summary + that day's changes as JSON (incl. discovery candidates)
`discovery_candidates`	Model names surfaced by discovery but not yet registered (proposals only, tiered by vendor)
`pending_changes`	Suspicious values quarantined by the anomaly gate (auto-confirmed after 2 consecutive identical readings)
`scrape_errors`	Scraper error log (incl. drift alerts)
`models_overview`(view)	Models + current price + Arena Elo aggregate, read by the frontend
`recent_scrape_issues`(view)	Sanitized projection of `scrape_errors` (no traceback/url), for /health

Data sources & attribution

This project aggregates publicly available information from:

Vendor pricing pages — listed in supabase/seed.sql per vendor (pricing_url)
LMSYS Chatbot Arena — Elo leaderboard
Artificial Analysis — independent third-party benchmarks
Official vendor announcements — for academic benchmark numbers (MMLU / GPQA / HumanEval / SWE-bench / MATH)

Every prices and benchmark_scores row carries a source_url pointing back to the upstream.

Disclaimer

Prices are best-effort. Always confirm against the vendor's official pricing page before making a procurement decision. Scrapers can miss page changes and the fallback_prices baked into vendor files may be stale.
Benchmark numbers are reported figures, not independently re-run. Where vendors and third parties disagree, both are shown when available; trust your own evaluation.
No affiliation. This project is not affiliated with or endorsed by any vendor listed.

License

MIT — see LICENSE file. Contributions welcome — see CONTRIBUTING.md. Security issues — see SECURITY.md.

Name		Name	Last commit message	Last commit date
.github		.github
apps/web		apps/web
docs/_archive		docs/_archive
scrapers		scrapers
scripts		scripts
supabase		supabase
.env.example		.env.example
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
CONTRIBUTING.md		CONTRIBUTING.md
IMPLEMENTATION_CHECKLIST.md		IMPLEMENTATION_CHECKLIST.md
LICENSE		LICENSE
README.md		README.md
README.zh-CN.md		README.zh-CN.md
SECURITY.md		SECURITY.md
vercel.json		vercel.json

Latest commit

History

Folders and files

Repository files navigation

model.tracker

Coverage

Architecture

Directory layout

Local development

Frontend

Scrapers

Supabase

Data schema

Data sources & attribution

Disclaimer

License

About

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

model.tracker

Coverage

Architecture

Directory layout

Local development

Frontend

Scrapers

Supabase

Data schema

Data sources & attribution

Disclaimer

License

About

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

model.tracker

Coverage

Architecture

Directory layout

Local development

Frontend

Scrapers

Supabase

Data schema

Data sources & attribution

Disclaimer

License

About

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors