fix(usage): canonical LiteLLM keys must win rate-table collisions by derektrimm · Pull Request #8548 · pingdotgg/t3code

derektrimm · 2026-08-28T14:43:05Z

What Changed

parseRateTable now gives a canonical (slash-free) LiteLLM key precedence over prefix-stripped keys that normalize to the same name. A prefix-stripped entry only fills a slot no canonical entry claims. Added usagePricing.test.ts (the file did not exist) covering both collision orders, differing base rates, prefixed-only models, the missing-cache-fields fallback, dotted Bedrock-style keys, and end-to-end cost + savings arithmetic in the exact #8534 shape.

Why

The rate table is built last-entry-wins over normalized names. In today's LiteLLM document the canonical claude-fable-5 / claude-opus-5 / claude-opus-4-8 / claude-sonnet-5 entries are shadowed by deepinfra/anthropic/* reseller entries that carry no cache_read_input_token_cost, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5 / gpt-5-mini / gpt-5-nano lose the same way to replicate/openai/*.

The reporter's screenshot in #8534 confirms this to the cent: fable-5 averages $10.14/M (full input rate) on a bucket that is overwhelmingly cache reads, the six model costs sum exactly to the $62,540.18 headline, and the $9.45 cache savings is producible only by the two models whose canonical keys happen to serialize last. Full arithmetic in the issue comment.

Deliberately unchanged: the ?? input fallback (correct for models that genuinely publish no cache pricing) and stripped-vs-stripped ordering for names with no canonical entry.

Note: beyond the cache column, this also corrects base rates for models whose canonical entries were shadowed by gateway variants with different prices (e.g. gemini-2.5-pro was priced at 2x from a vercel_ai_gateway/* entry). Every retarget moves toward the canonical price.

Fixes#8534

Checklist

This PR is small and focused
I explained what changed and why
I included before/after screenshots for any UI changes (server-side pricing only; no UI change)
I included a video for animation/interaction changes (n/a)

Note

Medium Risk
Changes reported usage cost and cache-savings math for models affected by LiteLLM key collisions; no auth or payment flow changes, but billing visibility shifts toward canonical rates.

Overview
Fixes #8534, where reseller/gateway LiteLLM keys (e.g. deepinfra/anthropic/claude-fable-5) that normalize to the same model name were overwriting canonical slash-free entries, stripping cache discount rates and sometimes doubling base input/output prices.

parseRateTable now treats keys without / as canonical: they always claim their normalized slot, and prefixed keys are skipped once a canonical name exists; prefixed-only models still populate the table when no canonical key is present.

Adds usagePricing.test.ts with collision ordering, gateway vs canonical base rates, cache fallbacks, dot-prefixed Bedrock-style keys, and end-to-end priceUsage / cacheSavingsUsd checks for the fable-5 scenario.

^{Reviewed by Cursor Bugbot for commit 9b49df8. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Fix `parseRateTable` to let canonical LiteLLM keys win rate-table collisions

parseRateTable now tracks canonical (slash-free) model names in a Set and skips prefixed entries (e.g. deepinfra/anthropic/claude-fable-5) when a canonical entry for the same normalized name already exists.
Cache read/creation costs default to the input cost when the rate entry omits them.
Adds tests in usagePricing.test.ts covering canonical-vs-prefixed collisions, cache-rate defaults, dot-prefixed names, and savings calculations.
Behavioral Change: prefixed entries that previously overwrote canonical rates for the same normalized name are now ignored; consumers relying on prefixed-override behavior will see the canonical rates instead.

^{Macroscope summarized 9b49df8.}

coderabbitai · 2026-08-28T14:43:12Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d48f4f90-9193-493a-b2c8-4b709a4b3450

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

macroscopeapp · 2026-08-28T14:51:48Z

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This production change alters model-derived usage-dollar and cache-savings calculations across the usage aggregation path. Although the implementation is small and regression-tested, pricing and metering behavior warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

parseRateTable strips provider prefixes for lookup, so several LiteLLM keys collapse onto one normalized name and the last entry in the document silently wins. For claude-fable-5, claude-opus-5, claude-opus-4-8 and claude-sonnet-5 that winner is a cache-less deepinfra/anthropic/* reseller entry, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5, gpt-5-mini and gpt-5-nano lose the same way to replicate/openai/*, and some base rates were shadowed too (gemini-2.5-pro at 2x from a vercel_ai_gateway entry). A slash-free key is the canonical entry for the name transcripts actually record, so it now always wins its slot; a prefix-stripped key only fills a slot no canonical entry claims. The ?? input fallback is unchanged: it is correct for models that genuinely publish no cache pricing. Fixespingdotgg#8534

derektrimm · 2026-08-31T19:14:57Z

Closing - #8806 landed a more complete version of this fix (preserving qualified keys rather than collapsing them) and users on #8534 confirm 0.0.37 resolves it. Glad it's fixed.

derektrimm · 2026-08-28T14:43:05Z

What Changed

parseRateTable now gives a canonical (slash-free) LiteLLM key precedence over prefix-stripped keys that normalize to the same name. A prefix-stripped entry only fills a slot no canonical entry claims. Added usagePricing.test.ts (the file did not exist) covering both collision orders, differing base rates, prefixed-only models, the missing-cache-fields fallback, dotted Bedrock-style keys, and end-to-end cost + savings arithmetic in the exact #8534 shape.

Why

The rate table is built last-entry-wins over normalized names. In today's LiteLLM document the canonical claude-fable-5 / claude-opus-5 / claude-opus-4-8 / claude-sonnet-5 entries are shadowed by deepinfra/anthropic/* reseller entries that carry no cache_read_input_token_cost, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5 / gpt-5-mini / gpt-5-nano lose the same way to replicate/openai/*.

The reporter's screenshot in #8534 confirms this to the cent: fable-5 averages $10.14/M (full input rate) on a bucket that is overwhelmingly cache reads, the six model costs sum exactly to the $62,540.18 headline, and the $9.45 cache savings is producible only by the two models whose canonical keys happen to serialize last. Full arithmetic in the issue comment.

Deliberately unchanged: the ?? input fallback (correct for models that genuinely publish no cache pricing) and stripped-vs-stripped ordering for names with no canonical entry.

Note: beyond the cache column, this also corrects base rates for models whose canonical entries were shadowed by gateway variants with different prices (e.g. gemini-2.5-pro was priced at 2x from a vercel_ai_gateway/* entry). Every retarget moves toward the canonical price.

Fixes#8534

Checklist

This PR is small and focused
I explained what changed and why
I included before/after screenshots for any UI changes (server-side pricing only; no UI change)
I included a video for animation/interaction changes (n/a)

Note

Medium Risk
Changes reported usage cost and cache-savings math for models affected by LiteLLM key collisions; no auth or payment flow changes, but billing visibility shifts toward canonical rates.

Overview
Fixes #8534, where reseller/gateway LiteLLM keys (e.g. deepinfra/anthropic/claude-fable-5) that normalize to the same model name were overwriting canonical slash-free entries, stripping cache discount rates and sometimes doubling base input/output prices.

parseRateTable now treats keys without / as canonical: they always claim their normalized slot, and prefixed keys are skipped once a canonical name exists; prefixed-only models still populate the table when no canonical key is present.

Adds usagePricing.test.ts with collision ordering, gateway vs canonical base rates, cache fallbacks, dot-prefixed Bedrock-style keys, and end-to-end priceUsage / cacheSavingsUsd checks for the fable-5 scenario.

^{Reviewed by Cursor Bugbot for commit 9b49df8. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Fix `parseRateTable` to let canonical LiteLLM keys win rate-table collisions

parseRateTable now tracks canonical (slash-free) model names in a Set and skips prefixed entries (e.g. deepinfra/anthropic/claude-fable-5) when a canonical entry for the same normalized name already exists.
Cache read/creation costs default to the input cost when the rate entry omits them.
Adds tests in usagePricing.test.ts covering canonical-vs-prefixed collisions, cache-rate defaults, dot-prefixed names, and savings calculations.
Behavioral Change: prefixed entries that previously overwrote canonical rates for the same normalized name are now ignored; consumers relying on prefixed-override behavior will see the canonical rates instead.

^{Macroscope summarized 9b49df8.}

coderabbitai · 2026-08-28T14:43:12Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d48f4f90-9193-493a-b2c8-4b709a4b3450

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

macroscopeapp · 2026-08-28T14:51:48Z

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This production change alters model-derived usage-dollar and cache-savings calculations across the usage aggregation path. Although the implementation is small and regression-tested, pricing and metering behavior warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

parseRateTable strips provider prefixes for lookup, so several LiteLLM keys collapse onto one normalized name and the last entry in the document silently wins. For claude-fable-5, claude-opus-5, claude-opus-4-8 and claude-sonnet-5 that winner is a cache-less deepinfra/anthropic/* reseller entry, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5, gpt-5-mini and gpt-5-nano lose the same way to replicate/openai/*, and some base rates were shadowed too (gemini-2.5-pro at 2x from a vercel_ai_gateway entry). A slash-free key is the canonical entry for the name transcripts actually record, so it now always wins its slot; a prefix-stripped key only fills a slot no canonical entry claims. The ?? input fallback is unchanged: it is correct for models that genuinely publish no cache pricing. Fixespingdotgg#8534

derektrimm · 2026-08-31T19:14:57Z

Closing - #8806 landed a more complete version of this fix (preserving qualified keys rather than collapsing them) and users on #8534 confirm 0.0.37 resolves it. Glad it's fixed.

derektrimm · 2026-08-28T14:43:05Z

What Changed

parseRateTable now gives a canonical (slash-free) LiteLLM key precedence over prefix-stripped keys that normalize to the same name. A prefix-stripped entry only fills a slot no canonical entry claims. Added usagePricing.test.ts (the file did not exist) covering both collision orders, differing base rates, prefixed-only models, the missing-cache-fields fallback, dotted Bedrock-style keys, and end-to-end cost + savings arithmetic in the exact #8534 shape.

Why

The rate table is built last-entry-wins over normalized names. In today's LiteLLM document the canonical claude-fable-5 / claude-opus-5 / claude-opus-4-8 / claude-sonnet-5 entries are shadowed by deepinfra/anthropic/* reseller entries that carry no cache_read_input_token_cost, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5 / gpt-5-mini / gpt-5-nano lose the same way to replicate/openai/*.

The reporter's screenshot in #8534 confirms this to the cent: fable-5 averages $10.14/M (full input rate) on a bucket that is overwhelmingly cache reads, the six model costs sum exactly to the $62,540.18 headline, and the $9.45 cache savings is producible only by the two models whose canonical keys happen to serialize last. Full arithmetic in the issue comment.

Deliberately unchanged: the ?? input fallback (correct for models that genuinely publish no cache pricing) and stripped-vs-stripped ordering for names with no canonical entry.

Note: beyond the cache column, this also corrects base rates for models whose canonical entries were shadowed by gateway variants with different prices (e.g. gemini-2.5-pro was priced at 2x from a vercel_ai_gateway/* entry). Every retarget moves toward the canonical price.

Fixes#8534

Checklist

This PR is small and focused
I explained what changed and why
I included before/after screenshots for any UI changes (server-side pricing only; no UI change)
I included a video for animation/interaction changes (n/a)

Note

Medium Risk
Changes reported usage cost and cache-savings math for models affected by LiteLLM key collisions; no auth or payment flow changes, but billing visibility shifts toward canonical rates.

Overview
Fixes #8534, where reseller/gateway LiteLLM keys (e.g. deepinfra/anthropic/claude-fable-5) that normalize to the same model name were overwriting canonical slash-free entries, stripping cache discount rates and sometimes doubling base input/output prices.

parseRateTable now treats keys without / as canonical: they always claim their normalized slot, and prefixed keys are skipped once a canonical name exists; prefixed-only models still populate the table when no canonical key is present.

Adds usagePricing.test.ts with collision ordering, gateway vs canonical base rates, cache fallbacks, dot-prefixed Bedrock-style keys, and end-to-end priceUsage / cacheSavingsUsd checks for the fable-5 scenario.

^{Reviewed by Cursor Bugbot for commit 9b49df8. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Fix `parseRateTable` to let canonical LiteLLM keys win rate-table collisions

parseRateTable now tracks canonical (slash-free) model names in a Set and skips prefixed entries (e.g. deepinfra/anthropic/claude-fable-5) when a canonical entry for the same normalized name already exists.
Cache read/creation costs default to the input cost when the rate entry omits them.
Adds tests in usagePricing.test.ts covering canonical-vs-prefixed collisions, cache-rate defaults, dot-prefixed names, and savings calculations.
Behavioral Change: prefixed entries that previously overwrote canonical rates for the same normalized name are now ignored; consumers relying on prefixed-override behavior will see the canonical rates instead.

^{Macroscope summarized 9b49df8.}

coderabbitai · 2026-08-28T14:43:12Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d48f4f90-9193-493a-b2c8-4b709a4b3450

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

macroscopeapp · 2026-08-28T14:51:48Z

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This production change alters model-derived usage-dollar and cache-savings calculations across the usage aggregation path. Although the implementation is small and regression-tested, pricing and metering behavior warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

parseRateTable strips provider prefixes for lookup, so several LiteLLM keys collapse onto one normalized name and the last entry in the document silently wins. For claude-fable-5, claude-opus-5, claude-opus-4-8 and claude-sonnet-5 that winner is a cache-less deepinfra/anthropic/* reseller entry, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5, gpt-5-mini and gpt-5-nano lose the same way to replicate/openai/*, and some base rates were shadowed too (gemini-2.5-pro at 2x from a vercel_ai_gateway entry). A slash-free key is the canonical entry for the name transcripts actually record, so it now always wins its slot; a prefix-stripped key only fills a slot no canonical entry claims. The ?? input fallback is unchanged: it is correct for models that genuinely publish no cache pricing. Fixespingdotgg#8534

derektrimm · 2026-08-31T19:14:57Z

Closing - #8806 landed a more complete version of this fix (preserving qualified keys rather than collapsing them) and users on #8534 confirm 0.0.37 resolves it. Glad it's fixed.

derektrimm · 2026-08-28T14:43:05Z

What Changed

parseRateTable now gives a canonical (slash-free) LiteLLM key precedence over prefix-stripped keys that normalize to the same name. A prefix-stripped entry only fills a slot no canonical entry claims. Added usagePricing.test.ts (the file did not exist) covering both collision orders, differing base rates, prefixed-only models, the missing-cache-fields fallback, dotted Bedrock-style keys, and end-to-end cost + savings arithmetic in the exact #8534 shape.

Why

The rate table is built last-entry-wins over normalized names. In today's LiteLLM document the canonical claude-fable-5 / claude-opus-5 / claude-opus-4-8 / claude-sonnet-5 entries are shadowed by deepinfra/anthropic/* reseller entries that carry no cache_read_input_token_cost, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5 / gpt-5-mini / gpt-5-nano lose the same way to replicate/openai/*.

The reporter's screenshot in #8534 confirms this to the cent: fable-5 averages $10.14/M (full input rate) on a bucket that is overwhelmingly cache reads, the six model costs sum exactly to the $62,540.18 headline, and the $9.45 cache savings is producible only by the two models whose canonical keys happen to serialize last. Full arithmetic in the issue comment.

Deliberately unchanged: the ?? input fallback (correct for models that genuinely publish no cache pricing) and stripped-vs-stripped ordering for names with no canonical entry.

Note: beyond the cache column, this also corrects base rates for models whose canonical entries were shadowed by gateway variants with different prices (e.g. gemini-2.5-pro was priced at 2x from a vercel_ai_gateway/* entry). Every retarget moves toward the canonical price.

Fixes#8534

Checklist

This PR is small and focused
I explained what changed and why
I included before/after screenshots for any UI changes (server-side pricing only; no UI change)
I included a video for animation/interaction changes (n/a)

Note

Medium Risk
Changes reported usage cost and cache-savings math for models affected by LiteLLM key collisions; no auth or payment flow changes, but billing visibility shifts toward canonical rates.

Overview
Fixes #8534, where reseller/gateway LiteLLM keys (e.g. deepinfra/anthropic/claude-fable-5) that normalize to the same model name were overwriting canonical slash-free entries, stripping cache discount rates and sometimes doubling base input/output prices.

parseRateTable now treats keys without / as canonical: they always claim their normalized slot, and prefixed keys are skipped once a canonical name exists; prefixed-only models still populate the table when no canonical key is present.

Adds usagePricing.test.ts with collision ordering, gateway vs canonical base rates, cache fallbacks, dot-prefixed Bedrock-style keys, and end-to-end priceUsage / cacheSavingsUsd checks for the fable-5 scenario.

^{Reviewed by Cursor Bugbot for commit 9b49df8. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Fix `parseRateTable` to let canonical LiteLLM keys win rate-table collisions

parseRateTable now tracks canonical (slash-free) model names in a Set and skips prefixed entries (e.g. deepinfra/anthropic/claude-fable-5) when a canonical entry for the same normalized name already exists.
Cache read/creation costs default to the input cost when the rate entry omits them.
Adds tests in usagePricing.test.ts covering canonical-vs-prefixed collisions, cache-rate defaults, dot-prefixed names, and savings calculations.
Behavioral Change: prefixed entries that previously overwrote canonical rates for the same normalized name are now ignored; consumers relying on prefixed-override behavior will see the canonical rates instead.

^{Macroscope summarized 9b49df8.}

coderabbitai · 2026-08-28T14:43:12Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d48f4f90-9193-493a-b2c8-4b709a4b3450

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

macroscopeapp · 2026-08-28T14:51:48Z

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This production change alters model-derived usage-dollar and cache-savings calculations across the usage aggregation path. Although the implementation is small and regression-tested, pricing and metering behavior warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

parseRateTable strips provider prefixes for lookup, so several LiteLLM keys collapse onto one normalized name and the last entry in the document silently wins. For claude-fable-5, claude-opus-5, claude-opus-4-8 and claude-sonnet-5 that winner is a cache-less deepinfra/anthropic/* reseller entry, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5, gpt-5-mini and gpt-5-nano lose the same way to replicate/openai/*, and some base rates were shadowed too (gemini-2.5-pro at 2x from a vercel_ai_gateway entry). A slash-free key is the canonical entry for the name transcripts actually record, so it now always wins its slot; a prefix-stripped key only fills a slot no canonical entry claims. The ?? input fallback is unchanged: it is correct for models that genuinely publish no cache pricing. Fixespingdotgg#8534

derektrimm · 2026-08-31T19:14:57Z

Closing - #8806 landed a more complete version of this fix (preserving qualified keys rather than collapsing them) and users on #8534 confirm 0.0.37 resolves it. Glad it's fixed.

derektrimm · 2026-08-28T14:43:05Z

What Changed

parseRateTable now gives a canonical (slash-free) LiteLLM key precedence over prefix-stripped keys that normalize to the same name. A prefix-stripped entry only fills a slot no canonical entry claims. Added usagePricing.test.ts (the file did not exist) covering both collision orders, differing base rates, prefixed-only models, the missing-cache-fields fallback, dotted Bedrock-style keys, and end-to-end cost + savings arithmetic in the exact #8534 shape.

Why

The rate table is built last-entry-wins over normalized names. In today's LiteLLM document the canonical claude-fable-5 / claude-opus-5 / claude-opus-4-8 / claude-sonnet-5 entries are shadowed by deepinfra/anthropic/* reseller entries that carry no cache_read_input_token_cost, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5 / gpt-5-mini / gpt-5-nano lose the same way to replicate/openai/*.

The reporter's screenshot in #8534 confirms this to the cent: fable-5 averages $10.14/M (full input rate) on a bucket that is overwhelmingly cache reads, the six model costs sum exactly to the $62,540.18 headline, and the $9.45 cache savings is producible only by the two models whose canonical keys happen to serialize last. Full arithmetic in the issue comment.

Deliberately unchanged: the ?? input fallback (correct for models that genuinely publish no cache pricing) and stripped-vs-stripped ordering for names with no canonical entry.

Note: beyond the cache column, this also corrects base rates for models whose canonical entries were shadowed by gateway variants with different prices (e.g. gemini-2.5-pro was priced at 2x from a vercel_ai_gateway/* entry). Every retarget moves toward the canonical price.

Fixes#8534

Checklist

This PR is small and focused
I explained what changed and why
I included before/after screenshots for any UI changes (server-side pricing only; no UI change)
I included a video for animation/interaction changes (n/a)

Note

Medium Risk
Changes reported usage cost and cache-savings math for models affected by LiteLLM key collisions; no auth or payment flow changes, but billing visibility shifts toward canonical rates.

Overview
Fixes #8534, where reseller/gateway LiteLLM keys (e.g. deepinfra/anthropic/claude-fable-5) that normalize to the same model name were overwriting canonical slash-free entries, stripping cache discount rates and sometimes doubling base input/output prices.

parseRateTable now treats keys without / as canonical: they always claim their normalized slot, and prefixed keys are skipped once a canonical name exists; prefixed-only models still populate the table when no canonical key is present.

Adds usagePricing.test.ts with collision ordering, gateway vs canonical base rates, cache fallbacks, dot-prefixed Bedrock-style keys, and end-to-end priceUsage / cacheSavingsUsd checks for the fable-5 scenario.

^{Reviewed by Cursor Bugbot for commit 9b49df8. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Fix `parseRateTable` to let canonical LiteLLM keys win rate-table collisions

parseRateTable now tracks canonical (slash-free) model names in a Set and skips prefixed entries (e.g. deepinfra/anthropic/claude-fable-5) when a canonical entry for the same normalized name already exists.
Cache read/creation costs default to the input cost when the rate entry omits them.
Adds tests in usagePricing.test.ts covering canonical-vs-prefixed collisions, cache-rate defaults, dot-prefixed names, and savings calculations.
Behavioral Change: prefixed entries that previously overwrote canonical rates for the same normalized name are now ignored; consumers relying on prefixed-override behavior will see the canonical rates instead.

^{Macroscope summarized 9b49df8.}

coderabbitai · 2026-08-28T14:43:12Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d48f4f90-9193-493a-b2c8-4b709a4b3450

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

macroscopeapp · 2026-08-28T14:51:48Z

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This production change alters model-derived usage-dollar and cache-savings calculations across the usage aggregation path. Although the implementation is small and regression-tested, pricing and metering behavior warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

parseRateTable strips provider prefixes for lookup, so several LiteLLM keys collapse onto one normalized name and the last entry in the document silently wins. For claude-fable-5, claude-opus-5, claude-opus-4-8 and claude-sonnet-5 that winner is a cache-less deepinfra/anthropic/* reseller entry, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5, gpt-5-mini and gpt-5-nano lose the same way to replicate/openai/*, and some base rates were shadowed too (gemini-2.5-pro at 2x from a vercel_ai_gateway entry). A slash-free key is the canonical entry for the name transcripts actually record, so it now always wins its slot; a prefix-stripped key only fills a slot no canonical entry claims. The ?? input fallback is unchanged: it is correct for models that genuinely publish no cache pricing. Fixespingdotgg#8534

derektrimm · 2026-08-31T19:14:57Z

Closing - #8806 landed a more complete version of this fix (preserving qualified keys rather than collapsing them) and users on #8534 confirm 0.0.37 resolves it. Glad it's fixed.

derektrimm · 2026-08-28T14:43:05Z

What Changed

parseRateTable now gives a canonical (slash-free) LiteLLM key precedence over prefix-stripped keys that normalize to the same name. A prefix-stripped entry only fills a slot no canonical entry claims. Added usagePricing.test.ts (the file did not exist) covering both collision orders, differing base rates, prefixed-only models, the missing-cache-fields fallback, dotted Bedrock-style keys, and end-to-end cost + savings arithmetic in the exact #8534 shape.

Why

The rate table is built last-entry-wins over normalized names. In today's LiteLLM document the canonical claude-fable-5 / claude-opus-5 / claude-opus-4-8 / claude-sonnet-5 entries are shadowed by deepinfra/anthropic/* reseller entries that carry no cache_read_input_token_cost, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5 / gpt-5-mini / gpt-5-nano lose the same way to replicate/openai/*.

The reporter's screenshot in #8534 confirms this to the cent: fable-5 averages $10.14/M (full input rate) on a bucket that is overwhelmingly cache reads, the six model costs sum exactly to the $62,540.18 headline, and the $9.45 cache savings is producible only by the two models whose canonical keys happen to serialize last. Full arithmetic in the issue comment.

Deliberately unchanged: the ?? input fallback (correct for models that genuinely publish no cache pricing) and stripped-vs-stripped ordering for names with no canonical entry.

Note: beyond the cache column, this also corrects base rates for models whose canonical entries were shadowed by gateway variants with different prices (e.g. gemini-2.5-pro was priced at 2x from a vercel_ai_gateway/* entry). Every retarget moves toward the canonical price.

Fixes#8534

Checklist

This PR is small and focused
I explained what changed and why
I included before/after screenshots for any UI changes (server-side pricing only; no UI change)
I included a video for animation/interaction changes (n/a)

Note

Medium Risk
Changes reported usage cost and cache-savings math for models affected by LiteLLM key collisions; no auth or payment flow changes, but billing visibility shifts toward canonical rates.

Overview
Fixes #8534, where reseller/gateway LiteLLM keys (e.g. deepinfra/anthropic/claude-fable-5) that normalize to the same model name were overwriting canonical slash-free entries, stripping cache discount rates and sometimes doubling base input/output prices.

parseRateTable now treats keys without / as canonical: they always claim their normalized slot, and prefixed keys are skipped once a canonical name exists; prefixed-only models still populate the table when no canonical key is present.

Adds usagePricing.test.ts with collision ordering, gateway vs canonical base rates, cache fallbacks, dot-prefixed Bedrock-style keys, and end-to-end priceUsage / cacheSavingsUsd checks for the fable-5 scenario.

^{Reviewed by Cursor Bugbot for commit 9b49df8. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Fix `parseRateTable` to let canonical LiteLLM keys win rate-table collisions

parseRateTable now tracks canonical (slash-free) model names in a Set and skips prefixed entries (e.g. deepinfra/anthropic/claude-fable-5) when a canonical entry for the same normalized name already exists.
Cache read/creation costs default to the input cost when the rate entry omits them.
Adds tests in usagePricing.test.ts covering canonical-vs-prefixed collisions, cache-rate defaults, dot-prefixed names, and savings calculations.
Behavioral Change: prefixed entries that previously overwrote canonical rates for the same normalized name are now ignored; consumers relying on prefixed-override behavior will see the canonical rates instead.

^{Macroscope summarized 9b49df8.}

coderabbitai · 2026-08-28T14:43:12Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d48f4f90-9193-493a-b2c8-4b709a4b3450

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

macroscopeapp · 2026-08-28T14:51:48Z

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This production change alters model-derived usage-dollar and cache-savings calculations across the usage aggregation path. Although the implementation is small and regression-tested, pricing and metering behavior warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

parseRateTable strips provider prefixes for lookup, so several LiteLLM keys collapse onto one normalized name and the last entry in the document silently wins. For claude-fable-5, claude-opus-5, claude-opus-4-8 and claude-sonnet-5 that winner is a cache-less deepinfra/anthropic/* reseller entry, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5, gpt-5-mini and gpt-5-nano lose the same way to replicate/openai/*, and some base rates were shadowed too (gemini-2.5-pro at 2x from a vercel_ai_gateway entry). A slash-free key is the canonical entry for the name transcripts actually record, so it now always wins its slot; a prefix-stripped key only fills a slot no canonical entry claims. The ?? input fallback is unchanged: it is correct for models that genuinely publish no cache pricing. Fixespingdotgg#8534

derektrimm · 2026-08-31T19:14:57Z

Closing - #8806 landed a more complete version of this fix (preserving qualified keys rather than collapsing them) and users on #8534 confirm 0.0.37 resolves it. Glad it's fixed.

derektrimm · 2026-08-28T14:43:05Z

What Changed

parseRateTable now gives a canonical (slash-free) LiteLLM key precedence over prefix-stripped keys that normalize to the same name. A prefix-stripped entry only fills a slot no canonical entry claims. Added usagePricing.test.ts (the file did not exist) covering both collision orders, differing base rates, prefixed-only models, the missing-cache-fields fallback, dotted Bedrock-style keys, and end-to-end cost + savings arithmetic in the exact #8534 shape.

Why

The rate table is built last-entry-wins over normalized names. In today's LiteLLM document the canonical claude-fable-5 / claude-opus-5 / claude-opus-4-8 / claude-sonnet-5 entries are shadowed by deepinfra/anthropic/* reseller entries that carry no cache_read_input_token_cost, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5 / gpt-5-mini / gpt-5-nano lose the same way to replicate/openai/*.

The reporter's screenshot in #8534 confirms this to the cent: fable-5 averages $10.14/M (full input rate) on a bucket that is overwhelmingly cache reads, the six model costs sum exactly to the $62,540.18 headline, and the $9.45 cache savings is producible only by the two models whose canonical keys happen to serialize last. Full arithmetic in the issue comment.

Deliberately unchanged: the ?? input fallback (correct for models that genuinely publish no cache pricing) and stripped-vs-stripped ordering for names with no canonical entry.

Note: beyond the cache column, this also corrects base rates for models whose canonical entries were shadowed by gateway variants with different prices (e.g. gemini-2.5-pro was priced at 2x from a vercel_ai_gateway/* entry). Every retarget moves toward the canonical price.

Fixes#8534

Checklist

This PR is small and focused
I explained what changed and why
I included before/after screenshots for any UI changes (server-side pricing only; no UI change)
I included a video for animation/interaction changes (n/a)

Note

Medium Risk
Changes reported usage cost and cache-savings math for models affected by LiteLLM key collisions; no auth or payment flow changes, but billing visibility shifts toward canonical rates.

Overview
Fixes #8534, where reseller/gateway LiteLLM keys (e.g. deepinfra/anthropic/claude-fable-5) that normalize to the same model name were overwriting canonical slash-free entries, stripping cache discount rates and sometimes doubling base input/output prices.

parseRateTable now treats keys without / as canonical: they always claim their normalized slot, and prefixed keys are skipped once a canonical name exists; prefixed-only models still populate the table when no canonical key is present.

Adds usagePricing.test.ts with collision ordering, gateway vs canonical base rates, cache fallbacks, dot-prefixed Bedrock-style keys, and end-to-end priceUsage / cacheSavingsUsd checks for the fable-5 scenario.

^{Reviewed by Cursor Bugbot for commit 9b49df8. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Fix `parseRateTable` to let canonical LiteLLM keys win rate-table collisions

parseRateTable now tracks canonical (slash-free) model names in a Set and skips prefixed entries (e.g. deepinfra/anthropic/claude-fable-5) when a canonical entry for the same normalized name already exists.
Cache read/creation costs default to the input cost when the rate entry omits them.
Adds tests in usagePricing.test.ts covering canonical-vs-prefixed collisions, cache-rate defaults, dot-prefixed names, and savings calculations.
Behavioral Change: prefixed entries that previously overwrote canonical rates for the same normalized name are now ignored; consumers relying on prefixed-override behavior will see the canonical rates instead.

^{Macroscope summarized 9b49df8.}

coderabbitai · 2026-08-28T14:43:12Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d48f4f90-9193-493a-b2c8-4b709a4b3450

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

macroscopeapp · 2026-08-28T14:51:48Z

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This production change alters model-derived usage-dollar and cache-savings calculations across the usage aggregation path. Although the implementation is small and regression-tested, pricing and metering behavior warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

parseRateTable strips provider prefixes for lookup, so several LiteLLM keys collapse onto one normalized name and the last entry in the document silently wins. For claude-fable-5, claude-opus-5, claude-opus-4-8 and claude-sonnet-5 that winner is a cache-less deepinfra/anthropic/* reseller entry, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5, gpt-5-mini and gpt-5-nano lose the same way to replicate/openai/*, and some base rates were shadowed too (gemini-2.5-pro at 2x from a vercel_ai_gateway entry). A slash-free key is the canonical entry for the name transcripts actually record, so it now always wins its slot; a prefix-stripped key only fills a slot no canonical entry claims. The ?? input fallback is unchanged: it is correct for models that genuinely publish no cache pricing. Fixespingdotgg#8534

derektrimm · 2026-08-31T19:14:57Z

Closing - #8806 landed a more complete version of this fix (preserving qualified keys rather than collapsing them) and users on #8534 confirm 0.0.37 resolves it. Glad it's fixed.

derektrimm · 2026-08-28T14:43:05Z

What Changed

parseRateTable now gives a canonical (slash-free) LiteLLM key precedence over prefix-stripped keys that normalize to the same name. A prefix-stripped entry only fills a slot no canonical entry claims. Added usagePricing.test.ts (the file did not exist) covering both collision orders, differing base rates, prefixed-only models, the missing-cache-fields fallback, dotted Bedrock-style keys, and end-to-end cost + savings arithmetic in the exact #8534 shape.

Why

The rate table is built last-entry-wins over normalized names. In today's LiteLLM document the canonical claude-fable-5 / claude-opus-5 / claude-opus-4-8 / claude-sonnet-5 entries are shadowed by deepinfra/anthropic/* reseller entries that carry no cache_read_input_token_cost, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5 / gpt-5-mini / gpt-5-nano lose the same way to replicate/openai/*.

The reporter's screenshot in #8534 confirms this to the cent: fable-5 averages $10.14/M (full input rate) on a bucket that is overwhelmingly cache reads, the six model costs sum exactly to the $62,540.18 headline, and the $9.45 cache savings is producible only by the two models whose canonical keys happen to serialize last. Full arithmetic in the issue comment.

Deliberately unchanged: the ?? input fallback (correct for models that genuinely publish no cache pricing) and stripped-vs-stripped ordering for names with no canonical entry.

Note: beyond the cache column, this also corrects base rates for models whose canonical entries were shadowed by gateway variants with different prices (e.g. gemini-2.5-pro was priced at 2x from a vercel_ai_gateway/* entry). Every retarget moves toward the canonical price.

Fixes#8534

Checklist

This PR is small and focused
I explained what changed and why
I included before/after screenshots for any UI changes (server-side pricing only; no UI change)
I included a video for animation/interaction changes (n/a)

Note

Medium Risk
Changes reported usage cost and cache-savings math for models affected by LiteLLM key collisions; no auth or payment flow changes, but billing visibility shifts toward canonical rates.

Overview
Fixes #8534, where reseller/gateway LiteLLM keys (e.g. deepinfra/anthropic/claude-fable-5) that normalize to the same model name were overwriting canonical slash-free entries, stripping cache discount rates and sometimes doubling base input/output prices.

parseRateTable now treats keys without / as canonical: they always claim their normalized slot, and prefixed keys are skipped once a canonical name exists; prefixed-only models still populate the table when no canonical key is present.

Adds usagePricing.test.ts with collision ordering, gateway vs canonical base rates, cache fallbacks, dot-prefixed Bedrock-style keys, and end-to-end priceUsage / cacheSavingsUsd checks for the fable-5 scenario.

^{Reviewed by Cursor Bugbot for commit 9b49df8. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Fix `parseRateTable` to let canonical LiteLLM keys win rate-table collisions

parseRateTable now tracks canonical (slash-free) model names in a Set and skips prefixed entries (e.g. deepinfra/anthropic/claude-fable-5) when a canonical entry for the same normalized name already exists.
Cache read/creation costs default to the input cost when the rate entry omits them.
Adds tests in usagePricing.test.ts covering canonical-vs-prefixed collisions, cache-rate defaults, dot-prefixed names, and savings calculations.
Behavioral Change: prefixed entries that previously overwrote canonical rates for the same normalized name are now ignored; consumers relying on prefixed-override behavior will see the canonical rates instead.

^{Macroscope summarized 9b49df8.}

coderabbitai · 2026-08-28T14:43:12Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d48f4f90-9193-493a-b2c8-4b709a4b3450

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

macroscopeapp · 2026-08-28T14:51:48Z

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This production change alters model-derived usage-dollar and cache-savings calculations across the usage aggregation path. Although the implementation is small and regression-tested, pricing and metering behavior warrants human review.

^{You can add or adjust custom eligibility rules. Learn more.}

parseRateTable strips provider prefixes for lookup, so several LiteLLM keys collapse onto one normalized name and the last entry in the document silently wins. For claude-fable-5, claude-opus-5, claude-opus-4-8 and claude-sonnet-5 that winner is a cache-less deepinfra/anthropic/* reseller entry, so the ?? input fallback priced every cache read at full input rate and cache savings computed as zero. gpt-5, gpt-5-mini and gpt-5-nano lose the same way to replicate/openai/*, and some base rates were shadowed too (gemini-2.5-pro at 2x from a vercel_ai_gateway entry). A slash-free key is the canonical entry for the name transcripts actually record, so it now always wins its slot; a prefix-stripped key only fills a slot no canonical entry claims. The ?? input fallback is unchanged: it is correct for models that genuinely publish no cache pricing. Fixespingdotgg#8534

derektrimm · 2026-08-31T19:14:57Z

Closing - #8806 landed a more complete version of this fix (preserving qualified keys rather than collapsing them) and users on #8534 confirm 0.0.37 resolves it. Glad it's fixed.

github-actionsBot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:S 10-29 changed lines (additions + deletions). labels Aug 28, 2026