Model-level rate limits are silently not enforced on /passthrough/*

[lxbme]

Summary

Rate limiting on /passthrough/:provider/*rest is broken along two independent paths:

1. RPM path (model layer). The model-layer rate limit — both the inline rate_limit on a Model and rate_limit_policies with scope: model — is never consulted for passthrough requests. This holds even when the request body names exactly the rate-limited model. Key/team/member request limits do apply.
2. TPM path (all layers). Token accounting never happens for passthrough traffic at any layer: a TPM limit configured on the api key (or any other scope) is never tripped by passthrough requests, and passthrough traffic consumes zero tokens from shared buckets. So even the layers whose request limits work on passthrough enforce only their RPM half.

Both failures are silent: configuration is accepted and distributed normally, the dashboard shows the limits as active, and no log or metric indicates that passthrough is exempt. The operator finds out from upstream bills or provider-side 429s.

Reported by a customer running embedding traffic through POST /passthrough/openai/embeddings/multimodals under a model-level 2 RPM / 1,000,000 TPM limit; both paths reproduced on the v0.3.1 release image.

Reproduction (v0.3.1)

Common setup: a ProviderKey with provider: <p>; one Model of that provider (upstream model_name: m); an ApiKey whose allowed_models includes it. Rate-limit windows are fixed per minute; each numbered experiment below runs in a fresh window.

A. Model-layer limits never consulted (RPM path)

Model configured with rate_limit: {rpm: 2, tpm: 1000000}, no key-level limit.

1) POST /passthrough/<p>/v1/chat/completions {"model": "m", ...} x4 -> 200, 200, 200, 200 (never 429)
 POST /v1/chat/completions {"model": "m", ...} x3 -> 200, 200, 429 (same window)

The passthrough requests target the exact model the limit is configured on, are never blocked, and are not even counted — the typed requests still get their full 2-request allowance in the same window.

Control: with an api-key-level rate_limit: {rpm: 2}, the 3rd passthrough request returns 429 immediately — key-layer request limiting works on passthrough, so the gap is specific to the model layer.

B. Token usage never accumulates (TPM path)

Api key configured with rate_limit: {tpm: 20} (a layer whose request limiting demonstrably works on passthrough); each chat call consumes ~11 tokens.

2) POST /v1/chat/completions {"model": "m", ...} x3 -> 200, 200, 429 ("token limit exceeded (tokens)")
3) POST /passthrough/<p>/v1/chat/completions {"model": "m", ...} x5 -> 200, 200, 200, 200, 200 (never 429)

Identical body, identical upstream model, identical key and TPM budget: the typed endpoint trips the token limit on the 3rd request; passthrough never trips it no matter how many requests are sent.

Root cause

The two paths fail by different mechanisms in crates/aisix-proxy/src/passthrough.rs.

RPM path: the model identity is never passed to the limiter

// Reserve the rate-limit layers AFTER the input guardrail so a content
// block doesn't burn an RPM slot, matching the typed endpoints. Passthrough
// has no resolved model, so only the api-key/team/member layers apply.
let _reservation = crate::quota::enforce(&state, auth, None).await?;

With model_rl = None, quota::reserve_layers skips Layer 2 (model inline limits) entirely, and Layer 3 policies with PolicyScope::Model can never match (model_rl.is_some_and(...)). None is passed unconditionally, regardless of what the request body contains — hence the bypass covers requests that name the rate-limited model.

The comment is stale: since #449 the dispatch path does resolve a borrowed model entry for the provider segment, and that entry already anchors the client-IP allowlist (#697), the guardrail chain (#911), cooldown marking (#701), and the request timeout (#554). It also does not reflect that the request body is fully buffered at this point (body_bytes, read for the input guardrail scan), so the request's own model field is in principle available.

TPM path: commit_tokens is never called

The limiter is two-phase (crates/aisix-ratelimit/src/limiter.rs): pre_commit reserves a request slot at admission, and the caller reports actual token usage after the upstream response via Reservation::commit_tokens(tokens); dropping the reservation without committing releases it with zero token debit.

Typed endpoints parse the provider response usage and commit it. Passthrough is a raw tunnel: it parses neither request nor response (prompt_tokens = 0 by design, #699), binds the reservation to _reservation, and never calls commit_tokens. Every token bucket it touches — key, team, member, and (if path 1 were fixed) model — therefore stays at zero forever, and token limits can never trip.

Note the two mechanisms are independent: fixing path 1 alone would make model-layer request limits work on passthrough, but the TPM half of any layer stays inert until passthrough has a token-usage source. The response body is in fact fully buffered as well (output guardrails scan it), so usage extraction is not architecturally ruled out — but no such parsing exists today.

Environment

• Reproduced on api7/aisix:0.3.1 (managed mode, control plane v0.3.1), code references at main (812fc72).

[coderabbitai]

🔗 Related PRs

#267 - feat: multi-dimension rate limiting (model, team, member) [merged]
#280 - feat: add RateLimitPolicy entity and policy-based rate limiting [merged]
#472 - fix(proxy): dispatch Model Groups across passthrough endpoints (#471) [merged]
#582 - feat(proxy): model-level client-IP CIDR access restriction (#557) [merged]
#683 - fix(security): data-plane P1 findings from AISIX-Cloud#911 [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
 auto_enrich:
 enabled: false

💬 Have feedback or questions? Drop into our discord!

[nic-6443]

passthrough 字面意义上就是网关不解析请求、响应，完全透传，所以无法执行基于 token 的限流很正常（请求限流技术则是可行的，是否有价值也难说），它的使用场景本身就是为了代理 provider 提供的非一等公民 api，例如我们文档中举例的 list models 这种 api，不存在需要限流、预算的场景。
issue中把 chat complation api 用 passthrough 功能做代理，这是错误的使用方式。

[jarvis9443]

Closing with #982 (merged): the implicit /passthrough/:provider/*rest tunnel this issue describes has been removed in favor of explicit passthrough_routes resources.

Where the two halves of this report landed:

• The RPM half (model-layer limits never consulted) was fixed earlier by fix(passthrough): enforce the body model's rate limit on the raw tunnel #805, and that body-model probe semantics carries over to inject-mode routes, scoped to the route's ProviderKey provider.
• The TPM half is structural for a raw tunnel — no parsing, no usage source — and stays documented as such for protocol: raw routes. The supported path to token accounting on passthrough traffic is now protocol: openai_chat / openai_completions, which extract usage (including the final streamed chunk) onto the usage event.