Uh oh!
There was an error while loading. Please reload this page.
docs: cover RateLimitPolicy, ttft_ms, runtime cooldown, on_all_filtered - #281
Merged
Conversation
…own / on_all_filtered (#268) Bring customer docs back in line with main after the post-#249 feature PRs: - rate-limits.md: rewrite around the three enforced layers (ApiKey inline, Model inline, RateLimitPolicy entities). Add scope/window reference, window-mapping table, two example policies, and the etcd-only provisioning path. - api-keys.md: add team_id / owner_id as bucket identifiers and drop the stale "ApiKey-only enforcement" framing. - models.md: document the cooldown config block (default_seconds, max_seconds, honor_retry_after, trigger_statuses, trigger_on_timeout, trigger_on_transport) alongside background_model_check, and surface the runtime-status linkage. - routing-and-failover.md: rewrite the runtime-filter description as a 4-step loop and document on_all_filtered (fail vs original_order). - errors-and-retries.md / headers-and-error-codes.md: add the 503 all_candidates_unavailable error, distinct from provider_unavailable. - usage-events-and-billing.md / metrics-and-logs.md: document the new ttft_ms field and the streaming-only contract. - resource-schemas.md / glossary.md / core-concepts.md / feature-matrix.md / admin-api-reference.md: add RateLimitPolicy to the resource maps, glossary, core concepts, feature matrix, and call out that it has no admin CRUD route today.
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (4)
✅ Files skipped from review due to trivial changes (4)
📝 WalkthroughWalkthroughDocumentation-only updates: introduce ChangesRate limiting, routing, model behavior, and observability
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Note 🎁 Summarized by CodeRabbit FreeYour organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above. Comment |
- rate-limits.md: out-of-enum windows are rejected at etcd load by the JSON Schema, not coerced into an unrestricted policy - models.md: clarify that effective cooldown defaults come from accessor fallbacks, not from schema-level field defaults (every field is null until set) - routing-and-failover.md: detangle the cooldown-fallback step — drop the tautological "non-unhealthy cooldown target" phrasing and describe the actual condition (any cooldown candidate exists, then dispatch to all non-unhealthy targets) - usage-events-and-billing.md: ttft_ms is set on the first chunk carrying generated output, which includes tool-call deltas, not just text content
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The customer docs hadn't picked up several feature PRs that landed since the docs rebuild in #249. This PR closes those gaps:
api_key/model/team/member), window (second/minute/hour), andmax_requests/max_tokens. Loaded from etcd directly (no admin CRUD).cooldownconfig block androuting.on_all_filteredpolicy were undocumented; onlybackground_model_checkhad partial coverage.UsageEvent, streaming-only.#278 (etcd endpoint derivation on subsequent boot) is a pure internal fix with no operator-visible behavior change, so it's not included here.
Changes
docs/configuration/rate-limits.mddocs/configuration/api-keys.mdteam_id/owner_idbucket fields; drop stale "ApiKey-only enforcement" framingdocs/configuration/models.mdcooldownconfig block (defaults: 30s / 600s cap /[401, 408, 429, 500, 502, 503, 504]triggers / honorRetry-After)docs/configuration/routing-and-failover.mdon_all_filtered(failvsoriginal_order)docs/integration/errors-and-retries.md503 all_candidates_unavailablerow, distinct fromprovider_unavailabledocs/reference/headers-and-error-codes.mdall_candidates_unavailableto the error-type list and 503 status notedocs/cloud/usage-events-and-billing.mdlatency_msandttft_msdocs/operations/metrics-and-logs.mddocs/reference/resource-schemas.mdRateLimitPolicyto the resource map and field-shape notedocs/reference/admin-api-reference.mdRateLimitPolicydocs/reference/glossary.mdRateLimitPolicytermdocs/overview/core-concepts.mdteam_id/owner_idto ApiKey; new "Rate Limit Policy" concept sectiondocs/overview/feature-matrix.mdTest plan
models.md#cooldownrouting-and-failover.md#all-targets-filtered-policyrouting-and-failover.md#runtime-filteringrate-limits.md#rate-limit-policy-entitiesrate-limits.md#troubleshootingcore-concepts.md#rate-limit-policyadmin-api-reference.md#runtime-model-statuscrates/aisix-core/src/models/schema.rsand the structs incrates/aisix-core/src/models/{rate_limit_policy,model,routing}.rs503 all_candidates_unavailabletext matches whatcrates/aisix-proxy/src/error.rsemitsSummary by CodeRabbit