From 16ec58af7dca40eb96b639eb0746d29361058f97 Mon Sep 17 00:00:00 2001 From: Kishore Kumar Date: Mon, 17 Aug 2026 10:56:29 +0530 Subject: [PATCH] docs(providers): a model server you run needs no API key MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit UZ-PROVIDER-003's copy said an API key is required for "a named provider". A local runtime is named too, so the message sent operators hunting for a key their own server never issues. It now names both exemptions — a custom endpoint and a model server you run yourself. The provider guide gained the section that was missing entirely: how to point at ollama, vllm, llama.cpp, lm-studio, sglang, litellm, or osaurus, with the two rules that differ there (no API key, and the model is not checked against the catalogue). Two earlier claims on that page were corrected to match — the API key is no longer unconditional, and the model check now names its exception. Companion to the agentsfleet M168 provider-curation PR. Co-Authored-By: Claude Opus 5 (1M context) --- api-reference/error-codes.mdx | 2 +- fleets/model-providers.mdx | 33 +++++++++++++++++++++++++++++++-- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/api-reference/error-codes.mdx b/api-reference/error-codes.mdx index 570dd0f..a4c9e10 100644 --- a/api-reference/error-codes.mdx +++ b/api-reference/error-codes.mdx @@ -190,7 +190,7 @@ A conflict response also contains `current_state`. Some errors contain `user_mes |---|---|---|---|---| | `UZ-PROVIDER-001` | 400 | secret_ref required when mode=self_managed | Pick a secret to activate. Choose a stored secret before switching to a self-managed model. | Configure a supported provider, model, and secret. | | `UZ-PROVIDER-002` | 400 | Secret not found | We couldn't find that secret. Store it under Secrets & ENVs, then try again. | Configure a supported provider, model, and secret. | -| `UZ-PROVIDER-003` | 400 | Secret JSON missing required field | That secret is missing required fields. It needs a provider set (and an API key for a named provider) — edit it under Secrets & ENVs and add them. | Configure a supported provider, model, and secret. | +| `UZ-PROVIDER-003` | 400 | Secret JSON missing required field | That secret needs a provider set. It needs an API key too, unless it points at a custom endpoint or a model server you run yourself. Edit it under Secrets & ENVs. | Configure a supported provider, model, and secret. | | `UZ-PROVIDER-004` | 400 | Model not in library | That model isn't in our library yet. Pick a listed model, or ask us to add support for it. | Configure a supported provider, model, and secret. | | `UZ-PROVIDER-005` | 400 | Custom endpoint base_url invalid or unsafe | That endpoint URL isn't allowed. Use a public https URL for your custom endpoint. | Configure a supported provider, model, and secret. | | `UZ-PROVIDER-006` | 404 | Library model not found | We couldn't find that model in the library. Refresh the list and try again. | Configure a supported provider, model, and secret. | diff --git a/fleets/model-providers.mdx b/fleets/model-providers.mdx index 275f24d..e31d85e 100644 --- a/fleets/model-providers.mdx +++ b/fleets/model-providers.mdx @@ -41,7 +41,9 @@ The provider column supplies `--provider` and the model column supplies ## Steps -1. Store the provider credential. The provider id, API key, and model are all required. +1. Store the provider credential. The provider id and model are always required. + So is the API key, unless the provider is a custom endpoint or a model server + you run yourself — both are covered below. ```bash agentsfleet secret create my-openai --provider openai --api-key sk-... --model gpt-5.6-sol @@ -100,6 +102,32 @@ agentsfleet secret create my-gateway --provider openai-compatible \ ✓ Secret 'my-gateway' stored in vault. ``` +## Point at a model server you run + +A model server on your own hardware takes its own provider id: `ollama`, +`vllm`, `llama.cpp`, `lm-studio`, `sglang`, `litellm`, or `osaurus`. Name the +model exactly as your server serves it — whatever `ollama pull` wrote, or +whatever you passed to `--served-model-name`. + +```bash +agentsfleet secret create my-ollama --provider ollama --model llama-3.3-70b-my-finetune +``` + +```text +✓ Secret 'my-ollama' stored in vault. +``` + +No `--api-key`: a server on your own hardware authenticates nobody, so none is +required. Pass one and it is stored, in case your server checks it. + +No `--base-url` either, and it is refused here. These providers dial a fixed +local address, so the endpoint is not yours to choose. Use the +`openai-compatible` form above for a server on a URL you pick. + +Your model is not checked against the catalogue for these providers — it cannot +be, since the set of models you might load is yours. A typo is accepted and +fails when the fleet first runs, so check the spelling. + ## Verify it works Run `agentsfleet tenant provider show` and check it names your secret. Then steer any fleet; its responses now come from your provider, and its usage lands on your provider account. @@ -119,7 +147,8 @@ that carry no API key, so a stored credential could never reach them. The `--api-key` or `--model` without `--provider` is refused. The three flags describe one credential and travel together. A `--model` the provider does not serve is refused, and the message lists that -provider's models. Run `agentsfleet models --provider ` for the list. +provider's models. Run `agentsfleet models --provider ` for the list. This +check does not apply to a model server you run yourself — see above. `--base-url` with a named provider is refused. A named provider carries its own endpoint; the flag belongs to `openai-compatible` alone.