Skip to content

docs(providers): a model server you run needs no API key#178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
indykish wants to merge 1 commit into from
Closed

docs(providers): a model server you run needs no API key #178

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api-reference/error-codes.mdx
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -190,7 +190,7 @@ A conflict response also contains `current_state`. Some errors contain `user_mes
|---|---|---|---|---|
| <span id="UZ-PROVIDER-001"></span>`UZ-PROVIDER-001` | 400 | secret_ref required when mode=self_managed | Pick a secret to activate. Choose a stored secret before switching to a self-managed model. | Configure a supported provider, model, and secret. |
| <span id="UZ-PROVIDER-002"></span>`UZ-PROVIDER-002` | 400 | Secret not found | We couldn't find that secret. Store it under Secrets & ENVs, then try again. | Configure a supported provider, model, and secret. |
| <span id="UZ-PROVIDER-003"></span>`UZ-PROVIDER-003` | 400 | Secret JSON missing required field | That secret is missing required fields. It needs a provider set (and an API key for a named provider) — edit it under Secrets & ENVs and add them. | Configure a supported provider, model, and secret. |
| <span id="UZ-PROVIDER-003"></span>`UZ-PROVIDER-003` | 400 | Secret JSON missing required field | That secret needs a provider set. It needs an API key too, unless it points at a custom endpoint or a model server you run yourself. Edit it under Secrets & ENVs. | Configure a supported provider, model, and secret. |
| <span id="UZ-PROVIDER-004"></span>`UZ-PROVIDER-004` | 400 | Model not in library | That model isn't in our library yet. Pick a listed model, or ask us to add support for it. | Configure a supported provider, model, and secret. |
| <span id="UZ-PROVIDER-005"></span>`UZ-PROVIDER-005` | 400 | Custom endpoint base_url invalid or unsafe | That endpoint URL isn't allowed. Use a public https URL for your custom endpoint. | Configure a supported provider, model, and secret. |
| <span id="UZ-PROVIDER-006"></span>`UZ-PROVIDER-006` | 404 | Library model not found | We couldn't find that model in the library. Refresh the list and try again. | Configure a supported provider, model, and secret. |
Expand Down
33 changes: 31 additions & 2 deletions fleets/model-providers.mdx
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -41,7 +41,9 @@ The provider column supplies `--provider` and the model column supplies

## Steps

1. Store the provider credential. The provider id, API key, and model are all required.
1. Store the provider credential. The provider id and model are always required.
So is the API key, unless the provider is a custom endpoint or a model server
you run yourself — both are covered below.

```bash
agentsfleet secret create my-openai --provider openai --api-key sk-... --model gpt-5.6-sol
Expand DownExpand Up@@ -100,6 +102,32 @@ agentsfleet secret create my-gateway --provider openai-compatible \
✓ Secret 'my-gateway' stored in vault.
```

## Point at a model server you run

A model server on your own hardware takes its own provider id: `ollama`,
`vllm`, `llama.cpp`, `lm-studio`, `sglang`, `litellm`, or `osaurus`. Name the
model exactly as your server serves it — whatever `ollama pull` wrote, or
whatever you passed to `--served-model-name`.

```bash
agentsfleet secret create my-ollama --provider ollama --model llama-3.3-70b-my-finetune
```

```text
✓ Secret 'my-ollama' stored in vault.
```

No `--api-key`: a server on your own hardware authenticates nobody, so none is
required. Pass one and it is stored, in case your server checks it.

No `--base-url` either, and it is refused here. These providers dial a fixed
local address, so the endpoint is not yours to choose. Use the
`openai-compatible` form above for a server on a URL you pick.

Your model is not checked against the catalogue for these providers — it cannot
be, since the set of models you might load is yours. A typo is accepted and
fails when the fleet first runs, so check the spelling.
Comment on lines +127 to +129

@greptile-appsgreptile-appsBotAug 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1Catalogue validation remains contradictory

When a user creates or updates a local model-server credential, this section permits an unchecked model name while the guide introduction and shared CLI reference still guarantee catalogue validation, causing users to store misspelled models that fail only when a fleet first runs.

Knowledge Base Used:Model-provider credentials

Prompt To Fix With AI
This is a comment left during a code review.
Path: fleets/model-providers.mdx
Line: 127-129
Comment:
**Catalogue validation remains contradictory**
When a user creates or updates a local model-server credential, this section permits an unchecked model name while the guide introduction and shared CLI reference still guarantee catalogue validation, causing users to store misspelled models that fail only when a fleet first runs.
**Knowledge Base Used:**[Model-provider credentials](https://app.greptile.com/agentsfleet/-/custom-context/knowledge-base/agentsfleet/docs/-/docs/model-providers.md)---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.


## Verify it works

Run `agentsfleet tenant provider show` and check it names your secret. Then steer any fleet; its responses now come from your provider, and its usage lands on your provider account.
Expand All@@ -119,7 +147,8 @@ that carry no API key, so a stored credential could never reach them. The
`--api-key` or `--model` without `--provider` is refused. The three flags describe one credential and travel together.

A `--model` the provider does not serve is refused, and the message lists that
provider's models. Run `agentsfleet models --provider <ID>` for the list.
provider's models. Run `agentsfleet models --provider <ID>` for the list. This
check does not apply to a model server you run yourself — see above.

`--base-url` with a named provider is refused. A named provider carries its own endpoint; the flag belongs to `openai-compatible` alone.

Expand Down
Loading