Add Novita as a model gateway backend#102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Shashankss1205 merged 3 commits into from
Aug 13, 2026
Merged

Add Novita as a model gateway backend #102

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7f98eaf
Add Novita as a model gateway backend
jax-novita Aug 5, 2026
8f177da
Recommend Novita's current flagship models
jax-novita Aug 5, 2026
2afecff
Fix column alignment for novita example in models cookbook
jax-novita Aug 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions docs/cookbook/02-models.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,7 @@ for spec in (
"claude-cli/claude-sonnet-5",
"openrouter/anthropic/claude-haiku-4.5",
"openai/gpt-4o-mini",
"novita/moonshotai/kimi-k3",
"ollama/llama3.1",
"mock/anything",
):
Expand All@@ -38,18 +39,20 @@ print(model.invoke("say hi").content)
{'spec': 'claude-cli/claude-sonnet-5', 'backend': 'claude-cli', 'model': 'claude-sonnet-5'}
{'spec': 'openrouter/anthropic/claude-haiku-4.5', 'backend': 'openrouter', 'model': 'anthropic/claude-haiku-4.5'}
{'spec': 'openai/gpt-4o-mini', 'backend': 'openai', 'model': 'gpt-4o-mini'}
{'spec': 'novita/moonshotai/kimi-k3', 'backend': 'novita', 'model': 'moonshotai/kimi-k3'}
{'spec': 'ollama/llama3.1', 'backend': 'ollama', 'model': 'llama3.1'}
{'spec': 'mock/anything', 'backend': 'mock', 'model': 'anything'}
hello from a scripted model
```

There are five backends:
There are six backends:

| backend | credential | what it is for |
| --- | --- | --- |
| `claude-cli` | a Claude subscription, no API key | text completion on quota you already pay for |
| `openrouter` | `OPENROUTER_API_KEY` | one key, most vendors, per-call cost in the response |
| `openai` | `OPENAI_API_KEY` | the OpenAI API directly, or any endpoint via `OPENAI_BASE_URL` |
| `novita` | `NOVITA_API_KEY` | Novita's own endpoint, token counts but no per-call cost |
| `ollama` | none — a local server | models on your own machine, free and offline |
| `mock` | none | a scripted test double |

Expand All@@ -63,8 +66,8 @@ needs it. Asking for `claude-cli` therefore does not require `langchain-openai`,
for `openrouter` does not require the Claude CLI to be installed. A missing optional
dependency fails for the backend that wanted it and nothing else.

`openrouter`, `openai`and `ollama` all speak the OpenAI wire format and share one base
class, so they behave identically on everything except money and routing: same
`openrouter`, `openai`, `novita` and `ollama` all speak the OpenAI wire format and share one
base class, so they behave identically on everything except money and routing: same
`bind_tools`, same `with_structured_output`, same streaming and async, same retry policy,
same usage envelope.

Expand DownExpand Up@@ -93,7 +96,7 @@ except UnknownBackendError as exc:
('openrouter', 'openai/gpt-4o-mini:floor')
('claude-cli', 'anthropic/claude-haiku-4.5')
('openai', 'gpt-4o-mini')
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, ollama, mock — or a bare model name for the claude-cli default
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, novita, ollama, mock — or a bare model name for the claude-cli default
```

Only the first segment is a backend, because OpenRouter model ids are themselves
Expand DownExpand Up@@ -122,16 +125,18 @@ model through the broker is still `openrouter/openai/gpt-4o-mini`.
<!-- verified: cli -->
```console
$ grapharc models
backends: claude-cli, openrouter, openai, ollama, mock
backends: claude-cli, openrouter, openai, novita, ollama, mock
openrouter key: <unset>
openai key: <unset>
novita key: <unset>
ollama url: http://localhost:11434/v1

examples:
claude-cli/claude-sonnet-5 subscription, no API key
openrouter/anthropic/claude-haiku-4.5 many providers, one key
openrouter/openai/gpt-4o-mini:floor cheapest provider for that model
openai/gpt-4o-mini the OpenAI API directly, your key
novita/moonshotai/kimi-k3 Novita's own endpoint, your key
ollama/llama3.1 a local server, no key and no bill

grapharc models --check probes which of these this machine can use
Expand DownExpand Up@@ -165,6 +170,8 @@ openrouter unusable no API key (set OPENROUTER_API_KEY, or add one to .env)
credential: <unset>
openai unusable no API key (set OPENAI_API_KEY, or add one to .env)
credential: <unset>
novita unusable no API key (set NOVITA_API_KEY, or add one to .env)
credential: <unset>
ollama usable local server at http://localhost:11434/v1
credential: none needed (local server)
mock usable scripted test double; never reaches a provider
Expand Down
4 changes: 4 additions & 0 deletions grapharc/cli/main.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -449,6 +449,7 @@ def _cmd_approve(args: argparse.Namespace) -> int:
def _cmd_models(args: argparse.Namespace) -> int:
from grapharc.gateway import (
describe,
novita_api_key,
ollama_base_url,
openai_api_key,
openrouter_api_key,
Expand DownExpand Up@@ -497,6 +498,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
"openrouter/anthropic/claude-haiku-4.5": "many providers, one key",
"openrouter/openai/gpt-4o-mini:floor": "cheapest provider for that model",
"openai/gpt-4o-mini": "the OpenAI API directly, your key",
"novita/moonshotai/kimi-k3": "Novita's own endpoint, your key",
"ollama/llama3.1": "a local server, no key and no bill",
}
payload = {
Expand All@@ -505,6 +507,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
"backends": list(BACKENDS),
"openrouter_key": redact(openrouter_api_key()),
"openai_key": redact(openai_api_key()),
"novita_key": redact(novita_api_key()),
# An address, not a secret: it is printed whole, and it is where a
# request would go rather than proof that anything is listening.
"ollama_base_url": ollama_base_url(),
Expand All@@ -516,6 +519,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
style.kv("backends", ", ".join(BACKENDS)),
style.kv("openrouter key", redact(openrouter_api_key())),
style.kv("openai key", redact(openai_api_key())),
style.kv("novita key", redact(novita_api_key())),
style.kv("ollama url", ollama_base_url(), tint=style.accent),
"",
style.heading("examples:"),
Expand Down
21 changes: 21 additions & 0 deletions grapharc/cli/probe.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -95,6 +95,26 @@ def _probe_openai() -> dict[str, Any]:
}


def _probe_novita() -> dict[str, Any]:
from grapharc.gateway import novita_api_key, redact

key = novita_api_key()
has_dependency = importlib.util.find_spec("langchain_openai") is not None
missing = []
if not key:
missing.append("no API key (set NOVITA_API_KEY, or add one to .env)")
if not has_dependency:
missing.append("langchain-openai not installed (uv sync --extra novita)")
return {
"backend": "novita",
"kind": KIND_PROVIDER,
"usable": bool(key) and has_dependency,
"credential": redact(key),
"detail": "; ".join(missing) or "api key configured and langchain-openai installed",
"checked": "credential presence only; no request was sent to api.novita.ai",
}


def _probe_ollama() -> dict[str, Any]:
from grapharc.gateway import ollama_base_url

Expand DownExpand Up@@ -152,6 +172,7 @@ def probe_backends(*, claude_path: str = "claude") -> list[dict[str, Any]]:
"claude-cli": lambda: _probe_claude_cli(claude_path),
"openrouter": _probe_openrouter,
"openai": _probe_openai,
"novita": _probe_novita,
"ollama": _probe_ollama,
"mock": _probe_mock,
}
Expand Down
9 changes: 7 additions & 2 deletions grapharc/gateway/__init__.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -6,6 +6,7 @@
get_model("claude-cli/claude-sonnet-5") # subscription, no API key
get_model("openrouter/anthropic/claude-sonnet-4.5") # many providers, one key
get_model("openai/gpt-4o-mini") # OPENAI_API_KEY
get_model("novita/moonshotai/kimi-k3") # NOVITA_API_KEY
get_model("ollama/llama3.1") # local server, no key
get_model("mock/x", responses=[...]) # deterministic tests

Expand All@@ -16,12 +17,13 @@
get_model(spec, cost_ceiling_usd=0.25) # raises when passed
get_model(spec, spend=shared_meter) # one ceiling, many models

The three OpenAI-wire backends (`openrouter`, `openai`, `ollama`) are imported
lazily — they need `langchain-openai`, which is an optional extra.
The four OpenAI-wire backends (`openrouter`, `openai`, `novita`, `ollama`) are
imported lazily — they need `langchain-openai`, which is an optional extra.
"""

from grapharc.gateway.claude_cli import ClaudeCodeCLIChatModel
from grapharc.gateway.config import (
novita_api_key,
ollama_api_key,
ollama_base_url,
openai_api_key,
Expand DownExpand Up@@ -66,6 +68,7 @@
"different_providers",
"get_model",
"is_transient",
"novita_api_key",
"ollama_api_key",
"ollama_base_url",
"openai_api_key",
Expand All@@ -85,6 +88,8 @@
"OpenRouterError": "openrouter",
"OpenAIChatModel": "openai",
"OpenAIError": "openai",
"NovitaChatModel": "novita",
"NovitaError": "novita",
"OllamaChatModel": "ollama",
"OllamaError": "ollama",
}
Expand Down
16 changes: 15 additions & 1 deletion grapharc/gateway/config.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -20,14 +20,17 @@
Secrets are returned, never logged. Anything that renders a config for humans
goes through `redact`.

Three key-holding backends, plus one that usually holds none:
Four key-holding backends, plus one that usually holds none:

- **OpenRouter** — `OPENROUTER_API_KEY`.
- **OpenAI** — `OPENAI_API_KEY`, optionally with `OPENAI_BASE_URL` for an
Azure-style or proxied endpoint. `langchain-openai` reads `OPENAI_API_KEY`
from the process environment on its own; going through here as well is what
adds `.env` support, the alternate spellings, and a failure that names the
variable instead of surfacing an SDK error.
- **Novita** — `NOVITA_API_KEY`. The endpoint is fixed
(`grapharc.gateway.novita.NOVITA_BASE_URL`), so unlike OpenAI there is no
base-url override to resolve here.
- **Ollama** — no credential by default: it is a server on your own machine.
What it needs is an address, so `ollama_base_url()` always returns one
(`OLLAMA_HOST` / `OLLAMA_BASE_URL`, else localhost). `OLLAMA_API_KEY` exists
Expand DownExpand Up@@ -56,6 +59,13 @@
"openai_api_key",
)

NOVITA_KEYS = (
"NOVITA_API_KEY",
"NOVITA_KEY",
"novita-api-key",
"novita_api_key",
)

# `OPENAI_API_BASE` is the older spelling and is still what a lot of tooling
# sets; both are accepted, the current one first.
OPENAI_BASE_URL_KEYS = (
Expand DownExpand Up@@ -133,6 +143,10 @@ def openai_api_key(*, env_file: Path | None = None) -> str | None:
return get_secret(OPENAI_KEYS, env_file=env_file)


def novita_api_key(*, env_file: Path | None = None) -> str | None:
return get_secret(NOVITA_KEYS, env_file=env_file)


def openai_base_url(*, env_file: Path | None = None) -> str | None:
"""An endpoint override, or None for api.openai.com.

Expand Down
59 changes: 59 additions & 0 deletions grapharc/gateway/novita.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
"""Novita backend — a GPU cloud hosting open-weight models, one key.

`novita/moonshotai/kimi-k3` reaches Novita's own OpenAI-compatible endpoint
(`https://api.novita.ai/openai`), not api.openai.com, so this builds on
`OpenAICompatChatModel` the same way `openrouter.py` and `ollama.py` do rather
than on `openai.py`: the endpoint is fixed, not an override of OpenAI's own.

Model ids on Novita are themselves `author/slug` — `moonshotai/kimi-k3`,
`zai-org/glm-5.2` — the same shape OpenRouter uses, so `vendor()` in
`registry.py` already reads the right author off a Novita spec with no
backend-specific handling: `BACKEND_VENDOR` stays absent for `novita`, exactly
as it is absent for `openrouter`.

**Novita reports no per-call cost.** Unlike OpenRouter, the chat-completions
response carries token counts and nothing else, so this backend is the OpenAI
one in that respect: `_provider_cost` is the base class's `None`, and
`cost_ceiling_usd` counts calls in `SpendMeter.unpriced_calls` unless a caller
supplies `price_per_million=`.
"""

from __future__ import annotations

from typing import Any

from grapharc.gateway.config import novita_api_key
from grapharc.gateway.openai_compat import OpenAICompatChatModel

NOVITA_BASE_URL = "https://api.novita.ai/openai"


class NovitaError(Exception):
"""The Novita backend could not be constructed or used."""


class NovitaChatModel(OpenAICompatChatModel):
"""A LangChain chat model over Novita's OpenAI-compatible endpoint."""

def __init__(self, model: str, /, **kwargs: Any) -> None:
api_key = kwargs.pop("api_key", None) or novita_api_key()
if not api_key:
raise NovitaError(
"No Novita API key found. Set NOVITA_API_KEY in the environment, "
"or add one of NOVITA_API_KEY / novita-api-key to a .env file."
)
# One retry layer, not two — same reasoning as the OpenRouter backend.
kwargs.setdefault("max_retries", 0)
super().__init__(
model=model,
api_key=api_key,
base_url=kwargs.pop("base_url", None) or NOVITA_BASE_URL,
**kwargs,
)

@property
def _llm_type(self) -> str:
return "grapharc-novita"


__all__ = ["NOVITA_BASE_URL", "NovitaChatModel", "NovitaError"]
15 changes: 14 additions & 1 deletion grapharc/gateway/registry.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,7 @@
openrouter/anthropic/claude-sonnet-4.5 -> OpenRouter
openrouter/openai/gpt-4o:floor -> OpenRouter, cheapest provider
openai/gpt-4o-mini -> OpenAI directly (OPENAI_API_KEY)
novita/moonshotai/kimi-k3 -> Novita (NOVITA_API_KEY)
ollama/llama3.1 -> a local Ollama server, no key
mock/anything -> scripted test double

Expand All@@ -28,7 +29,7 @@ class UnknownBackendError(Exception):
"""The spec named a backend that is not registered."""


BACKENDS = ("claude-cli", "openrouter", "openai", "ollama", "mock")
BACKENDS = ("claude-cli", "openrouter", "openai", "novita", "ollama", "mock")

# Authors that appear in OpenRouter model ids. A spec starting with one of
# these is a model name, not a mistyped backend — `anthropic/claude-haiku-4.5`
Expand DownExpand Up@@ -71,6 +72,7 @@ class UnknownBackendError(Exception):
_BARE_BACKEND_EXAMPLE = {
"openrouter": "openrouter/anthropic/claude-sonnet-4.5",
"openai": "openai/gpt-4o-mini",
"novita": "novita/moonshotai/kimi-k3",
"ollama": "ollama/llama3.1",
}

Expand DownExpand Up@@ -160,6 +162,17 @@ def get_model(spec: str, **kwargs: Any) -> BaseChatModel:

return OpenAIChatModel(model, **kwargs)

if backend == "novita":
try:
from grapharc.gateway.novita import NovitaChatModel
except ImportError as exc: # pragma: no cover - depends on install extras
raise UnknownBackendError(
"The Novita backend needs langchain-openai. "
"Install it with: uv sync --extra novita"
) from exc

return NovitaChatModel(model, **kwargs)

if backend == "ollama":
try:
from grapharc.gateway.ollama import OllamaChatModel
Expand Down
8 changes: 7 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -71,6 +71,12 @@ openrouter = [
openai = [
"langchain-openai>=0.2",
]
# Imported by grapharc/gateway/novita.py. Novita speaks the OpenAI wire format
# against its own endpoint, so this needs the same client and no Novita-specific
# package.
novita = [
"langchain-openai>=0.2",
]
# Imported by grapharc/gateway/ollama.py. Ollama speaks the OpenAI wire format,
# so the local backend needs the same client and no ollama-specific package.
ollama = [
Expand DownExpand Up@@ -110,7 +116,7 @@ slack = [
]
# Everything above. Self-referential so it cannot drift out of sync.
all = [
"grapharc[api,ladybug,mcp,ollama,openai,openrouter,otel,server,slack]",
"grapharc[api,ladybug,mcp,novita,ollama,openai,openrouter,otel,server,slack]",
]

[dependency-groups]
Expand Down
4 changes: 3 additions & 1 deletion tests/test_cli.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,7 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
for name in (
"OPENROUTER_API_KEY", "OPENROUTER_KEY", "open-router-api-key",
"OPENAI_API_KEY", "OPENAI_KEY", "openai-api-key",
"NOVITA_API_KEY", "NOVITA_KEY", "novita-api-key",
"OLLAMA_HOST", "OLLAMA_BASE_URL",
):
monkeypatch.delenv(name, raising=False)
Expand All@@ -452,10 +453,11 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
"claude-cli": False,
"openrouter": False,
"openai": False,
"novita": False,
"ollama": False,
"mock": True,
}
for backend in ("openrouter", "openai"):
for backend in ("openrouter", "openai", "novita"):
assert next(b for b in payload["backends"] if b["backend"] == backend)[
"credential"
] == "<unset>"
Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Add Novita as a model gateway backend#102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Shashankss1205
merged 3 commits into

from

Aug 13, 2026
 Merged


Add Novita as a model gateway backend
#102
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
7f98eaf

Add Novita as a model gateway backend


jax-novita Aug 5, 2026
8f177da

Recommend Novita's current flagship models


jax-novita Aug 5, 2026
2afecff

Fix column alignment for novita example in models cookbook


jax-novita Aug 10, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 12 additions & 5 deletions
docs/cookbook/02-models.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,7 @@ for spec in (
 "claude-cli/claude-sonnet-5",
 "openrouter/anthropic/claude-haiku-4.5",
 "openai/gpt-4o-mini",
 "novita/moonshotai/kimi-k3",
 "ollama/llama3.1",
 "mock/anything",
):
Expand All@@ -38,18 +39,20 @@ print(model.invoke("say hi").content)
{'spec': 'claude-cli/claude-sonnet-5', 'backend': 'claude-cli', 'model': 'claude-sonnet-5'}
{'spec': 'openrouter/anthropic/claude-haiku-4.5', 'backend': 'openrouter', 'model': 'anthropic/claude-haiku-4.5'}
{'spec': 'openai/gpt-4o-mini', 'backend': 'openai', 'model': 'gpt-4o-mini'}
{'spec': 'novita/moonshotai/kimi-k3', 'backend': 'novita', 'model': 'moonshotai/kimi-k3'}
{'spec': 'ollama/llama3.1', 'backend': 'ollama', 'model': 'llama3.1'}
{'spec': 'mock/anything', 'backend': 'mock', 'model': 'anything'}
hello from a scripted model
```

There are five backends:
There are six backends:

| backend | credential | what it is for |
| --- | --- | --- |
| `claude-cli` | a Claude subscription, no API key | text completion on quota you already pay for |
| `openrouter` | `OPENROUTER_API_KEY` | one key, most vendors, per-call cost in the response |
| `openai` | `OPENAI_API_KEY` | the OpenAI API directly, or any endpoint via `OPENAI_BASE_URL` |
| `novita` | `NOVITA_API_KEY` | Novita's own endpoint, token counts but no per-call cost |
| `ollama` | none — a local server | models on your own machine, free and offline |
| `mock` | none | a scripted test double |

Expand All@@ -63,8 +66,8 @@ needs it. Asking for `claude-cli` therefore does not require `langchain-openai`,
for `openrouter` does not require the Claude CLI to be installed. A missing optional
dependency fails for the backend that wanted it and nothing else.

`openrouter`, `openai`and `ollama` all speak the OpenAI wire format and share one base
class, so they behave identically on everything except money and routing: same
`openrouter`, `openai`, `novita` and `ollama` all speak the OpenAI wire format and share one
base class, so they behave identically on everything except money and routing: same
`bind_tools`, same `with_structured_output`, same streaming and async, same retry policy,
same usage envelope.

Expand DownExpand Up@@ -93,7 +96,7 @@ except UnknownBackendError as exc:
('openrouter', 'openai/gpt-4o-mini:floor')
('claude-cli', 'anthropic/claude-haiku-4.5')
('openai', 'gpt-4o-mini')
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, ollama, mock — or a bare model name for the claude-cli default
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, novita, ollama, mock — or a bare model name for the claude-cli default
```

Only the first segment is a backend, because OpenRouter model ids are themselves
Expand DownExpand Up@@ -122,16 +125,18 @@ model through the broker is still `openrouter/openai/gpt-4o-mini`.
<!-- verified: cli -->
```console
$ grapharc models
backends: claude-cli, openrouter, openai, ollama, mock
backends: claude-cli, openrouter, openai, novita, ollama, mock
openrouter key: <unset>
openai key: <unset>
novita key: <unset>
ollama url: http://localhost:11434/v1

examples:
 claude-cli/claude-sonnet-5 subscription, no API key
 openrouter/anthropic/claude-haiku-4.5 many providers, one key
 openrouter/openai/gpt-4o-mini:floor cheapest provider for that model
 openai/gpt-4o-mini the OpenAI API directly, your key
 novita/moonshotai/kimi-k3 Novita's own endpoint, your key
 ollama/llama3.1 a local server, no key and no bill

grapharc models --check probes which of these this machine can use
Expand DownExpand Up@@ -165,6 +170,8 @@ openrouter unusable no API key (set OPENROUTER_API_KEY, or add one to .env)
 credential: <unset>
openai unusable no API key (set OPENAI_API_KEY, or add one to .env)
 credential: <unset>
novita unusable no API key (set NOVITA_API_KEY, or add one to .env)
 credential: <unset>
ollama usable local server at http://localhost:11434/v1
 credential: none needed (local server)
mock usable scripted test double; never reaches a provider
Expand Down

4 changes: 4 additions & 0 deletions
grapharc/cli/main.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -449,6 +449,7 @@ def _cmd_approve(args: argparse.Namespace) -> int:
def _cmd_models(args: argparse.Namespace) -> int:
 from grapharc.gateway import (
 describe,
 novita_api_key,
 ollama_base_url,
 openai_api_key,
 openrouter_api_key,
Expand DownExpand Up@@ -497,6 +498,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "openrouter/anthropic/claude-haiku-4.5": "many providers, one key",
 "openrouter/openai/gpt-4o-mini:floor": "cheapest provider for that model",
 "openai/gpt-4o-mini": "the OpenAI API directly, your key",
 "novita/moonshotai/kimi-k3": "Novita's own endpoint, your key",
 "ollama/llama3.1": "a local server, no key and no bill",
 }
 payload = {
Expand All@@ -505,6 +507,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "backends": list(BACKENDS),
 "openrouter_key": redact(openrouter_api_key()),
 "openai_key": redact(openai_api_key()),
 "novita_key": redact(novita_api_key()),
 # An address, not a secret: it is printed whole, and it is where a
 # request would go rather than proof that anything is listening.
 "ollama_base_url": ollama_base_url(),
Expand All@@ -516,6 +519,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 style.kv("backends", ", ".join(BACKENDS)),
 style.kv("openrouter key", redact(openrouter_api_key())),
 style.kv("openai key", redact(openai_api_key())),
 style.kv("novita key", redact(novita_api_key())),
 style.kv("ollama url", ollama_base_url(), tint=style.accent),
 "",
 style.heading("examples:"),
Expand Down

21 changes: 21 additions & 0 deletions
grapharc/cli/probe.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -95,6 +95,26 @@ def _probe_openai() -> dict[str, Any]:
 }


def _probe_novita() -> dict[str, Any]:
 from grapharc.gateway import novita_api_key, redact

 key = novita_api_key()
 has_dependency = importlib.util.find_spec("langchain_openai") is not None
 missing = []
 if not key:
 missing.append("no API key (set NOVITA_API_KEY, or add one to .env)")
 if not has_dependency:
 missing.append("langchain-openai not installed (uv sync --extra novita)")
 return {
 "backend": "novita",
 "kind": KIND_PROVIDER,
 "usable": bool(key) and has_dependency,
 "credential": redact(key),
 "detail": "; ".join(missing) or "api key configured and langchain-openai installed",
 "checked": "credential presence only; no request was sent to api.novita.ai",
 }


def _probe_ollama() -> dict[str, Any]:
 from grapharc.gateway import ollama_base_url

Expand DownExpand Up@@ -152,6 +172,7 @@ def probe_backends(*, claude_path: str = "claude") -> list[dict[str, Any]]:
 "claude-cli": lambda: _probe_claude_cli(claude_path),
 "openrouter": _probe_openrouter,
 "openai": _probe_openai,
 "novita": _probe_novita,
 "ollama": _probe_ollama,
 "mock": _probe_mock,
 }
Expand Down

9 changes: 7 additions & 2 deletions
grapharc/gateway/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6,6 +6,7 @@
 get_model("claude-cli/claude-sonnet-5") # subscription, no API key
 get_model("openrouter/anthropic/claude-sonnet-4.5") # many providers, one key
 get_model("openai/gpt-4o-mini") # OPENAI_API_KEY
 get_model("novita/moonshotai/kimi-k3") # NOVITA_API_KEY
 get_model("ollama/llama3.1") # local server, no key
 get_model("mock/x", responses=[...]) # deterministic tests

Expand All@@ -16,12 +17,13 @@
 get_model(spec, cost_ceiling_usd=0.25) # raises when passed
 get_model(spec, spend=shared_meter) # one ceiling, many models

The three OpenAI-wire backends (`openrouter`, `openai`, `ollama`) are imported
lazily — they need `langchain-openai`, which is an optional extra.
The four OpenAI-wire backends (`openrouter`, `openai`, `novita`, `ollama`) are
imported lazily — they need `langchain-openai`, which is an optional extra.
"""

from grapharc.gateway.claude_cli import ClaudeCodeCLIChatModel
from grapharc.gateway.config import (
 novita_api_key,
 ollama_api_key,
 ollama_base_url,
 openai_api_key,
Expand DownExpand Up@@ -66,6 +68,7 @@
 "different_providers",
 "get_model",
 "is_transient",
 "novita_api_key",
 "ollama_api_key",
 "ollama_base_url",
 "openai_api_key",
Expand All@@ -85,6 +88,8 @@
 "OpenRouterError": "openrouter",
 "OpenAIChatModel": "openai",
 "OpenAIError": "openai",
 "NovitaChatModel": "novita",
 "NovitaError": "novita",
 "OllamaChatModel": "ollama",
 "OllamaError": "ollama",
}
Expand Down

16 changes: 15 additions & 1 deletion
grapharc/gateway/config.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -20,14 +20,17 @@
Secrets are returned, never logged. Anything that renders a config for humans
goes through `redact`.

Three key-holding backends, plus one that usually holds none:
Four key-holding backends, plus one that usually holds none:

- **OpenRouter** — `OPENROUTER_API_KEY`.
- **OpenAI** — `OPENAI_API_KEY`, optionally with `OPENAI_BASE_URL` for an
 Azure-style or proxied endpoint. `langchain-openai` reads `OPENAI_API_KEY`
 from the process environment on its own; going through here as well is what
 adds `.env` support, the alternate spellings, and a failure that names the
 variable instead of surfacing an SDK error.
- **Novita** — `NOVITA_API_KEY`. The endpoint is fixed
 (`grapharc.gateway.novita.NOVITA_BASE_URL`), so unlike OpenAI there is no
 base-url override to resolve here.
- **Ollama** — no credential by default: it is a server on your own machine.
 What it needs is an address, so `ollama_base_url()` always returns one
 (`OLLAMA_HOST` / `OLLAMA_BASE_URL`, else localhost). `OLLAMA_API_KEY` exists
Expand DownExpand Up@@ -56,6 +59,13 @@
 "openai_api_key",
)

NOVITA_KEYS = (
 "NOVITA_API_KEY",
 "NOVITA_KEY",
 "novita-api-key",
 "novita_api_key",
)

# `OPENAI_API_BASE` is the older spelling and is still what a lot of tooling
# sets; both are accepted, the current one first.
OPENAI_BASE_URL_KEYS = (
Expand DownExpand Up@@ -133,6 +143,10 @@ def openai_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(OPENAI_KEYS, env_file=env_file)


def novita_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(NOVITA_KEYS, env_file=env_file)


def openai_base_url(*, env_file: Path | None = None) -> str | None:
 """An endpoint override, or None for api.openai.com.

Expand Down

59 changes: 59 additions & 0 deletions
grapharc/gateway/novita.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
"""Novita backend — a GPU cloud hosting open-weight models, one key.

`novita/moonshotai/kimi-k3` reaches Novita's own OpenAI-compatible endpoint
(`https://api.novita.ai/openai`), not api.openai.com, so this builds on
`OpenAICompatChatModel` the same way `openrouter.py` and `ollama.py` do rather
than on `openai.py`: the endpoint is fixed, not an override of OpenAI's own.

Model ids on Novita are themselves `author/slug` — `moonshotai/kimi-k3`,
`zai-org/glm-5.2` — the same shape OpenRouter uses, so `vendor()` in
`registry.py` already reads the right author off a Novita spec with no
backend-specific handling: `BACKEND_VENDOR` stays absent for `novita`, exactly
as it is absent for `openrouter`.

**Novita reports no per-call cost.** Unlike OpenRouter, the chat-completions
response carries token counts and nothing else, so this backend is the OpenAI
one in that respect: `_provider_cost` is the base class's `None`, and
`cost_ceiling_usd` counts calls in `SpendMeter.unpriced_calls` unless a caller
supplies `price_per_million=`.
"""

from __future__ import annotations

from typing import Any

from grapharc.gateway.config import novita_api_key
from grapharc.gateway.openai_compat import OpenAICompatChatModel

NOVITA_BASE_URL = "https://api.novita.ai/openai"


class NovitaError(Exception):
 """The Novita backend could not be constructed or used."""


class NovitaChatModel(OpenAICompatChatModel):
 """A LangChain chat model over Novita's OpenAI-compatible endpoint."""

 def __init__(self, model: str, /, **kwargs: Any) -> None:
 api_key = kwargs.pop("api_key", None) or novita_api_key()
 if not api_key:
 raise NovitaError(
 "No Novita API key found. Set NOVITA_API_KEY in the environment, "
 "or add one of NOVITA_API_KEY / novita-api-key to a .env file."
 )
 # One retry layer, not two — same reasoning as the OpenRouter backend.
 kwargs.setdefault("max_retries", 0)
 super().__init__(
 model=model,
 api_key=api_key,
 base_url=kwargs.pop("base_url", None) or NOVITA_BASE_URL,
 **kwargs,
 )

 @property
 def _llm_type(self) -> str:
 return "grapharc-novita"


__all__ = ["NOVITA_BASE_URL", "NovitaChatModel", "NovitaError"]

15 changes: 14 additions & 1 deletion
grapharc/gateway/registry.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,7 @@
 openrouter/anthropic/claude-sonnet-4.5 -> OpenRouter
 openrouter/openai/gpt-4o:floor -> OpenRouter, cheapest provider
 openai/gpt-4o-mini -> OpenAI directly (OPENAI_API_KEY)
 novita/moonshotai/kimi-k3 -> Novita (NOVITA_API_KEY)
 ollama/llama3.1 -> a local Ollama server, no key
 mock/anything -> scripted test double

Expand All@@ -28,7 +29,7 @@ class UnknownBackendError(Exception):
 """The spec named a backend that is not registered."""


BACKENDS = ("claude-cli", "openrouter", "openai", "ollama", "mock")
BACKENDS = ("claude-cli", "openrouter", "openai", "novita", "ollama", "mock")

# Authors that appear in OpenRouter model ids. A spec starting with one of
# these is a model name, not a mistyped backend — `anthropic/claude-haiku-4.5`
Expand DownExpand Up@@ -71,6 +72,7 @@ class UnknownBackendError(Exception):
_BARE_BACKEND_EXAMPLE = {
 "openrouter": "openrouter/anthropic/claude-sonnet-4.5",
 "openai": "openai/gpt-4o-mini",
 "novita": "novita/moonshotai/kimi-k3",
 "ollama": "ollama/llama3.1",
}

Expand DownExpand Up@@ -160,6 +162,17 @@ def get_model(spec: str, **kwargs: Any) -> BaseChatModel:

 return OpenAIChatModel(model, **kwargs)

 if backend == "novita":
 try:
 from grapharc.gateway.novita import NovitaChatModel
 except ImportError as exc: # pragma: no cover - depends on install extras
 raise UnknownBackendError(
 "The Novita backend needs langchain-openai. "
 "Install it with: uv sync --extra novita"
 ) from exc

 return NovitaChatModel(model, **kwargs)

 if backend == "ollama":
 try:
 from grapharc.gateway.ollama import OllamaChatModel
Expand Down

8 changes: 7 additions & 1 deletion
pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -71,6 +71,12 @@ openrouter = [
openai = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/novita.py. Novita speaks the OpenAI wire format
# against its own endpoint, so this needs the same client and no Novita-specific
# package.
novita = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/ollama.py. Ollama speaks the OpenAI wire format,
# so the local backend needs the same client and no ollama-specific package.
ollama = [
Expand DownExpand Up@@ -110,7 +116,7 @@ slack = [
]
# Everything above. Self-referential so it cannot drift out of sync.
all = [
 "grapharc[api,ladybug,mcp,ollama,openai,openrouter,otel,server,slack]",
 "grapharc[api,ladybug,mcp,novita,ollama,openai,openrouter,otel,server,slack]",
]

[dependency-groups]
Expand Down

4 changes: 3 additions & 1 deletion
tests/test_cli.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,7 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 for name in (
 "OPENROUTER_API_KEY", "OPENROUTER_KEY", "open-router-api-key",
 "OPENAI_API_KEY", "OPENAI_KEY", "openai-api-key",
 "NOVITA_API_KEY", "NOVITA_KEY", "novita-api-key",
 "OLLAMA_HOST", "OLLAMA_BASE_URL",
 ):
 monkeypatch.delenv(name, raising=False)
Expand All@@ -452,10 +453,11 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 "claude-cli": False,
 "openrouter": False,
 "openai": False,
 "novita": False,
 "ollama": False,
 "mock": True,
 }
 for backend in ("openrouter", "openai"):
 for backend in ("openrouter", "openai", "novita"):
 assert next(b for b in payload["backends"] if b["backend"] == backend)[
 "credential"
 ] == "<unset>"
Expand Down
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Add Novita as a model gateway backend#102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Shashankss1205
merged 3 commits into

from

Aug 13, 2026
 Merged


Add Novita as a model gateway backend
#102
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
7f98eaf

Add Novita as a model gateway backend


jax-novita Aug 5, 2026
8f177da

Recommend Novita's current flagship models


jax-novita Aug 5, 2026
2afecff

Fix column alignment for novita example in models cookbook


jax-novita Aug 10, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 12 additions & 5 deletions
docs/cookbook/02-models.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,7 @@ for spec in (
 "claude-cli/claude-sonnet-5",
 "openrouter/anthropic/claude-haiku-4.5",
 "openai/gpt-4o-mini",
 "novita/moonshotai/kimi-k3",
 "ollama/llama3.1",
 "mock/anything",
):
Expand All@@ -38,18 +39,20 @@ print(model.invoke("say hi").content)
{'spec': 'claude-cli/claude-sonnet-5', 'backend': 'claude-cli', 'model': 'claude-sonnet-5'}
{'spec': 'openrouter/anthropic/claude-haiku-4.5', 'backend': 'openrouter', 'model': 'anthropic/claude-haiku-4.5'}
{'spec': 'openai/gpt-4o-mini', 'backend': 'openai', 'model': 'gpt-4o-mini'}
{'spec': 'novita/moonshotai/kimi-k3', 'backend': 'novita', 'model': 'moonshotai/kimi-k3'}
{'spec': 'ollama/llama3.1', 'backend': 'ollama', 'model': 'llama3.1'}
{'spec': 'mock/anything', 'backend': 'mock', 'model': 'anything'}
hello from a scripted model
```

There are five backends:
There are six backends:

| backend | credential | what it is for |
| --- | --- | --- |
| `claude-cli` | a Claude subscription, no API key | text completion on quota you already pay for |
| `openrouter` | `OPENROUTER_API_KEY` | one key, most vendors, per-call cost in the response |
| `openai` | `OPENAI_API_KEY` | the OpenAI API directly, or any endpoint via `OPENAI_BASE_URL` |
| `novita` | `NOVITA_API_KEY` | Novita's own endpoint, token counts but no per-call cost |
| `ollama` | none — a local server | models on your own machine, free and offline |
| `mock` | none | a scripted test double |

Expand All@@ -63,8 +66,8 @@ needs it. Asking for `claude-cli` therefore does not require `langchain-openai`,
for `openrouter` does not require the Claude CLI to be installed. A missing optional
dependency fails for the backend that wanted it and nothing else.

`openrouter`, `openai`and `ollama` all speak the OpenAI wire format and share one base
class, so they behave identically on everything except money and routing: same
`openrouter`, `openai`, `novita` and `ollama` all speak the OpenAI wire format and share one
base class, so they behave identically on everything except money and routing: same
`bind_tools`, same `with_structured_output`, same streaming and async, same retry policy,
same usage envelope.

Expand DownExpand Up@@ -93,7 +96,7 @@ except UnknownBackendError as exc:
('openrouter', 'openai/gpt-4o-mini:floor')
('claude-cli', 'anthropic/claude-haiku-4.5')
('openai', 'gpt-4o-mini')
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, ollama, mock — or a bare model name for the claude-cli default
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, novita, ollama, mock — or a bare model name for the claude-cli default
```

Only the first segment is a backend, because OpenRouter model ids are themselves
Expand DownExpand Up@@ -122,16 +125,18 @@ model through the broker is still `openrouter/openai/gpt-4o-mini`.
<!-- verified: cli -->
```console
$ grapharc models
backends: claude-cli, openrouter, openai, ollama, mock
backends: claude-cli, openrouter, openai, novita, ollama, mock
openrouter key: <unset>
openai key: <unset>
novita key: <unset>
ollama url: http://localhost:11434/v1

examples:
 claude-cli/claude-sonnet-5 subscription, no API key
 openrouter/anthropic/claude-haiku-4.5 many providers, one key
 openrouter/openai/gpt-4o-mini:floor cheapest provider for that model
 openai/gpt-4o-mini the OpenAI API directly, your key
 novita/moonshotai/kimi-k3 Novita's own endpoint, your key
 ollama/llama3.1 a local server, no key and no bill

grapharc models --check probes which of these this machine can use
Expand DownExpand Up@@ -165,6 +170,8 @@ openrouter unusable no API key (set OPENROUTER_API_KEY, or add one to .env)
 credential: <unset>
openai unusable no API key (set OPENAI_API_KEY, or add one to .env)
 credential: <unset>
novita unusable no API key (set NOVITA_API_KEY, or add one to .env)
 credential: <unset>
ollama usable local server at http://localhost:11434/v1
 credential: none needed (local server)
mock usable scripted test double; never reaches a provider
Expand Down

4 changes: 4 additions & 0 deletions
grapharc/cli/main.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -449,6 +449,7 @@ def _cmd_approve(args: argparse.Namespace) -> int:
def _cmd_models(args: argparse.Namespace) -> int:
 from grapharc.gateway import (
 describe,
 novita_api_key,
 ollama_base_url,
 openai_api_key,
 openrouter_api_key,
Expand DownExpand Up@@ -497,6 +498,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "openrouter/anthropic/claude-haiku-4.5": "many providers, one key",
 "openrouter/openai/gpt-4o-mini:floor": "cheapest provider for that model",
 "openai/gpt-4o-mini": "the OpenAI API directly, your key",
 "novita/moonshotai/kimi-k3": "Novita's own endpoint, your key",
 "ollama/llama3.1": "a local server, no key and no bill",
 }
 payload = {
Expand All@@ -505,6 +507,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "backends": list(BACKENDS),
 "openrouter_key": redact(openrouter_api_key()),
 "openai_key": redact(openai_api_key()),
 "novita_key": redact(novita_api_key()),
 # An address, not a secret: it is printed whole, and it is where a
 # request would go rather than proof that anything is listening.
 "ollama_base_url": ollama_base_url(),
Expand All@@ -516,6 +519,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 style.kv("backends", ", ".join(BACKENDS)),
 style.kv("openrouter key", redact(openrouter_api_key())),
 style.kv("openai key", redact(openai_api_key())),
 style.kv("novita key", redact(novita_api_key())),
 style.kv("ollama url", ollama_base_url(), tint=style.accent),
 "",
 style.heading("examples:"),
Expand Down

21 changes: 21 additions & 0 deletions
grapharc/cli/probe.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -95,6 +95,26 @@ def _probe_openai() -> dict[str, Any]:
 }


def _probe_novita() -> dict[str, Any]:
 from grapharc.gateway import novita_api_key, redact

 key = novita_api_key()
 has_dependency = importlib.util.find_spec("langchain_openai") is not None
 missing = []
 if not key:
 missing.append("no API key (set NOVITA_API_KEY, or add one to .env)")
 if not has_dependency:
 missing.append("langchain-openai not installed (uv sync --extra novita)")
 return {
 "backend": "novita",
 "kind": KIND_PROVIDER,
 "usable": bool(key) and has_dependency,
 "credential": redact(key),
 "detail": "; ".join(missing) or "api key configured and langchain-openai installed",
 "checked": "credential presence only; no request was sent to api.novita.ai",
 }


def _probe_ollama() -> dict[str, Any]:
 from grapharc.gateway import ollama_base_url

Expand DownExpand Up@@ -152,6 +172,7 @@ def probe_backends(*, claude_path: str = "claude") -> list[dict[str, Any]]:
 "claude-cli": lambda: _probe_claude_cli(claude_path),
 "openrouter": _probe_openrouter,
 "openai": _probe_openai,
 "novita": _probe_novita,
 "ollama": _probe_ollama,
 "mock": _probe_mock,
 }
Expand Down

9 changes: 7 additions & 2 deletions
grapharc/gateway/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6,6 +6,7 @@
 get_model("claude-cli/claude-sonnet-5") # subscription, no API key
 get_model("openrouter/anthropic/claude-sonnet-4.5") # many providers, one key
 get_model("openai/gpt-4o-mini") # OPENAI_API_KEY
 get_model("novita/moonshotai/kimi-k3") # NOVITA_API_KEY
 get_model("ollama/llama3.1") # local server, no key
 get_model("mock/x", responses=[...]) # deterministic tests

Expand All@@ -16,12 +17,13 @@
 get_model(spec, cost_ceiling_usd=0.25) # raises when passed
 get_model(spec, spend=shared_meter) # one ceiling, many models

The three OpenAI-wire backends (`openrouter`, `openai`, `ollama`) are imported
lazily — they need `langchain-openai`, which is an optional extra.
The four OpenAI-wire backends (`openrouter`, `openai`, `novita`, `ollama`) are
imported lazily — they need `langchain-openai`, which is an optional extra.
"""

from grapharc.gateway.claude_cli import ClaudeCodeCLIChatModel
from grapharc.gateway.config import (
 novita_api_key,
 ollama_api_key,
 ollama_base_url,
 openai_api_key,
Expand DownExpand Up@@ -66,6 +68,7 @@
 "different_providers",
 "get_model",
 "is_transient",
 "novita_api_key",
 "ollama_api_key",
 "ollama_base_url",
 "openai_api_key",
Expand All@@ -85,6 +88,8 @@
 "OpenRouterError": "openrouter",
 "OpenAIChatModel": "openai",
 "OpenAIError": "openai",
 "NovitaChatModel": "novita",
 "NovitaError": "novita",
 "OllamaChatModel": "ollama",
 "OllamaError": "ollama",
}
Expand Down

16 changes: 15 additions & 1 deletion
grapharc/gateway/config.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -20,14 +20,17 @@
Secrets are returned, never logged. Anything that renders a config for humans
goes through `redact`.

Three key-holding backends, plus one that usually holds none:
Four key-holding backends, plus one that usually holds none:

- **OpenRouter** — `OPENROUTER_API_KEY`.
- **OpenAI** — `OPENAI_API_KEY`, optionally with `OPENAI_BASE_URL` for an
 Azure-style or proxied endpoint. `langchain-openai` reads `OPENAI_API_KEY`
 from the process environment on its own; going through here as well is what
 adds `.env` support, the alternate spellings, and a failure that names the
 variable instead of surfacing an SDK error.
- **Novita** — `NOVITA_API_KEY`. The endpoint is fixed
 (`grapharc.gateway.novita.NOVITA_BASE_URL`), so unlike OpenAI there is no
 base-url override to resolve here.
- **Ollama** — no credential by default: it is a server on your own machine.
 What it needs is an address, so `ollama_base_url()` always returns one
 (`OLLAMA_HOST` / `OLLAMA_BASE_URL`, else localhost). `OLLAMA_API_KEY` exists
Expand DownExpand Up@@ -56,6 +59,13 @@
 "openai_api_key",
)

NOVITA_KEYS = (
 "NOVITA_API_KEY",
 "NOVITA_KEY",
 "novita-api-key",
 "novita_api_key",
)

# `OPENAI_API_BASE` is the older spelling and is still what a lot of tooling
# sets; both are accepted, the current one first.
OPENAI_BASE_URL_KEYS = (
Expand DownExpand Up@@ -133,6 +143,10 @@ def openai_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(OPENAI_KEYS, env_file=env_file)


def novita_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(NOVITA_KEYS, env_file=env_file)


def openai_base_url(*, env_file: Path | None = None) -> str | None:
 """An endpoint override, or None for api.openai.com.

Expand Down

59 changes: 59 additions & 0 deletions
grapharc/gateway/novita.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
"""Novita backend — a GPU cloud hosting open-weight models, one key.

`novita/moonshotai/kimi-k3` reaches Novita's own OpenAI-compatible endpoint
(`https://api.novita.ai/openai`), not api.openai.com, so this builds on
`OpenAICompatChatModel` the same way `openrouter.py` and `ollama.py` do rather
than on `openai.py`: the endpoint is fixed, not an override of OpenAI's own.

Model ids on Novita are themselves `author/slug` — `moonshotai/kimi-k3`,
`zai-org/glm-5.2` — the same shape OpenRouter uses, so `vendor()` in
`registry.py` already reads the right author off a Novita spec with no
backend-specific handling: `BACKEND_VENDOR` stays absent for `novita`, exactly
as it is absent for `openrouter`.

**Novita reports no per-call cost.** Unlike OpenRouter, the chat-completions
response carries token counts and nothing else, so this backend is the OpenAI
one in that respect: `_provider_cost` is the base class's `None`, and
`cost_ceiling_usd` counts calls in `SpendMeter.unpriced_calls` unless a caller
supplies `price_per_million=`.
"""

from __future__ import annotations

from typing import Any

from grapharc.gateway.config import novita_api_key
from grapharc.gateway.openai_compat import OpenAICompatChatModel

NOVITA_BASE_URL = "https://api.novita.ai/openai"


class NovitaError(Exception):
 """The Novita backend could not be constructed or used."""


class NovitaChatModel(OpenAICompatChatModel):
 """A LangChain chat model over Novita's OpenAI-compatible endpoint."""

 def __init__(self, model: str, /, **kwargs: Any) -> None:
 api_key = kwargs.pop("api_key", None) or novita_api_key()
 if not api_key:
 raise NovitaError(
 "No Novita API key found. Set NOVITA_API_KEY in the environment, "
 "or add one of NOVITA_API_KEY / novita-api-key to a .env file."
 )
 # One retry layer, not two — same reasoning as the OpenRouter backend.
 kwargs.setdefault("max_retries", 0)
 super().__init__(
 model=model,
 api_key=api_key,
 base_url=kwargs.pop("base_url", None) or NOVITA_BASE_URL,
 **kwargs,
 )

 @property
 def _llm_type(self) -> str:
 return "grapharc-novita"


__all__ = ["NOVITA_BASE_URL", "NovitaChatModel", "NovitaError"]

15 changes: 14 additions & 1 deletion
grapharc/gateway/registry.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,7 @@
 openrouter/anthropic/claude-sonnet-4.5 -> OpenRouter
 openrouter/openai/gpt-4o:floor -> OpenRouter, cheapest provider
 openai/gpt-4o-mini -> OpenAI directly (OPENAI_API_KEY)
 novita/moonshotai/kimi-k3 -> Novita (NOVITA_API_KEY)
 ollama/llama3.1 -> a local Ollama server, no key
 mock/anything -> scripted test double

Expand All@@ -28,7 +29,7 @@ class UnknownBackendError(Exception):
 """The spec named a backend that is not registered."""


BACKENDS = ("claude-cli", "openrouter", "openai", "ollama", "mock")
BACKENDS = ("claude-cli", "openrouter", "openai", "novita", "ollama", "mock")

# Authors that appear in OpenRouter model ids. A spec starting with one of
# these is a model name, not a mistyped backend — `anthropic/claude-haiku-4.5`
Expand DownExpand Up@@ -71,6 +72,7 @@ class UnknownBackendError(Exception):
_BARE_BACKEND_EXAMPLE = {
 "openrouter": "openrouter/anthropic/claude-sonnet-4.5",
 "openai": "openai/gpt-4o-mini",
 "novita": "novita/moonshotai/kimi-k3",
 "ollama": "ollama/llama3.1",
}

Expand DownExpand Up@@ -160,6 +162,17 @@ def get_model(spec: str, **kwargs: Any) -> BaseChatModel:

 return OpenAIChatModel(model, **kwargs)

 if backend == "novita":
 try:
 from grapharc.gateway.novita import NovitaChatModel
 except ImportError as exc: # pragma: no cover - depends on install extras
 raise UnknownBackendError(
 "The Novita backend needs langchain-openai. "
 "Install it with: uv sync --extra novita"
 ) from exc

 return NovitaChatModel(model, **kwargs)

 if backend == "ollama":
 try:
 from grapharc.gateway.ollama import OllamaChatModel
Expand Down

8 changes: 7 additions & 1 deletion
pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -71,6 +71,12 @@ openrouter = [
openai = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/novita.py. Novita speaks the OpenAI wire format
# against its own endpoint, so this needs the same client and no Novita-specific
# package.
novita = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/ollama.py. Ollama speaks the OpenAI wire format,
# so the local backend needs the same client and no ollama-specific package.
ollama = [
Expand DownExpand Up@@ -110,7 +116,7 @@ slack = [
]
# Everything above. Self-referential so it cannot drift out of sync.
all = [
 "grapharc[api,ladybug,mcp,ollama,openai,openrouter,otel,server,slack]",
 "grapharc[api,ladybug,mcp,novita,ollama,openai,openrouter,otel,server,slack]",
]

[dependency-groups]
Expand Down

4 changes: 3 additions & 1 deletion
tests/test_cli.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,7 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 for name in (
 "OPENROUTER_API_KEY", "OPENROUTER_KEY", "open-router-api-key",
 "OPENAI_API_KEY", "OPENAI_KEY", "openai-api-key",
 "NOVITA_API_KEY", "NOVITA_KEY", "novita-api-key",
 "OLLAMA_HOST", "OLLAMA_BASE_URL",
 ):
 monkeypatch.delenv(name, raising=False)
Expand All@@ -452,10 +453,11 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 "claude-cli": False,
 "openrouter": False,
 "openai": False,
 "novita": False,
 "ollama": False,
 "mock": True,
 }
 for backend in ("openrouter", "openai"):
 for backend in ("openrouter", "openai", "novita"):
 assert next(b for b in payload["backends"] if b["backend"] == backend)[
 "credential"
 ] == "<unset>"
Expand Down
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Add Novita as a model gateway backend#102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Shashankss1205
merged 3 commits into

from

Aug 13, 2026
 Merged


Add Novita as a model gateway backend
#102
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
7f98eaf

Add Novita as a model gateway backend


jax-novita Aug 5, 2026
8f177da

Recommend Novita's current flagship models


jax-novita Aug 5, 2026
2afecff

Fix column alignment for novita example in models cookbook


jax-novita Aug 10, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 12 additions & 5 deletions
docs/cookbook/02-models.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,7 @@ for spec in (
 "claude-cli/claude-sonnet-5",
 "openrouter/anthropic/claude-haiku-4.5",
 "openai/gpt-4o-mini",
 "novita/moonshotai/kimi-k3",
 "ollama/llama3.1",
 "mock/anything",
):
Expand All@@ -38,18 +39,20 @@ print(model.invoke("say hi").content)
{'spec': 'claude-cli/claude-sonnet-5', 'backend': 'claude-cli', 'model': 'claude-sonnet-5'}
{'spec': 'openrouter/anthropic/claude-haiku-4.5', 'backend': 'openrouter', 'model': 'anthropic/claude-haiku-4.5'}
{'spec': 'openai/gpt-4o-mini', 'backend': 'openai', 'model': 'gpt-4o-mini'}
{'spec': 'novita/moonshotai/kimi-k3', 'backend': 'novita', 'model': 'moonshotai/kimi-k3'}
{'spec': 'ollama/llama3.1', 'backend': 'ollama', 'model': 'llama3.1'}
{'spec': 'mock/anything', 'backend': 'mock', 'model': 'anything'}
hello from a scripted model
```

There are five backends:
There are six backends:

| backend | credential | what it is for |
| --- | --- | --- |
| `claude-cli` | a Claude subscription, no API key | text completion on quota you already pay for |
| `openrouter` | `OPENROUTER_API_KEY` | one key, most vendors, per-call cost in the response |
| `openai` | `OPENAI_API_KEY` | the OpenAI API directly, or any endpoint via `OPENAI_BASE_URL` |
| `novita` | `NOVITA_API_KEY` | Novita's own endpoint, token counts but no per-call cost |
| `ollama` | none — a local server | models on your own machine, free and offline |
| `mock` | none | a scripted test double |

Expand All@@ -63,8 +66,8 @@ needs it. Asking for `claude-cli` therefore does not require `langchain-openai`,
for `openrouter` does not require the Claude CLI to be installed. A missing optional
dependency fails for the backend that wanted it and nothing else.

`openrouter`, `openai`and `ollama` all speak the OpenAI wire format and share one base
class, so they behave identically on everything except money and routing: same
`openrouter`, `openai`, `novita` and `ollama` all speak the OpenAI wire format and share one
base class, so they behave identically on everything except money and routing: same
`bind_tools`, same `with_structured_output`, same streaming and async, same retry policy,
same usage envelope.

Expand DownExpand Up@@ -93,7 +96,7 @@ except UnknownBackendError as exc:
('openrouter', 'openai/gpt-4o-mini:floor')
('claude-cli', 'anthropic/claude-haiku-4.5')
('openai', 'gpt-4o-mini')
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, ollama, mock — or a bare model name for the claude-cli default
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, novita, ollama, mock — or a bare model name for the claude-cli default
```

Only the first segment is a backend, because OpenRouter model ids are themselves
Expand DownExpand Up@@ -122,16 +125,18 @@ model through the broker is still `openrouter/openai/gpt-4o-mini`.
<!-- verified: cli -->
```console
$ grapharc models
backends: claude-cli, openrouter, openai, ollama, mock
backends: claude-cli, openrouter, openai, novita, ollama, mock
openrouter key: <unset>
openai key: <unset>
novita key: <unset>
ollama url: http://localhost:11434/v1

examples:
 claude-cli/claude-sonnet-5 subscription, no API key
 openrouter/anthropic/claude-haiku-4.5 many providers, one key
 openrouter/openai/gpt-4o-mini:floor cheapest provider for that model
 openai/gpt-4o-mini the OpenAI API directly, your key
 novita/moonshotai/kimi-k3 Novita's own endpoint, your key
 ollama/llama3.1 a local server, no key and no bill

grapharc models --check probes which of these this machine can use
Expand DownExpand Up@@ -165,6 +170,8 @@ openrouter unusable no API key (set OPENROUTER_API_KEY, or add one to .env)
 credential: <unset>
openai unusable no API key (set OPENAI_API_KEY, or add one to .env)
 credential: <unset>
novita unusable no API key (set NOVITA_API_KEY, or add one to .env)
 credential: <unset>
ollama usable local server at http://localhost:11434/v1
 credential: none needed (local server)
mock usable scripted test double; never reaches a provider
Expand Down

4 changes: 4 additions & 0 deletions
grapharc/cli/main.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -449,6 +449,7 @@ def _cmd_approve(args: argparse.Namespace) -> int:
def _cmd_models(args: argparse.Namespace) -> int:
 from grapharc.gateway import (
 describe,
 novita_api_key,
 ollama_base_url,
 openai_api_key,
 openrouter_api_key,
Expand DownExpand Up@@ -497,6 +498,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "openrouter/anthropic/claude-haiku-4.5": "many providers, one key",
 "openrouter/openai/gpt-4o-mini:floor": "cheapest provider for that model",
 "openai/gpt-4o-mini": "the OpenAI API directly, your key",
 "novita/moonshotai/kimi-k3": "Novita's own endpoint, your key",
 "ollama/llama3.1": "a local server, no key and no bill",
 }
 payload = {
Expand All@@ -505,6 +507,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "backends": list(BACKENDS),
 "openrouter_key": redact(openrouter_api_key()),
 "openai_key": redact(openai_api_key()),
 "novita_key": redact(novita_api_key()),
 # An address, not a secret: it is printed whole, and it is where a
 # request would go rather than proof that anything is listening.
 "ollama_base_url": ollama_base_url(),
Expand All@@ -516,6 +519,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 style.kv("backends", ", ".join(BACKENDS)),
 style.kv("openrouter key", redact(openrouter_api_key())),
 style.kv("openai key", redact(openai_api_key())),
 style.kv("novita key", redact(novita_api_key())),
 style.kv("ollama url", ollama_base_url(), tint=style.accent),
 "",
 style.heading("examples:"),
Expand Down

21 changes: 21 additions & 0 deletions
grapharc/cli/probe.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -95,6 +95,26 @@ def _probe_openai() -> dict[str, Any]:
 }


def _probe_novita() -> dict[str, Any]:
 from grapharc.gateway import novita_api_key, redact

 key = novita_api_key()
 has_dependency = importlib.util.find_spec("langchain_openai") is not None
 missing = []
 if not key:
 missing.append("no API key (set NOVITA_API_KEY, or add one to .env)")
 if not has_dependency:
 missing.append("langchain-openai not installed (uv sync --extra novita)")
 return {
 "backend": "novita",
 "kind": KIND_PROVIDER,
 "usable": bool(key) and has_dependency,
 "credential": redact(key),
 "detail": "; ".join(missing) or "api key configured and langchain-openai installed",
 "checked": "credential presence only; no request was sent to api.novita.ai",
 }


def _probe_ollama() -> dict[str, Any]:
 from grapharc.gateway import ollama_base_url

Expand DownExpand Up@@ -152,6 +172,7 @@ def probe_backends(*, claude_path: str = "claude") -> list[dict[str, Any]]:
 "claude-cli": lambda: _probe_claude_cli(claude_path),
 "openrouter": _probe_openrouter,
 "openai": _probe_openai,
 "novita": _probe_novita,
 "ollama": _probe_ollama,
 "mock": _probe_mock,
 }
Expand Down

9 changes: 7 additions & 2 deletions
grapharc/gateway/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6,6 +6,7 @@
 get_model("claude-cli/claude-sonnet-5") # subscription, no API key
 get_model("openrouter/anthropic/claude-sonnet-4.5") # many providers, one key
 get_model("openai/gpt-4o-mini") # OPENAI_API_KEY
 get_model("novita/moonshotai/kimi-k3") # NOVITA_API_KEY
 get_model("ollama/llama3.1") # local server, no key
 get_model("mock/x", responses=[...]) # deterministic tests

Expand All@@ -16,12 +17,13 @@
 get_model(spec, cost_ceiling_usd=0.25) # raises when passed
 get_model(spec, spend=shared_meter) # one ceiling, many models

The three OpenAI-wire backends (`openrouter`, `openai`, `ollama`) are imported
lazily — they need `langchain-openai`, which is an optional extra.
The four OpenAI-wire backends (`openrouter`, `openai`, `novita`, `ollama`) are
imported lazily — they need `langchain-openai`, which is an optional extra.
"""

from grapharc.gateway.claude_cli import ClaudeCodeCLIChatModel
from grapharc.gateway.config import (
 novita_api_key,
 ollama_api_key,
 ollama_base_url,
 openai_api_key,
Expand DownExpand Up@@ -66,6 +68,7 @@
 "different_providers",
 "get_model",
 "is_transient",
 "novita_api_key",
 "ollama_api_key",
 "ollama_base_url",
 "openai_api_key",
Expand All@@ -85,6 +88,8 @@
 "OpenRouterError": "openrouter",
 "OpenAIChatModel": "openai",
 "OpenAIError": "openai",
 "NovitaChatModel": "novita",
 "NovitaError": "novita",
 "OllamaChatModel": "ollama",
 "OllamaError": "ollama",
}
Expand Down

16 changes: 15 additions & 1 deletion
grapharc/gateway/config.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -20,14 +20,17 @@
Secrets are returned, never logged. Anything that renders a config for humans
goes through `redact`.

Three key-holding backends, plus one that usually holds none:
Four key-holding backends, plus one that usually holds none:

- **OpenRouter** — `OPENROUTER_API_KEY`.
- **OpenAI** — `OPENAI_API_KEY`, optionally with `OPENAI_BASE_URL` for an
 Azure-style or proxied endpoint. `langchain-openai` reads `OPENAI_API_KEY`
 from the process environment on its own; going through here as well is what
 adds `.env` support, the alternate spellings, and a failure that names the
 variable instead of surfacing an SDK error.
- **Novita** — `NOVITA_API_KEY`. The endpoint is fixed
 (`grapharc.gateway.novita.NOVITA_BASE_URL`), so unlike OpenAI there is no
 base-url override to resolve here.
- **Ollama** — no credential by default: it is a server on your own machine.
 What it needs is an address, so `ollama_base_url()` always returns one
 (`OLLAMA_HOST` / `OLLAMA_BASE_URL`, else localhost). `OLLAMA_API_KEY` exists
Expand DownExpand Up@@ -56,6 +59,13 @@
 "openai_api_key",
)

NOVITA_KEYS = (
 "NOVITA_API_KEY",
 "NOVITA_KEY",
 "novita-api-key",
 "novita_api_key",
)

# `OPENAI_API_BASE` is the older spelling and is still what a lot of tooling
# sets; both are accepted, the current one first.
OPENAI_BASE_URL_KEYS = (
Expand DownExpand Up@@ -133,6 +143,10 @@ def openai_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(OPENAI_KEYS, env_file=env_file)


def novita_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(NOVITA_KEYS, env_file=env_file)


def openai_base_url(*, env_file: Path | None = None) -> str | None:
 """An endpoint override, or None for api.openai.com.

Expand Down

59 changes: 59 additions & 0 deletions
grapharc/gateway/novita.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
"""Novita backend — a GPU cloud hosting open-weight models, one key.

`novita/moonshotai/kimi-k3` reaches Novita's own OpenAI-compatible endpoint
(`https://api.novita.ai/openai`), not api.openai.com, so this builds on
`OpenAICompatChatModel` the same way `openrouter.py` and `ollama.py` do rather
than on `openai.py`: the endpoint is fixed, not an override of OpenAI's own.

Model ids on Novita are themselves `author/slug` — `moonshotai/kimi-k3`,
`zai-org/glm-5.2` — the same shape OpenRouter uses, so `vendor()` in
`registry.py` already reads the right author off a Novita spec with no
backend-specific handling: `BACKEND_VENDOR` stays absent for `novita`, exactly
as it is absent for `openrouter`.

**Novita reports no per-call cost.** Unlike OpenRouter, the chat-completions
response carries token counts and nothing else, so this backend is the OpenAI
one in that respect: `_provider_cost` is the base class's `None`, and
`cost_ceiling_usd` counts calls in `SpendMeter.unpriced_calls` unless a caller
supplies `price_per_million=`.
"""

from __future__ import annotations

from typing import Any

from grapharc.gateway.config import novita_api_key
from grapharc.gateway.openai_compat import OpenAICompatChatModel

NOVITA_BASE_URL = "https://api.novita.ai/openai"


class NovitaError(Exception):
 """The Novita backend could not be constructed or used."""


class NovitaChatModel(OpenAICompatChatModel):
 """A LangChain chat model over Novita's OpenAI-compatible endpoint."""

 def __init__(self, model: str, /, **kwargs: Any) -> None:
 api_key = kwargs.pop("api_key", None) or novita_api_key()
 if not api_key:
 raise NovitaError(
 "No Novita API key found. Set NOVITA_API_KEY in the environment, "
 "or add one of NOVITA_API_KEY / novita-api-key to a .env file."
 )
 # One retry layer, not two — same reasoning as the OpenRouter backend.
 kwargs.setdefault("max_retries", 0)
 super().__init__(
 model=model,
 api_key=api_key,
 base_url=kwargs.pop("base_url", None) or NOVITA_BASE_URL,
 **kwargs,
 )

 @property
 def _llm_type(self) -> str:
 return "grapharc-novita"


__all__ = ["NOVITA_BASE_URL", "NovitaChatModel", "NovitaError"]

15 changes: 14 additions & 1 deletion
grapharc/gateway/registry.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,7 @@
 openrouter/anthropic/claude-sonnet-4.5 -> OpenRouter
 openrouter/openai/gpt-4o:floor -> OpenRouter, cheapest provider
 openai/gpt-4o-mini -> OpenAI directly (OPENAI_API_KEY)
 novita/moonshotai/kimi-k3 -> Novita (NOVITA_API_KEY)
 ollama/llama3.1 -> a local Ollama server, no key
 mock/anything -> scripted test double

Expand All@@ -28,7 +29,7 @@ class UnknownBackendError(Exception):
 """The spec named a backend that is not registered."""


BACKENDS = ("claude-cli", "openrouter", "openai", "ollama", "mock")
BACKENDS = ("claude-cli", "openrouter", "openai", "novita", "ollama", "mock")

# Authors that appear in OpenRouter model ids. A spec starting with one of
# these is a model name, not a mistyped backend — `anthropic/claude-haiku-4.5`
Expand DownExpand Up@@ -71,6 +72,7 @@ class UnknownBackendError(Exception):
_BARE_BACKEND_EXAMPLE = {
 "openrouter": "openrouter/anthropic/claude-sonnet-4.5",
 "openai": "openai/gpt-4o-mini",
 "novita": "novita/moonshotai/kimi-k3",
 "ollama": "ollama/llama3.1",
}

Expand DownExpand Up@@ -160,6 +162,17 @@ def get_model(spec: str, **kwargs: Any) -> BaseChatModel:

 return OpenAIChatModel(model, **kwargs)

 if backend == "novita":
 try:
 from grapharc.gateway.novita import NovitaChatModel
 except ImportError as exc: # pragma: no cover - depends on install extras
 raise UnknownBackendError(
 "The Novita backend needs langchain-openai. "
 "Install it with: uv sync --extra novita"
 ) from exc

 return NovitaChatModel(model, **kwargs)

 if backend == "ollama":
 try:
 from grapharc.gateway.ollama import OllamaChatModel
Expand Down

8 changes: 7 additions & 1 deletion
pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -71,6 +71,12 @@ openrouter = [
openai = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/novita.py. Novita speaks the OpenAI wire format
# against its own endpoint, so this needs the same client and no Novita-specific
# package.
novita = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/ollama.py. Ollama speaks the OpenAI wire format,
# so the local backend needs the same client and no ollama-specific package.
ollama = [
Expand DownExpand Up@@ -110,7 +116,7 @@ slack = [
]
# Everything above. Self-referential so it cannot drift out of sync.
all = [
 "grapharc[api,ladybug,mcp,ollama,openai,openrouter,otel,server,slack]",
 "grapharc[api,ladybug,mcp,novita,ollama,openai,openrouter,otel,server,slack]",
]

[dependency-groups]
Expand Down

4 changes: 3 additions & 1 deletion
tests/test_cli.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,7 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 for name in (
 "OPENROUTER_API_KEY", "OPENROUTER_KEY", "open-router-api-key",
 "OPENAI_API_KEY", "OPENAI_KEY", "openai-api-key",
 "NOVITA_API_KEY", "NOVITA_KEY", "novita-api-key",
 "OLLAMA_HOST", "OLLAMA_BASE_URL",
 ):
 monkeypatch.delenv(name, raising=False)
Expand All@@ -452,10 +453,11 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 "claude-cli": False,
 "openrouter": False,
 "openai": False,
 "novita": False,
 "ollama": False,
 "mock": True,
 }
 for backend in ("openrouter", "openai"):
 for backend in ("openrouter", "openai", "novita"):
 assert next(b for b in payload["backends"] if b["backend"] == backend)[
 "credential"
 ] == "<unset>"
Expand Down
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Add Novita as a model gateway backend#102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Shashankss1205
merged 3 commits into

from

Aug 13, 2026
 Merged


Add Novita as a model gateway backend
#102
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
7f98eaf

Add Novita as a model gateway backend


jax-novita Aug 5, 2026
8f177da

Recommend Novita's current flagship models


jax-novita Aug 5, 2026
2afecff

Fix column alignment for novita example in models cookbook


jax-novita Aug 10, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 12 additions & 5 deletions
docs/cookbook/02-models.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,7 @@ for spec in (
 "claude-cli/claude-sonnet-5",
 "openrouter/anthropic/claude-haiku-4.5",
 "openai/gpt-4o-mini",
 "novita/moonshotai/kimi-k3",
 "ollama/llama3.1",
 "mock/anything",
):
Expand All@@ -38,18 +39,20 @@ print(model.invoke("say hi").content)
{'spec': 'claude-cli/claude-sonnet-5', 'backend': 'claude-cli', 'model': 'claude-sonnet-5'}
{'spec': 'openrouter/anthropic/claude-haiku-4.5', 'backend': 'openrouter', 'model': 'anthropic/claude-haiku-4.5'}
{'spec': 'openai/gpt-4o-mini', 'backend': 'openai', 'model': 'gpt-4o-mini'}
{'spec': 'novita/moonshotai/kimi-k3', 'backend': 'novita', 'model': 'moonshotai/kimi-k3'}
{'spec': 'ollama/llama3.1', 'backend': 'ollama', 'model': 'llama3.1'}
{'spec': 'mock/anything', 'backend': 'mock', 'model': 'anything'}
hello from a scripted model
```

There are five backends:
There are six backends:

| backend | credential | what it is for |
| --- | --- | --- |
| `claude-cli` | a Claude subscription, no API key | text completion on quota you already pay for |
| `openrouter` | `OPENROUTER_API_KEY` | one key, most vendors, per-call cost in the response |
| `openai` | `OPENAI_API_KEY` | the OpenAI API directly, or any endpoint via `OPENAI_BASE_URL` |
| `novita` | `NOVITA_API_KEY` | Novita's own endpoint, token counts but no per-call cost |
| `ollama` | none — a local server | models on your own machine, free and offline |
| `mock` | none | a scripted test double |

Expand All@@ -63,8 +66,8 @@ needs it. Asking for `claude-cli` therefore does not require `langchain-openai`,
for `openrouter` does not require the Claude CLI to be installed. A missing optional
dependency fails for the backend that wanted it and nothing else.

`openrouter`, `openai`and `ollama` all speak the OpenAI wire format and share one base
class, so they behave identically on everything except money and routing: same
`openrouter`, `openai`, `novita` and `ollama` all speak the OpenAI wire format and share one
base class, so they behave identically on everything except money and routing: same
`bind_tools`, same `with_structured_output`, same streaming and async, same retry policy,
same usage envelope.

Expand DownExpand Up@@ -93,7 +96,7 @@ except UnknownBackendError as exc:
('openrouter', 'openai/gpt-4o-mini:floor')
('claude-cli', 'anthropic/claude-haiku-4.5')
('openai', 'gpt-4o-mini')
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, ollama, mock — or a bare model name for the claude-cli default
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, novita, ollama, mock — or a bare model name for the claude-cli default
```

Only the first segment is a backend, because OpenRouter model ids are themselves
Expand DownExpand Up@@ -122,16 +125,18 @@ model through the broker is still `openrouter/openai/gpt-4o-mini`.
<!-- verified: cli -->
```console
$ grapharc models
backends: claude-cli, openrouter, openai, ollama, mock
backends: claude-cli, openrouter, openai, novita, ollama, mock
openrouter key: <unset>
openai key: <unset>
novita key: <unset>
ollama url: http://localhost:11434/v1

examples:
 claude-cli/claude-sonnet-5 subscription, no API key
 openrouter/anthropic/claude-haiku-4.5 many providers, one key
 openrouter/openai/gpt-4o-mini:floor cheapest provider for that model
 openai/gpt-4o-mini the OpenAI API directly, your key
 novita/moonshotai/kimi-k3 Novita's own endpoint, your key
 ollama/llama3.1 a local server, no key and no bill

grapharc models --check probes which of these this machine can use
Expand DownExpand Up@@ -165,6 +170,8 @@ openrouter unusable no API key (set OPENROUTER_API_KEY, or add one to .env)
 credential: <unset>
openai unusable no API key (set OPENAI_API_KEY, or add one to .env)
 credential: <unset>
novita unusable no API key (set NOVITA_API_KEY, or add one to .env)
 credential: <unset>
ollama usable local server at http://localhost:11434/v1
 credential: none needed (local server)
mock usable scripted test double; never reaches a provider
Expand Down

4 changes: 4 additions & 0 deletions
grapharc/cli/main.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -449,6 +449,7 @@ def _cmd_approve(args: argparse.Namespace) -> int:
def _cmd_models(args: argparse.Namespace) -> int:
 from grapharc.gateway import (
 describe,
 novita_api_key,
 ollama_base_url,
 openai_api_key,
 openrouter_api_key,
Expand DownExpand Up@@ -497,6 +498,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "openrouter/anthropic/claude-haiku-4.5": "many providers, one key",
 "openrouter/openai/gpt-4o-mini:floor": "cheapest provider for that model",
 "openai/gpt-4o-mini": "the OpenAI API directly, your key",
 "novita/moonshotai/kimi-k3": "Novita's own endpoint, your key",
 "ollama/llama3.1": "a local server, no key and no bill",
 }
 payload = {
Expand All@@ -505,6 +507,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "backends": list(BACKENDS),
 "openrouter_key": redact(openrouter_api_key()),
 "openai_key": redact(openai_api_key()),
 "novita_key": redact(novita_api_key()),
 # An address, not a secret: it is printed whole, and it is where a
 # request would go rather than proof that anything is listening.
 "ollama_base_url": ollama_base_url(),
Expand All@@ -516,6 +519,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 style.kv("backends", ", ".join(BACKENDS)),
 style.kv("openrouter key", redact(openrouter_api_key())),
 style.kv("openai key", redact(openai_api_key())),
 style.kv("novita key", redact(novita_api_key())),
 style.kv("ollama url", ollama_base_url(), tint=style.accent),
 "",
 style.heading("examples:"),
Expand Down

21 changes: 21 additions & 0 deletions
grapharc/cli/probe.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -95,6 +95,26 @@ def _probe_openai() -> dict[str, Any]:
 }


def _probe_novita() -> dict[str, Any]:
 from grapharc.gateway import novita_api_key, redact

 key = novita_api_key()
 has_dependency = importlib.util.find_spec("langchain_openai") is not None
 missing = []
 if not key:
 missing.append("no API key (set NOVITA_API_KEY, or add one to .env)")
 if not has_dependency:
 missing.append("langchain-openai not installed (uv sync --extra novita)")
 return {
 "backend": "novita",
 "kind": KIND_PROVIDER,
 "usable": bool(key) and has_dependency,
 "credential": redact(key),
 "detail": "; ".join(missing) or "api key configured and langchain-openai installed",
 "checked": "credential presence only; no request was sent to api.novita.ai",
 }


def _probe_ollama() -> dict[str, Any]:
 from grapharc.gateway import ollama_base_url

Expand DownExpand Up@@ -152,6 +172,7 @@ def probe_backends(*, claude_path: str = "claude") -> list[dict[str, Any]]:
 "claude-cli": lambda: _probe_claude_cli(claude_path),
 "openrouter": _probe_openrouter,
 "openai": _probe_openai,
 "novita": _probe_novita,
 "ollama": _probe_ollama,
 "mock": _probe_mock,
 }
Expand Down

9 changes: 7 additions & 2 deletions
grapharc/gateway/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6,6 +6,7 @@
 get_model("claude-cli/claude-sonnet-5") # subscription, no API key
 get_model("openrouter/anthropic/claude-sonnet-4.5") # many providers, one key
 get_model("openai/gpt-4o-mini") # OPENAI_API_KEY
 get_model("novita/moonshotai/kimi-k3") # NOVITA_API_KEY
 get_model("ollama/llama3.1") # local server, no key
 get_model("mock/x", responses=[...]) # deterministic tests

Expand All@@ -16,12 +17,13 @@
 get_model(spec, cost_ceiling_usd=0.25) # raises when passed
 get_model(spec, spend=shared_meter) # one ceiling, many models

The three OpenAI-wire backends (`openrouter`, `openai`, `ollama`) are imported
lazily — they need `langchain-openai`, which is an optional extra.
The four OpenAI-wire backends (`openrouter`, `openai`, `novita`, `ollama`) are
imported lazily — they need `langchain-openai`, which is an optional extra.
"""

from grapharc.gateway.claude_cli import ClaudeCodeCLIChatModel
from grapharc.gateway.config import (
 novita_api_key,
 ollama_api_key,
 ollama_base_url,
 openai_api_key,
Expand DownExpand Up@@ -66,6 +68,7 @@
 "different_providers",
 "get_model",
 "is_transient",
 "novita_api_key",
 "ollama_api_key",
 "ollama_base_url",
 "openai_api_key",
Expand All@@ -85,6 +88,8 @@
 "OpenRouterError": "openrouter",
 "OpenAIChatModel": "openai",
 "OpenAIError": "openai",
 "NovitaChatModel": "novita",
 "NovitaError": "novita",
 "OllamaChatModel": "ollama",
 "OllamaError": "ollama",
}
Expand Down

16 changes: 15 additions & 1 deletion
grapharc/gateway/config.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -20,14 +20,17 @@
Secrets are returned, never logged. Anything that renders a config for humans
goes through `redact`.

Three key-holding backends, plus one that usually holds none:
Four key-holding backends, plus one that usually holds none:

- **OpenRouter** — `OPENROUTER_API_KEY`.
- **OpenAI** — `OPENAI_API_KEY`, optionally with `OPENAI_BASE_URL` for an
 Azure-style or proxied endpoint. `langchain-openai` reads `OPENAI_API_KEY`
 from the process environment on its own; going through here as well is what
 adds `.env` support, the alternate spellings, and a failure that names the
 variable instead of surfacing an SDK error.
- **Novita** — `NOVITA_API_KEY`. The endpoint is fixed
 (`grapharc.gateway.novita.NOVITA_BASE_URL`), so unlike OpenAI there is no
 base-url override to resolve here.
- **Ollama** — no credential by default: it is a server on your own machine.
 What it needs is an address, so `ollama_base_url()` always returns one
 (`OLLAMA_HOST` / `OLLAMA_BASE_URL`, else localhost). `OLLAMA_API_KEY` exists
Expand DownExpand Up@@ -56,6 +59,13 @@
 "openai_api_key",
)

NOVITA_KEYS = (
 "NOVITA_API_KEY",
 "NOVITA_KEY",
 "novita-api-key",
 "novita_api_key",
)

# `OPENAI_API_BASE` is the older spelling and is still what a lot of tooling
# sets; both are accepted, the current one first.
OPENAI_BASE_URL_KEYS = (
Expand DownExpand Up@@ -133,6 +143,10 @@ def openai_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(OPENAI_KEYS, env_file=env_file)


def novita_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(NOVITA_KEYS, env_file=env_file)


def openai_base_url(*, env_file: Path | None = None) -> str | None:
 """An endpoint override, or None for api.openai.com.

Expand Down

59 changes: 59 additions & 0 deletions
grapharc/gateway/novita.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
"""Novita backend — a GPU cloud hosting open-weight models, one key.

`novita/moonshotai/kimi-k3` reaches Novita's own OpenAI-compatible endpoint
(`https://api.novita.ai/openai`), not api.openai.com, so this builds on
`OpenAICompatChatModel` the same way `openrouter.py` and `ollama.py` do rather
than on `openai.py`: the endpoint is fixed, not an override of OpenAI's own.

Model ids on Novita are themselves `author/slug` — `moonshotai/kimi-k3`,
`zai-org/glm-5.2` — the same shape OpenRouter uses, so `vendor()` in
`registry.py` already reads the right author off a Novita spec with no
backend-specific handling: `BACKEND_VENDOR` stays absent for `novita`, exactly
as it is absent for `openrouter`.

**Novita reports no per-call cost.** Unlike OpenRouter, the chat-completions
response carries token counts and nothing else, so this backend is the OpenAI
one in that respect: `_provider_cost` is the base class's `None`, and
`cost_ceiling_usd` counts calls in `SpendMeter.unpriced_calls` unless a caller
supplies `price_per_million=`.
"""

from __future__ import annotations

from typing import Any

from grapharc.gateway.config import novita_api_key
from grapharc.gateway.openai_compat import OpenAICompatChatModel

NOVITA_BASE_URL = "https://api.novita.ai/openai"


class NovitaError(Exception):
 """The Novita backend could not be constructed or used."""


class NovitaChatModel(OpenAICompatChatModel):
 """A LangChain chat model over Novita's OpenAI-compatible endpoint."""

 def __init__(self, model: str, /, **kwargs: Any) -> None:
 api_key = kwargs.pop("api_key", None) or novita_api_key()
 if not api_key:
 raise NovitaError(
 "No Novita API key found. Set NOVITA_API_KEY in the environment, "
 "or add one of NOVITA_API_KEY / novita-api-key to a .env file."
 )
 # One retry layer, not two — same reasoning as the OpenRouter backend.
 kwargs.setdefault("max_retries", 0)
 super().__init__(
 model=model,
 api_key=api_key,
 base_url=kwargs.pop("base_url", None) or NOVITA_BASE_URL,
 **kwargs,
 )

 @property
 def _llm_type(self) -> str:
 return "grapharc-novita"


__all__ = ["NOVITA_BASE_URL", "NovitaChatModel", "NovitaError"]

15 changes: 14 additions & 1 deletion
grapharc/gateway/registry.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,7 @@
 openrouter/anthropic/claude-sonnet-4.5 -> OpenRouter
 openrouter/openai/gpt-4o:floor -> OpenRouter, cheapest provider
 openai/gpt-4o-mini -> OpenAI directly (OPENAI_API_KEY)
 novita/moonshotai/kimi-k3 -> Novita (NOVITA_API_KEY)
 ollama/llama3.1 -> a local Ollama server, no key
 mock/anything -> scripted test double

Expand All@@ -28,7 +29,7 @@ class UnknownBackendError(Exception):
 """The spec named a backend that is not registered."""


BACKENDS = ("claude-cli", "openrouter", "openai", "ollama", "mock")
BACKENDS = ("claude-cli", "openrouter", "openai", "novita", "ollama", "mock")

# Authors that appear in OpenRouter model ids. A spec starting with one of
# these is a model name, not a mistyped backend — `anthropic/claude-haiku-4.5`
Expand DownExpand Up@@ -71,6 +72,7 @@ class UnknownBackendError(Exception):
_BARE_BACKEND_EXAMPLE = {
 "openrouter": "openrouter/anthropic/claude-sonnet-4.5",
 "openai": "openai/gpt-4o-mini",
 "novita": "novita/moonshotai/kimi-k3",
 "ollama": "ollama/llama3.1",
}

Expand DownExpand Up@@ -160,6 +162,17 @@ def get_model(spec: str, **kwargs: Any) -> BaseChatModel:

 return OpenAIChatModel(model, **kwargs)

 if backend == "novita":
 try:
 from grapharc.gateway.novita import NovitaChatModel
 except ImportError as exc: # pragma: no cover - depends on install extras
 raise UnknownBackendError(
 "The Novita backend needs langchain-openai. "
 "Install it with: uv sync --extra novita"
 ) from exc

 return NovitaChatModel(model, **kwargs)

 if backend == "ollama":
 try:
 from grapharc.gateway.ollama import OllamaChatModel
Expand Down

8 changes: 7 additions & 1 deletion
pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -71,6 +71,12 @@ openrouter = [
openai = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/novita.py. Novita speaks the OpenAI wire format
# against its own endpoint, so this needs the same client and no Novita-specific
# package.
novita = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/ollama.py. Ollama speaks the OpenAI wire format,
# so the local backend needs the same client and no ollama-specific package.
ollama = [
Expand DownExpand Up@@ -110,7 +116,7 @@ slack = [
]
# Everything above. Self-referential so it cannot drift out of sync.
all = [
 "grapharc[api,ladybug,mcp,ollama,openai,openrouter,otel,server,slack]",
 "grapharc[api,ladybug,mcp,novita,ollama,openai,openrouter,otel,server,slack]",
]

[dependency-groups]
Expand Down

4 changes: 3 additions & 1 deletion
tests/test_cli.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,7 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 for name in (
 "OPENROUTER_API_KEY", "OPENROUTER_KEY", "open-router-api-key",
 "OPENAI_API_KEY", "OPENAI_KEY", "openai-api-key",
 "NOVITA_API_KEY", "NOVITA_KEY", "novita-api-key",
 "OLLAMA_HOST", "OLLAMA_BASE_URL",
 ):
 monkeypatch.delenv(name, raising=False)
Expand All@@ -452,10 +453,11 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 "claude-cli": False,
 "openrouter": False,
 "openai": False,
 "novita": False,
 "ollama": False,
 "mock": True,
 }
 for backend in ("openrouter", "openai"):
 for backend in ("openrouter", "openai", "novita"):
 assert next(b for b in payload["backends"] if b["backend"] == backend)[
 "credential"
 ] == "<unset>"
Expand Down
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Add Novita as a model gateway backend#102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Shashankss1205
merged 3 commits into

from

Aug 13, 2026
 Merged


Add Novita as a model gateway backend
#102
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
7f98eaf

Add Novita as a model gateway backend


jax-novita Aug 5, 2026
8f177da

Recommend Novita's current flagship models


jax-novita Aug 5, 2026
2afecff

Fix column alignment for novita example in models cookbook


jax-novita Aug 10, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 12 additions & 5 deletions
docs/cookbook/02-models.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,7 @@ for spec in (
 "claude-cli/claude-sonnet-5",
 "openrouter/anthropic/claude-haiku-4.5",
 "openai/gpt-4o-mini",
 "novita/moonshotai/kimi-k3",
 "ollama/llama3.1",
 "mock/anything",
):
Expand All@@ -38,18 +39,20 @@ print(model.invoke("say hi").content)
{'spec': 'claude-cli/claude-sonnet-5', 'backend': 'claude-cli', 'model': 'claude-sonnet-5'}
{'spec': 'openrouter/anthropic/claude-haiku-4.5', 'backend': 'openrouter', 'model': 'anthropic/claude-haiku-4.5'}
{'spec': 'openai/gpt-4o-mini', 'backend': 'openai', 'model': 'gpt-4o-mini'}
{'spec': 'novita/moonshotai/kimi-k3', 'backend': 'novita', 'model': 'moonshotai/kimi-k3'}
{'spec': 'ollama/llama3.1', 'backend': 'ollama', 'model': 'llama3.1'}
{'spec': 'mock/anything', 'backend': 'mock', 'model': 'anything'}
hello from a scripted model
```

There are five backends:
There are six backends:

| backend | credential | what it is for |
| --- | --- | --- |
| `claude-cli` | a Claude subscription, no API key | text completion on quota you already pay for |
| `openrouter` | `OPENROUTER_API_KEY` | one key, most vendors, per-call cost in the response |
| `openai` | `OPENAI_API_KEY` | the OpenAI API directly, or any endpoint via `OPENAI_BASE_URL` |
| `novita` | `NOVITA_API_KEY` | Novita's own endpoint, token counts but no per-call cost |
| `ollama` | none — a local server | models on your own machine, free and offline |
| `mock` | none | a scripted test double |

Expand All@@ -63,8 +66,8 @@ needs it. Asking for `claude-cli` therefore does not require `langchain-openai`,
for `openrouter` does not require the Claude CLI to be installed. A missing optional
dependency fails for the backend that wanted it and nothing else.

`openrouter`, `openai`and `ollama` all speak the OpenAI wire format and share one base
class, so they behave identically on everything except money and routing: same
`openrouter`, `openai`, `novita` and `ollama` all speak the OpenAI wire format and share one
base class, so they behave identically on everything except money and routing: same
`bind_tools`, same `with_structured_output`, same streaming and async, same retry policy,
same usage envelope.

Expand DownExpand Up@@ -93,7 +96,7 @@ except UnknownBackendError as exc:
('openrouter', 'openai/gpt-4o-mini:floor')
('claude-cli', 'anthropic/claude-haiku-4.5')
('openai', 'gpt-4o-mini')
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, ollama, mock — or a bare model name for the claude-cli default
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, novita, ollama, mock — or a bare model name for the claude-cli default
```

Only the first segment is a backend, because OpenRouter model ids are themselves
Expand DownExpand Up@@ -122,16 +125,18 @@ model through the broker is still `openrouter/openai/gpt-4o-mini`.
<!-- verified: cli -->
```console
$ grapharc models
backends: claude-cli, openrouter, openai, ollama, mock
backends: claude-cli, openrouter, openai, novita, ollama, mock
openrouter key: <unset>
openai key: <unset>
novita key: <unset>
ollama url: http://localhost:11434/v1

examples:
 claude-cli/claude-sonnet-5 subscription, no API key
 openrouter/anthropic/claude-haiku-4.5 many providers, one key
 openrouter/openai/gpt-4o-mini:floor cheapest provider for that model
 openai/gpt-4o-mini the OpenAI API directly, your key
 novita/moonshotai/kimi-k3 Novita's own endpoint, your key
 ollama/llama3.1 a local server, no key and no bill

grapharc models --check probes which of these this machine can use
Expand DownExpand Up@@ -165,6 +170,8 @@ openrouter unusable no API key (set OPENROUTER_API_KEY, or add one to .env)
 credential: <unset>
openai unusable no API key (set OPENAI_API_KEY, or add one to .env)
 credential: <unset>
novita unusable no API key (set NOVITA_API_KEY, or add one to .env)
 credential: <unset>
ollama usable local server at http://localhost:11434/v1
 credential: none needed (local server)
mock usable scripted test double; never reaches a provider
Expand Down

4 changes: 4 additions & 0 deletions
grapharc/cli/main.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -449,6 +449,7 @@ def _cmd_approve(args: argparse.Namespace) -> int:
def _cmd_models(args: argparse.Namespace) -> int:
 from grapharc.gateway import (
 describe,
 novita_api_key,
 ollama_base_url,
 openai_api_key,
 openrouter_api_key,
Expand DownExpand Up@@ -497,6 +498,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "openrouter/anthropic/claude-haiku-4.5": "many providers, one key",
 "openrouter/openai/gpt-4o-mini:floor": "cheapest provider for that model",
 "openai/gpt-4o-mini": "the OpenAI API directly, your key",
 "novita/moonshotai/kimi-k3": "Novita's own endpoint, your key",
 "ollama/llama3.1": "a local server, no key and no bill",
 }
 payload = {
Expand All@@ -505,6 +507,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "backends": list(BACKENDS),
 "openrouter_key": redact(openrouter_api_key()),
 "openai_key": redact(openai_api_key()),
 "novita_key": redact(novita_api_key()),
 # An address, not a secret: it is printed whole, and it is where a
 # request would go rather than proof that anything is listening.
 "ollama_base_url": ollama_base_url(),
Expand All@@ -516,6 +519,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 style.kv("backends", ", ".join(BACKENDS)),
 style.kv("openrouter key", redact(openrouter_api_key())),
 style.kv("openai key", redact(openai_api_key())),
 style.kv("novita key", redact(novita_api_key())),
 style.kv("ollama url", ollama_base_url(), tint=style.accent),
 "",
 style.heading("examples:"),
Expand Down

21 changes: 21 additions & 0 deletions
grapharc/cli/probe.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -95,6 +95,26 @@ def _probe_openai() -> dict[str, Any]:
 }


def _probe_novita() -> dict[str, Any]:
 from grapharc.gateway import novita_api_key, redact

 key = novita_api_key()
 has_dependency = importlib.util.find_spec("langchain_openai") is not None
 missing = []
 if not key:
 missing.append("no API key (set NOVITA_API_KEY, or add one to .env)")
 if not has_dependency:
 missing.append("langchain-openai not installed (uv sync --extra novita)")
 return {
 "backend": "novita",
 "kind": KIND_PROVIDER,
 "usable": bool(key) and has_dependency,
 "credential": redact(key),
 "detail": "; ".join(missing) or "api key configured and langchain-openai installed",
 "checked": "credential presence only; no request was sent to api.novita.ai",
 }


def _probe_ollama() -> dict[str, Any]:
 from grapharc.gateway import ollama_base_url

Expand DownExpand Up@@ -152,6 +172,7 @@ def probe_backends(*, claude_path: str = "claude") -> list[dict[str, Any]]:
 "claude-cli": lambda: _probe_claude_cli(claude_path),
 "openrouter": _probe_openrouter,
 "openai": _probe_openai,
 "novita": _probe_novita,
 "ollama": _probe_ollama,
 "mock": _probe_mock,
 }
Expand Down

9 changes: 7 additions & 2 deletions
grapharc/gateway/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6,6 +6,7 @@
 get_model("claude-cli/claude-sonnet-5") # subscription, no API key
 get_model("openrouter/anthropic/claude-sonnet-4.5") # many providers, one key
 get_model("openai/gpt-4o-mini") # OPENAI_API_KEY
 get_model("novita/moonshotai/kimi-k3") # NOVITA_API_KEY
 get_model("ollama/llama3.1") # local server, no key
 get_model("mock/x", responses=[...]) # deterministic tests

Expand All@@ -16,12 +17,13 @@
 get_model(spec, cost_ceiling_usd=0.25) # raises when passed
 get_model(spec, spend=shared_meter) # one ceiling, many models

The three OpenAI-wire backends (`openrouter`, `openai`, `ollama`) are imported
lazily — they need `langchain-openai`, which is an optional extra.
The four OpenAI-wire backends (`openrouter`, `openai`, `novita`, `ollama`) are
imported lazily — they need `langchain-openai`, which is an optional extra.
"""

from grapharc.gateway.claude_cli import ClaudeCodeCLIChatModel
from grapharc.gateway.config import (
 novita_api_key,
 ollama_api_key,
 ollama_base_url,
 openai_api_key,
Expand DownExpand Up@@ -66,6 +68,7 @@
 "different_providers",
 "get_model",
 "is_transient",
 "novita_api_key",
 "ollama_api_key",
 "ollama_base_url",
 "openai_api_key",
Expand All@@ -85,6 +88,8 @@
 "OpenRouterError": "openrouter",
 "OpenAIChatModel": "openai",
 "OpenAIError": "openai",
 "NovitaChatModel": "novita",
 "NovitaError": "novita",
 "OllamaChatModel": "ollama",
 "OllamaError": "ollama",
}
Expand Down

16 changes: 15 additions & 1 deletion
grapharc/gateway/config.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -20,14 +20,17 @@
Secrets are returned, never logged. Anything that renders a config for humans
goes through `redact`.

Three key-holding backends, plus one that usually holds none:
Four key-holding backends, plus one that usually holds none:

- **OpenRouter** — `OPENROUTER_API_KEY`.
- **OpenAI** — `OPENAI_API_KEY`, optionally with `OPENAI_BASE_URL` for an
 Azure-style or proxied endpoint. `langchain-openai` reads `OPENAI_API_KEY`
 from the process environment on its own; going through here as well is what
 adds `.env` support, the alternate spellings, and a failure that names the
 variable instead of surfacing an SDK error.
- **Novita** — `NOVITA_API_KEY`. The endpoint is fixed
 (`grapharc.gateway.novita.NOVITA_BASE_URL`), so unlike OpenAI there is no
 base-url override to resolve here.
- **Ollama** — no credential by default: it is a server on your own machine.
 What it needs is an address, so `ollama_base_url()` always returns one
 (`OLLAMA_HOST` / `OLLAMA_BASE_URL`, else localhost). `OLLAMA_API_KEY` exists
Expand DownExpand Up@@ -56,6 +59,13 @@
 "openai_api_key",
)

NOVITA_KEYS = (
 "NOVITA_API_KEY",
 "NOVITA_KEY",
 "novita-api-key",
 "novita_api_key",
)

# `OPENAI_API_BASE` is the older spelling and is still what a lot of tooling
# sets; both are accepted, the current one first.
OPENAI_BASE_URL_KEYS = (
Expand DownExpand Up@@ -133,6 +143,10 @@ def openai_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(OPENAI_KEYS, env_file=env_file)


def novita_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(NOVITA_KEYS, env_file=env_file)


def openai_base_url(*, env_file: Path | None = None) -> str | None:
 """An endpoint override, or None for api.openai.com.

Expand Down

59 changes: 59 additions & 0 deletions
grapharc/gateway/novita.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
"""Novita backend — a GPU cloud hosting open-weight models, one key.

`novita/moonshotai/kimi-k3` reaches Novita's own OpenAI-compatible endpoint
(`https://api.novita.ai/openai`), not api.openai.com, so this builds on
`OpenAICompatChatModel` the same way `openrouter.py` and `ollama.py` do rather
than on `openai.py`: the endpoint is fixed, not an override of OpenAI's own.

Model ids on Novita are themselves `author/slug` — `moonshotai/kimi-k3`,
`zai-org/glm-5.2` — the same shape OpenRouter uses, so `vendor()` in
`registry.py` already reads the right author off a Novita spec with no
backend-specific handling: `BACKEND_VENDOR` stays absent for `novita`, exactly
as it is absent for `openrouter`.

**Novita reports no per-call cost.** Unlike OpenRouter, the chat-completions
response carries token counts and nothing else, so this backend is the OpenAI
one in that respect: `_provider_cost` is the base class's `None`, and
`cost_ceiling_usd` counts calls in `SpendMeter.unpriced_calls` unless a caller
supplies `price_per_million=`.
"""

from __future__ import annotations

from typing import Any

from grapharc.gateway.config import novita_api_key
from grapharc.gateway.openai_compat import OpenAICompatChatModel

NOVITA_BASE_URL = "https://api.novita.ai/openai"


class NovitaError(Exception):
 """The Novita backend could not be constructed or used."""


class NovitaChatModel(OpenAICompatChatModel):
 """A LangChain chat model over Novita's OpenAI-compatible endpoint."""

 def __init__(self, model: str, /, **kwargs: Any) -> None:
 api_key = kwargs.pop("api_key", None) or novita_api_key()
 if not api_key:
 raise NovitaError(
 "No Novita API key found. Set NOVITA_API_KEY in the environment, "
 "or add one of NOVITA_API_KEY / novita-api-key to a .env file."
 )
 # One retry layer, not two — same reasoning as the OpenRouter backend.
 kwargs.setdefault("max_retries", 0)
 super().__init__(
 model=model,
 api_key=api_key,
 base_url=kwargs.pop("base_url", None) or NOVITA_BASE_URL,
 **kwargs,
 )

 @property
 def _llm_type(self) -> str:
 return "grapharc-novita"


__all__ = ["NOVITA_BASE_URL", "NovitaChatModel", "NovitaError"]

15 changes: 14 additions & 1 deletion
grapharc/gateway/registry.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,7 @@
 openrouter/anthropic/claude-sonnet-4.5 -> OpenRouter
 openrouter/openai/gpt-4o:floor -> OpenRouter, cheapest provider
 openai/gpt-4o-mini -> OpenAI directly (OPENAI_API_KEY)
 novita/moonshotai/kimi-k3 -> Novita (NOVITA_API_KEY)
 ollama/llama3.1 -> a local Ollama server, no key
 mock/anything -> scripted test double

Expand All@@ -28,7 +29,7 @@ class UnknownBackendError(Exception):
 """The spec named a backend that is not registered."""


BACKENDS = ("claude-cli", "openrouter", "openai", "ollama", "mock")
BACKENDS = ("claude-cli", "openrouter", "openai", "novita", "ollama", "mock")

# Authors that appear in OpenRouter model ids. A spec starting with one of
# these is a model name, not a mistyped backend — `anthropic/claude-haiku-4.5`
Expand DownExpand Up@@ -71,6 +72,7 @@ class UnknownBackendError(Exception):
_BARE_BACKEND_EXAMPLE = {
 "openrouter": "openrouter/anthropic/claude-sonnet-4.5",
 "openai": "openai/gpt-4o-mini",
 "novita": "novita/moonshotai/kimi-k3",
 "ollama": "ollama/llama3.1",
}

Expand DownExpand Up@@ -160,6 +162,17 @@ def get_model(spec: str, **kwargs: Any) -> BaseChatModel:

 return OpenAIChatModel(model, **kwargs)

 if backend == "novita":
 try:
 from grapharc.gateway.novita import NovitaChatModel
 except ImportError as exc: # pragma: no cover - depends on install extras
 raise UnknownBackendError(
 "The Novita backend needs langchain-openai. "
 "Install it with: uv sync --extra novita"
 ) from exc

 return NovitaChatModel(model, **kwargs)

 if backend == "ollama":
 try:
 from grapharc.gateway.ollama import OllamaChatModel
Expand Down

8 changes: 7 additions & 1 deletion
pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -71,6 +71,12 @@ openrouter = [
openai = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/novita.py. Novita speaks the OpenAI wire format
# against its own endpoint, so this needs the same client and no Novita-specific
# package.
novita = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/ollama.py. Ollama speaks the OpenAI wire format,
# so the local backend needs the same client and no ollama-specific package.
ollama = [
Expand DownExpand Up@@ -110,7 +116,7 @@ slack = [
]
# Everything above. Self-referential so it cannot drift out of sync.
all = [
 "grapharc[api,ladybug,mcp,ollama,openai,openrouter,otel,server,slack]",
 "grapharc[api,ladybug,mcp,novita,ollama,openai,openrouter,otel,server,slack]",
]

[dependency-groups]
Expand Down

4 changes: 3 additions & 1 deletion
tests/test_cli.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,7 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 for name in (
 "OPENROUTER_API_KEY", "OPENROUTER_KEY", "open-router-api-key",
 "OPENAI_API_KEY", "OPENAI_KEY", "openai-api-key",
 "NOVITA_API_KEY", "NOVITA_KEY", "novita-api-key",
 "OLLAMA_HOST", "OLLAMA_BASE_URL",
 ):
 monkeypatch.delenv(name, raising=False)
Expand All@@ -452,10 +453,11 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 "claude-cli": False,
 "openrouter": False,
 "openai": False,
 "novita": False,
 "ollama": False,
 "mock": True,
 }
 for backend in ("openrouter", "openai"):
 for backend in ("openrouter", "openai", "novita"):
 assert next(b for b in payload["backends"] if b["backend"] == backend)[
 "credential"
 ] == "<unset>"
Expand Down
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Add Novita as a model gateway backend#102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Shashankss1205
merged 3 commits into

from

Aug 13, 2026
 Merged


Add Novita as a model gateway backend
#102
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
7f98eaf

Add Novita as a model gateway backend


jax-novita Aug 5, 2026
8f177da

Recommend Novita's current flagship models


jax-novita Aug 5, 2026
2afecff

Fix column alignment for novita example in models cookbook


jax-novita Aug 10, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 12 additions & 5 deletions
docs/cookbook/02-models.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,7 @@ for spec in (
 "claude-cli/claude-sonnet-5",
 "openrouter/anthropic/claude-haiku-4.5",
 "openai/gpt-4o-mini",
 "novita/moonshotai/kimi-k3",
 "ollama/llama3.1",
 "mock/anything",
):
Expand All@@ -38,18 +39,20 @@ print(model.invoke("say hi").content)
{'spec': 'claude-cli/claude-sonnet-5', 'backend': 'claude-cli', 'model': 'claude-sonnet-5'}
{'spec': 'openrouter/anthropic/claude-haiku-4.5', 'backend': 'openrouter', 'model': 'anthropic/claude-haiku-4.5'}
{'spec': 'openai/gpt-4o-mini', 'backend': 'openai', 'model': 'gpt-4o-mini'}
{'spec': 'novita/moonshotai/kimi-k3', 'backend': 'novita', 'model': 'moonshotai/kimi-k3'}
{'spec': 'ollama/llama3.1', 'backend': 'ollama', 'model': 'llama3.1'}
{'spec': 'mock/anything', 'backend': 'mock', 'model': 'anything'}
hello from a scripted model
```

There are five backends:
There are six backends:

| backend | credential | what it is for |
| --- | --- | --- |
| `claude-cli` | a Claude subscription, no API key | text completion on quota you already pay for |
| `openrouter` | `OPENROUTER_API_KEY` | one key, most vendors, per-call cost in the response |
| `openai` | `OPENAI_API_KEY` | the OpenAI API directly, or any endpoint via `OPENAI_BASE_URL` |
| `novita` | `NOVITA_API_KEY` | Novita's own endpoint, token counts but no per-call cost |
| `ollama` | none — a local server | models on your own machine, free and offline |
| `mock` | none | a scripted test double |

Expand All@@ -63,8 +66,8 @@ needs it. Asking for `claude-cli` therefore does not require `langchain-openai`,
for `openrouter` does not require the Claude CLI to be installed. A missing optional
dependency fails for the backend that wanted it and nothing else.

`openrouter`, `openai`and `ollama` all speak the OpenAI wire format and share one base
class, so they behave identically on everything except money and routing: same
`openrouter`, `openai`, `novita` and `ollama` all speak the OpenAI wire format and share one
base class, so they behave identically on everything except money and routing: same
`bind_tools`, same `with_structured_output`, same streaming and async, same retry policy,
same usage envelope.

Expand DownExpand Up@@ -93,7 +96,7 @@ except UnknownBackendError as exc:
('openrouter', 'openai/gpt-4o-mini:floor')
('claude-cli', 'anthropic/claude-haiku-4.5')
('openai', 'gpt-4o-mini')
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, ollama, mock — or a bare model name for the claude-cli default
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, novita, ollama, mock — or a bare model name for the claude-cli default
```

Only the first segment is a backend, because OpenRouter model ids are themselves
Expand DownExpand Up@@ -122,16 +125,18 @@ model through the broker is still `openrouter/openai/gpt-4o-mini`.
<!-- verified: cli -->
```console
$ grapharc models
backends: claude-cli, openrouter, openai, ollama, mock
backends: claude-cli, openrouter, openai, novita, ollama, mock
openrouter key: <unset>
openai key: <unset>
novita key: <unset>
ollama url: http://localhost:11434/v1

examples:
 claude-cli/claude-sonnet-5 subscription, no API key
 openrouter/anthropic/claude-haiku-4.5 many providers, one key
 openrouter/openai/gpt-4o-mini:floor cheapest provider for that model
 openai/gpt-4o-mini the OpenAI API directly, your key
 novita/moonshotai/kimi-k3 Novita's own endpoint, your key
 ollama/llama3.1 a local server, no key and no bill

grapharc models --check probes which of these this machine can use
Expand DownExpand Up@@ -165,6 +170,8 @@ openrouter unusable no API key (set OPENROUTER_API_KEY, or add one to .env)
 credential: <unset>
openai unusable no API key (set OPENAI_API_KEY, or add one to .env)
 credential: <unset>
novita unusable no API key (set NOVITA_API_KEY, or add one to .env)
 credential: <unset>
ollama usable local server at http://localhost:11434/v1
 credential: none needed (local server)
mock usable scripted test double; never reaches a provider
Expand Down

4 changes: 4 additions & 0 deletions
grapharc/cli/main.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -449,6 +449,7 @@ def _cmd_approve(args: argparse.Namespace) -> int:
def _cmd_models(args: argparse.Namespace) -> int:
 from grapharc.gateway import (
 describe,
 novita_api_key,
 ollama_base_url,
 openai_api_key,
 openrouter_api_key,
Expand DownExpand Up@@ -497,6 +498,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "openrouter/anthropic/claude-haiku-4.5": "many providers, one key",
 "openrouter/openai/gpt-4o-mini:floor": "cheapest provider for that model",
 "openai/gpt-4o-mini": "the OpenAI API directly, your key",
 "novita/moonshotai/kimi-k3": "Novita's own endpoint, your key",
 "ollama/llama3.1": "a local server, no key and no bill",
 }
 payload = {
Expand All@@ -505,6 +507,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "backends": list(BACKENDS),
 "openrouter_key": redact(openrouter_api_key()),
 "openai_key": redact(openai_api_key()),
 "novita_key": redact(novita_api_key()),
 # An address, not a secret: it is printed whole, and it is where a
 # request would go rather than proof that anything is listening.
 "ollama_base_url": ollama_base_url(),
Expand All@@ -516,6 +519,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 style.kv("backends", ", ".join(BACKENDS)),
 style.kv("openrouter key", redact(openrouter_api_key())),
 style.kv("openai key", redact(openai_api_key())),
 style.kv("novita key", redact(novita_api_key())),
 style.kv("ollama url", ollama_base_url(), tint=style.accent),
 "",
 style.heading("examples:"),
Expand Down

21 changes: 21 additions & 0 deletions
grapharc/cli/probe.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -95,6 +95,26 @@ def _probe_openai() -> dict[str, Any]:
 }


def _probe_novita() -> dict[str, Any]:
 from grapharc.gateway import novita_api_key, redact

 key = novita_api_key()
 has_dependency = importlib.util.find_spec("langchain_openai") is not None
 missing = []
 if not key:
 missing.append("no API key (set NOVITA_API_KEY, or add one to .env)")
 if not has_dependency:
 missing.append("langchain-openai not installed (uv sync --extra novita)")
 return {
 "backend": "novita",
 "kind": KIND_PROVIDER,
 "usable": bool(key) and has_dependency,
 "credential": redact(key),
 "detail": "; ".join(missing) or "api key configured and langchain-openai installed",
 "checked": "credential presence only; no request was sent to api.novita.ai",
 }


def _probe_ollama() -> dict[str, Any]:
 from grapharc.gateway import ollama_base_url

Expand DownExpand Up@@ -152,6 +172,7 @@ def probe_backends(*, claude_path: str = "claude") -> list[dict[str, Any]]:
 "claude-cli": lambda: _probe_claude_cli(claude_path),
 "openrouter": _probe_openrouter,
 "openai": _probe_openai,
 "novita": _probe_novita,
 "ollama": _probe_ollama,
 "mock": _probe_mock,
 }
Expand Down

9 changes: 7 additions & 2 deletions
grapharc/gateway/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6,6 +6,7 @@
 get_model("claude-cli/claude-sonnet-5") # subscription, no API key
 get_model("openrouter/anthropic/claude-sonnet-4.5") # many providers, one key
 get_model("openai/gpt-4o-mini") # OPENAI_API_KEY
 get_model("novita/moonshotai/kimi-k3") # NOVITA_API_KEY
 get_model("ollama/llama3.1") # local server, no key
 get_model("mock/x", responses=[...]) # deterministic tests

Expand All@@ -16,12 +17,13 @@
 get_model(spec, cost_ceiling_usd=0.25) # raises when passed
 get_model(spec, spend=shared_meter) # one ceiling, many models

The three OpenAI-wire backends (`openrouter`, `openai`, `ollama`) are imported
lazily — they need `langchain-openai`, which is an optional extra.
The four OpenAI-wire backends (`openrouter`, `openai`, `novita`, `ollama`) are
imported lazily — they need `langchain-openai`, which is an optional extra.
"""

from grapharc.gateway.claude_cli import ClaudeCodeCLIChatModel
from grapharc.gateway.config import (
 novita_api_key,
 ollama_api_key,
 ollama_base_url,
 openai_api_key,
Expand DownExpand Up@@ -66,6 +68,7 @@
 "different_providers",
 "get_model",
 "is_transient",
 "novita_api_key",
 "ollama_api_key",
 "ollama_base_url",
 "openai_api_key",
Expand All@@ -85,6 +88,8 @@
 "OpenRouterError": "openrouter",
 "OpenAIChatModel": "openai",
 "OpenAIError": "openai",
 "NovitaChatModel": "novita",
 "NovitaError": "novita",
 "OllamaChatModel": "ollama",
 "OllamaError": "ollama",
}
Expand Down

16 changes: 15 additions & 1 deletion
grapharc/gateway/config.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -20,14 +20,17 @@
Secrets are returned, never logged. Anything that renders a config for humans
goes through `redact`.

Three key-holding backends, plus one that usually holds none:
Four key-holding backends, plus one that usually holds none:

- **OpenRouter** — `OPENROUTER_API_KEY`.
- **OpenAI** — `OPENAI_API_KEY`, optionally with `OPENAI_BASE_URL` for an
 Azure-style or proxied endpoint. `langchain-openai` reads `OPENAI_API_KEY`
 from the process environment on its own; going through here as well is what
 adds `.env` support, the alternate spellings, and a failure that names the
 variable instead of surfacing an SDK error.
- **Novita** — `NOVITA_API_KEY`. The endpoint is fixed
 (`grapharc.gateway.novita.NOVITA_BASE_URL`), so unlike OpenAI there is no
 base-url override to resolve here.
- **Ollama** — no credential by default: it is a server on your own machine.
 What it needs is an address, so `ollama_base_url()` always returns one
 (`OLLAMA_HOST` / `OLLAMA_BASE_URL`, else localhost). `OLLAMA_API_KEY` exists
Expand DownExpand Up@@ -56,6 +59,13 @@
 "openai_api_key",
)

NOVITA_KEYS = (
 "NOVITA_API_KEY",
 "NOVITA_KEY",
 "novita-api-key",
 "novita_api_key",
)

# `OPENAI_API_BASE` is the older spelling and is still what a lot of tooling
# sets; both are accepted, the current one first.
OPENAI_BASE_URL_KEYS = (
Expand DownExpand Up@@ -133,6 +143,10 @@ def openai_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(OPENAI_KEYS, env_file=env_file)


def novita_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(NOVITA_KEYS, env_file=env_file)


def openai_base_url(*, env_file: Path | None = None) -> str | None:
 """An endpoint override, or None for api.openai.com.

Expand Down

59 changes: 59 additions & 0 deletions
grapharc/gateway/novita.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
"""Novita backend — a GPU cloud hosting open-weight models, one key.

`novita/moonshotai/kimi-k3` reaches Novita's own OpenAI-compatible endpoint
(`https://api.novita.ai/openai`), not api.openai.com, so this builds on
`OpenAICompatChatModel` the same way `openrouter.py` and `ollama.py` do rather
than on `openai.py`: the endpoint is fixed, not an override of OpenAI's own.

Model ids on Novita are themselves `author/slug` — `moonshotai/kimi-k3`,
`zai-org/glm-5.2` — the same shape OpenRouter uses, so `vendor()` in
`registry.py` already reads the right author off a Novita spec with no
backend-specific handling: `BACKEND_VENDOR` stays absent for `novita`, exactly
as it is absent for `openrouter`.

**Novita reports no per-call cost.** Unlike OpenRouter, the chat-completions
response carries token counts and nothing else, so this backend is the OpenAI
one in that respect: `_provider_cost` is the base class's `None`, and
`cost_ceiling_usd` counts calls in `SpendMeter.unpriced_calls` unless a caller
supplies `price_per_million=`.
"""

from __future__ import annotations

from typing import Any

from grapharc.gateway.config import novita_api_key
from grapharc.gateway.openai_compat import OpenAICompatChatModel

NOVITA_BASE_URL = "https://api.novita.ai/openai"


class NovitaError(Exception):
 """The Novita backend could not be constructed or used."""


class NovitaChatModel(OpenAICompatChatModel):
 """A LangChain chat model over Novita's OpenAI-compatible endpoint."""

 def __init__(self, model: str, /, **kwargs: Any) -> None:
 api_key = kwargs.pop("api_key", None) or novita_api_key()
 if not api_key:
 raise NovitaError(
 "No Novita API key found. Set NOVITA_API_KEY in the environment, "
 "or add one of NOVITA_API_KEY / novita-api-key to a .env file."
 )
 # One retry layer, not two — same reasoning as the OpenRouter backend.
 kwargs.setdefault("max_retries", 0)
 super().__init__(
 model=model,
 api_key=api_key,
 base_url=kwargs.pop("base_url", None) or NOVITA_BASE_URL,
 **kwargs,
 )

 @property
 def _llm_type(self) -> str:
 return "grapharc-novita"


__all__ = ["NOVITA_BASE_URL", "NovitaChatModel", "NovitaError"]

15 changes: 14 additions & 1 deletion
grapharc/gateway/registry.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,7 @@
 openrouter/anthropic/claude-sonnet-4.5 -> OpenRouter
 openrouter/openai/gpt-4o:floor -> OpenRouter, cheapest provider
 openai/gpt-4o-mini -> OpenAI directly (OPENAI_API_KEY)
 novita/moonshotai/kimi-k3 -> Novita (NOVITA_API_KEY)
 ollama/llama3.1 -> a local Ollama server, no key
 mock/anything -> scripted test double

Expand All@@ -28,7 +29,7 @@ class UnknownBackendError(Exception):
 """The spec named a backend that is not registered."""


BACKENDS = ("claude-cli", "openrouter", "openai", "ollama", "mock")
BACKENDS = ("claude-cli", "openrouter", "openai", "novita", "ollama", "mock")

# Authors that appear in OpenRouter model ids. A spec starting with one of
# these is a model name, not a mistyped backend — `anthropic/claude-haiku-4.5`
Expand DownExpand Up@@ -71,6 +72,7 @@ class UnknownBackendError(Exception):
_BARE_BACKEND_EXAMPLE = {
 "openrouter": "openrouter/anthropic/claude-sonnet-4.5",
 "openai": "openai/gpt-4o-mini",
 "novita": "novita/moonshotai/kimi-k3",
 "ollama": "ollama/llama3.1",
}

Expand DownExpand Up@@ -160,6 +162,17 @@ def get_model(spec: str, **kwargs: Any) -> BaseChatModel:

 return OpenAIChatModel(model, **kwargs)

 if backend == "novita":
 try:
 from grapharc.gateway.novita import NovitaChatModel
 except ImportError as exc: # pragma: no cover - depends on install extras
 raise UnknownBackendError(
 "The Novita backend needs langchain-openai. "
 "Install it with: uv sync --extra novita"
 ) from exc

 return NovitaChatModel(model, **kwargs)

 if backend == "ollama":
 try:
 from grapharc.gateway.ollama import OllamaChatModel
Expand Down

8 changes: 7 additions & 1 deletion
pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -71,6 +71,12 @@ openrouter = [
openai = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/novita.py. Novita speaks the OpenAI wire format
# against its own endpoint, so this needs the same client and no Novita-specific
# package.
novita = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/ollama.py. Ollama speaks the OpenAI wire format,
# so the local backend needs the same client and no ollama-specific package.
ollama = [
Expand DownExpand Up@@ -110,7 +116,7 @@ slack = [
]
# Everything above. Self-referential so it cannot drift out of sync.
all = [
 "grapharc[api,ladybug,mcp,ollama,openai,openrouter,otel,server,slack]",
 "grapharc[api,ladybug,mcp,novita,ollama,openai,openrouter,otel,server,slack]",
]

[dependency-groups]
Expand Down

4 changes: 3 additions & 1 deletion
tests/test_cli.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,7 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 for name in (
 "OPENROUTER_API_KEY", "OPENROUTER_KEY", "open-router-api-key",
 "OPENAI_API_KEY", "OPENAI_KEY", "openai-api-key",
 "NOVITA_API_KEY", "NOVITA_KEY", "novita-api-key",
 "OLLAMA_HOST", "OLLAMA_BASE_URL",
 ):
 monkeypatch.delenv(name, raising=False)
Expand All@@ -452,10 +453,11 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 "claude-cli": False,
 "openrouter": False,
 "openai": False,
 "novita": False,
 "ollama": False,
 "mock": True,
 }
 for backend in ("openrouter", "openai"):
 for backend in ("openrouter", "openai", "novita"):
 assert next(b for b in payload["backends"] if b["backend"] == backend)[
 "credential"
 ] == "<unset>"
Expand Down
Loading
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Add Novita as a model gateway backend#102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

Shashankss1205
merged 3 commits into

from

Aug 13, 2026
 Merged


Add Novita as a model gateway backend
#102
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


3 commits


Select commit
Hold shift + click to select a range
7f98eaf

Add Novita as a model gateway backend


jax-novita Aug 5, 2026
8f177da

Recommend Novita's current flagship models


jax-novita Aug 5, 2026
2afecff

Fix column alignment for novita example in models cookbook


jax-novita Aug 10, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 12 additions & 5 deletions
docs/cookbook/02-models.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -24,6 +24,7 @@ for spec in (
 "claude-cli/claude-sonnet-5",
 "openrouter/anthropic/claude-haiku-4.5",
 "openai/gpt-4o-mini",
 "novita/moonshotai/kimi-k3",
 "ollama/llama3.1",
 "mock/anything",
):
Expand All@@ -38,18 +39,20 @@ print(model.invoke("say hi").content)
{'spec': 'claude-cli/claude-sonnet-5', 'backend': 'claude-cli', 'model': 'claude-sonnet-5'}
{'spec': 'openrouter/anthropic/claude-haiku-4.5', 'backend': 'openrouter', 'model': 'anthropic/claude-haiku-4.5'}
{'spec': 'openai/gpt-4o-mini', 'backend': 'openai', 'model': 'gpt-4o-mini'}
{'spec': 'novita/moonshotai/kimi-k3', 'backend': 'novita', 'model': 'moonshotai/kimi-k3'}
{'spec': 'ollama/llama3.1', 'backend': 'ollama', 'model': 'llama3.1'}
{'spec': 'mock/anything', 'backend': 'mock', 'model': 'anything'}
hello from a scripted model
```

There are five backends:
There are six backends:

| backend | credential | what it is for |
| --- | --- | --- |
| `claude-cli` | a Claude subscription, no API key | text completion on quota you already pay for |
| `openrouter` | `OPENROUTER_API_KEY` | one key, most vendors, per-call cost in the response |
| `openai` | `OPENAI_API_KEY` | the OpenAI API directly, or any endpoint via `OPENAI_BASE_URL` |
| `novita` | `NOVITA_API_KEY` | Novita's own endpoint, token counts but no per-call cost |
| `ollama` | none — a local server | models on your own machine, free and offline |
| `mock` | none | a scripted test double |

Expand All@@ -63,8 +66,8 @@ needs it. Asking for `claude-cli` therefore does not require `langchain-openai`,
for `openrouter` does not require the Claude CLI to be installed. A missing optional
dependency fails for the backend that wanted it and nothing else.

`openrouter`, `openai`and `ollama` all speak the OpenAI wire format and share one base
class, so they behave identically on everything except money and routing: same
`openrouter`, `openai`, `novita` and `ollama` all speak the OpenAI wire format and share one
base class, so they behave identically on everything except money and routing: same
`bind_tools`, same `with_structured_output`, same streaming and async, same retry policy,
same usage envelope.

Expand DownExpand Up@@ -93,7 +96,7 @@ except UnknownBackendError as exc:
('openrouter', 'openai/gpt-4o-mini:floor')
('claude-cli', 'anthropic/claude-haiku-4.5')
('openai', 'gpt-4o-mini')
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, ollama, mock — or a bare model name for the claude-cli default
UnknownBackendError: unknown backend 'opnerouter' in spec 'opnerouter/openai/gpt-4o-mini'; expected one of: claude-cli, openrouter, openai, novita, ollama, mock — or a bare model name for the claude-cli default
```

Only the first segment is a backend, because OpenRouter model ids are themselves
Expand DownExpand Up@@ -122,16 +125,18 @@ model through the broker is still `openrouter/openai/gpt-4o-mini`.
<!-- verified: cli -->
```console
$ grapharc models
backends: claude-cli, openrouter, openai, ollama, mock
backends: claude-cli, openrouter, openai, novita, ollama, mock
openrouter key: <unset>
openai key: <unset>
novita key: <unset>
ollama url: http://localhost:11434/v1

examples:
 claude-cli/claude-sonnet-5 subscription, no API key
 openrouter/anthropic/claude-haiku-4.5 many providers, one key
 openrouter/openai/gpt-4o-mini:floor cheapest provider for that model
 openai/gpt-4o-mini the OpenAI API directly, your key
 novita/moonshotai/kimi-k3 Novita's own endpoint, your key
 ollama/llama3.1 a local server, no key and no bill

grapharc models --check probes which of these this machine can use
Expand DownExpand Up@@ -165,6 +170,8 @@ openrouter unusable no API key (set OPENROUTER_API_KEY, or add one to .env)
 credential: <unset>
openai unusable no API key (set OPENAI_API_KEY, or add one to .env)
 credential: <unset>
novita unusable no API key (set NOVITA_API_KEY, or add one to .env)
 credential: <unset>
ollama usable local server at http://localhost:11434/v1
 credential: none needed (local server)
mock usable scripted test double; never reaches a provider
Expand Down

4 changes: 4 additions & 0 deletions
grapharc/cli/main.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -449,6 +449,7 @@ def _cmd_approve(args: argparse.Namespace) -> int:
def _cmd_models(args: argparse.Namespace) -> int:
 from grapharc.gateway import (
 describe,
 novita_api_key,
 ollama_base_url,
 openai_api_key,
 openrouter_api_key,
Expand DownExpand Up@@ -497,6 +498,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "openrouter/anthropic/claude-haiku-4.5": "many providers, one key",
 "openrouter/openai/gpt-4o-mini:floor": "cheapest provider for that model",
 "openai/gpt-4o-mini": "the OpenAI API directly, your key",
 "novita/moonshotai/kimi-k3": "Novita's own endpoint, your key",
 "ollama/llama3.1": "a local server, no key and no bill",
 }
 payload = {
Expand All@@ -505,6 +507,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 "backends": list(BACKENDS),
 "openrouter_key": redact(openrouter_api_key()),
 "openai_key": redact(openai_api_key()),
 "novita_key": redact(novita_api_key()),
 # An address, not a secret: it is printed whole, and it is where a
 # request would go rather than proof that anything is listening.
 "ollama_base_url": ollama_base_url(),
Expand All@@ -516,6 +519,7 @@ def _cmd_models(args: argparse.Namespace) -> int:
 style.kv("backends", ", ".join(BACKENDS)),
 style.kv("openrouter key", redact(openrouter_api_key())),
 style.kv("openai key", redact(openai_api_key())),
 style.kv("novita key", redact(novita_api_key())),
 style.kv("ollama url", ollama_base_url(), tint=style.accent),
 "",
 style.heading("examples:"),
Expand Down

21 changes: 21 additions & 0 deletions
grapharc/cli/probe.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -95,6 +95,26 @@ def _probe_openai() -> dict[str, Any]:
 }


def _probe_novita() -> dict[str, Any]:
 from grapharc.gateway import novita_api_key, redact

 key = novita_api_key()
 has_dependency = importlib.util.find_spec("langchain_openai") is not None
 missing = []
 if not key:
 missing.append("no API key (set NOVITA_API_KEY, or add one to .env)")
 if not has_dependency:
 missing.append("langchain-openai not installed (uv sync --extra novita)")
 return {
 "backend": "novita",
 "kind": KIND_PROVIDER,
 "usable": bool(key) and has_dependency,
 "credential": redact(key),
 "detail": "; ".join(missing) or "api key configured and langchain-openai installed",
 "checked": "credential presence only; no request was sent to api.novita.ai",
 }


def _probe_ollama() -> dict[str, Any]:
 from grapharc.gateway import ollama_base_url

Expand DownExpand Up@@ -152,6 +172,7 @@ def probe_backends(*, claude_path: str = "claude") -> list[dict[str, Any]]:
 "claude-cli": lambda: _probe_claude_cli(claude_path),
 "openrouter": _probe_openrouter,
 "openai": _probe_openai,
 "novita": _probe_novita,
 "ollama": _probe_ollama,
 "mock": _probe_mock,
 }
Expand Down

9 changes: 7 additions & 2 deletions
grapharc/gateway/__init__.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -6,6 +6,7 @@
 get_model("claude-cli/claude-sonnet-5") # subscription, no API key
 get_model("openrouter/anthropic/claude-sonnet-4.5") # many providers, one key
 get_model("openai/gpt-4o-mini") # OPENAI_API_KEY
 get_model("novita/moonshotai/kimi-k3") # NOVITA_API_KEY
 get_model("ollama/llama3.1") # local server, no key
 get_model("mock/x", responses=[...]) # deterministic tests

Expand All@@ -16,12 +17,13 @@
 get_model(spec, cost_ceiling_usd=0.25) # raises when passed
 get_model(spec, spend=shared_meter) # one ceiling, many models

The three OpenAI-wire backends (`openrouter`, `openai`, `ollama`) are imported
lazily — they need `langchain-openai`, which is an optional extra.
The four OpenAI-wire backends (`openrouter`, `openai`, `novita`, `ollama`) are
imported lazily — they need `langchain-openai`, which is an optional extra.
"""

from grapharc.gateway.claude_cli import ClaudeCodeCLIChatModel
from grapharc.gateway.config import (
 novita_api_key,
 ollama_api_key,
 ollama_base_url,
 openai_api_key,
Expand DownExpand Up@@ -66,6 +68,7 @@
 "different_providers",
 "get_model",
 "is_transient",
 "novita_api_key",
 "ollama_api_key",
 "ollama_base_url",
 "openai_api_key",
Expand All@@ -85,6 +88,8 @@
 "OpenRouterError": "openrouter",
 "OpenAIChatModel": "openai",
 "OpenAIError": "openai",
 "NovitaChatModel": "novita",
 "NovitaError": "novita",
 "OllamaChatModel": "ollama",
 "OllamaError": "ollama",
}
Expand Down

16 changes: 15 additions & 1 deletion
grapharc/gateway/config.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -20,14 +20,17 @@
Secrets are returned, never logged. Anything that renders a config for humans
goes through `redact`.

Three key-holding backends, plus one that usually holds none:
Four key-holding backends, plus one that usually holds none:

- **OpenRouter** — `OPENROUTER_API_KEY`.
- **OpenAI** — `OPENAI_API_KEY`, optionally with `OPENAI_BASE_URL` for an
 Azure-style or proxied endpoint. `langchain-openai` reads `OPENAI_API_KEY`
 from the process environment on its own; going through here as well is what
 adds `.env` support, the alternate spellings, and a failure that names the
 variable instead of surfacing an SDK error.
- **Novita** — `NOVITA_API_KEY`. The endpoint is fixed
 (`grapharc.gateway.novita.NOVITA_BASE_URL`), so unlike OpenAI there is no
 base-url override to resolve here.
- **Ollama** — no credential by default: it is a server on your own machine.
 What it needs is an address, so `ollama_base_url()` always returns one
 (`OLLAMA_HOST` / `OLLAMA_BASE_URL`, else localhost). `OLLAMA_API_KEY` exists
Expand DownExpand Up@@ -56,6 +59,13 @@
 "openai_api_key",
)

NOVITA_KEYS = (
 "NOVITA_API_KEY",
 "NOVITA_KEY",
 "novita-api-key",
 "novita_api_key",
)

# `OPENAI_API_BASE` is the older spelling and is still what a lot of tooling
# sets; both are accepted, the current one first.
OPENAI_BASE_URL_KEYS = (
Expand DownExpand Up@@ -133,6 +143,10 @@ def openai_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(OPENAI_KEYS, env_file=env_file)


def novita_api_key(*, env_file: Path | None = None) -> str | None:
 return get_secret(NOVITA_KEYS, env_file=env_file)


def openai_base_url(*, env_file: Path | None = None) -> str | None:
 """An endpoint override, or None for api.openai.com.

Expand Down

59 changes: 59 additions & 0 deletions
grapharc/gateway/novita.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
"""Novita backend — a GPU cloud hosting open-weight models, one key.

`novita/moonshotai/kimi-k3` reaches Novita's own OpenAI-compatible endpoint
(`https://api.novita.ai/openai`), not api.openai.com, so this builds on
`OpenAICompatChatModel` the same way `openrouter.py` and `ollama.py` do rather
than on `openai.py`: the endpoint is fixed, not an override of OpenAI's own.

Model ids on Novita are themselves `author/slug` — `moonshotai/kimi-k3`,
`zai-org/glm-5.2` — the same shape OpenRouter uses, so `vendor()` in
`registry.py` already reads the right author off a Novita spec with no
backend-specific handling: `BACKEND_VENDOR` stays absent for `novita`, exactly
as it is absent for `openrouter`.

**Novita reports no per-call cost.** Unlike OpenRouter, the chat-completions
response carries token counts and nothing else, so this backend is the OpenAI
one in that respect: `_provider_cost` is the base class's `None`, and
`cost_ceiling_usd` counts calls in `SpendMeter.unpriced_calls` unless a caller
supplies `price_per_million=`.
"""

from __future__ import annotations

from typing import Any

from grapharc.gateway.config import novita_api_key
from grapharc.gateway.openai_compat import OpenAICompatChatModel

NOVITA_BASE_URL = "https://api.novita.ai/openai"


class NovitaError(Exception):
 """The Novita backend could not be constructed or used."""


class NovitaChatModel(OpenAICompatChatModel):
 """A LangChain chat model over Novita's OpenAI-compatible endpoint."""

 def __init__(self, model: str, /, **kwargs: Any) -> None:
 api_key = kwargs.pop("api_key", None) or novita_api_key()
 if not api_key:
 raise NovitaError(
 "No Novita API key found. Set NOVITA_API_KEY in the environment, "
 "or add one of NOVITA_API_KEY / novita-api-key to a .env file."
 )
 # One retry layer, not two — same reasoning as the OpenRouter backend.
 kwargs.setdefault("max_retries", 0)
 super().__init__(
 model=model,
 api_key=api_key,
 base_url=kwargs.pop("base_url", None) or NOVITA_BASE_URL,
 **kwargs,
 )

 @property
 def _llm_type(self) -> str:
 return "grapharc-novita"


__all__ = ["NOVITA_BASE_URL", "NovitaChatModel", "NovitaError"]

15 changes: 14 additions & 1 deletion
grapharc/gateway/registry.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -7,6 +7,7 @@
 openrouter/anthropic/claude-sonnet-4.5 -> OpenRouter
 openrouter/openai/gpt-4o:floor -> OpenRouter, cheapest provider
 openai/gpt-4o-mini -> OpenAI directly (OPENAI_API_KEY)
 novita/moonshotai/kimi-k3 -> Novita (NOVITA_API_KEY)
 ollama/llama3.1 -> a local Ollama server, no key
 mock/anything -> scripted test double

Expand All@@ -28,7 +29,7 @@ class UnknownBackendError(Exception):
 """The spec named a backend that is not registered."""


BACKENDS = ("claude-cli", "openrouter", "openai", "ollama", "mock")
BACKENDS = ("claude-cli", "openrouter", "openai", "novita", "ollama", "mock")

# Authors that appear in OpenRouter model ids. A spec starting with one of
# these is a model name, not a mistyped backend — `anthropic/claude-haiku-4.5`
Expand DownExpand Up@@ -71,6 +72,7 @@ class UnknownBackendError(Exception):
_BARE_BACKEND_EXAMPLE = {
 "openrouter": "openrouter/anthropic/claude-sonnet-4.5",
 "openai": "openai/gpt-4o-mini",
 "novita": "novita/moonshotai/kimi-k3",
 "ollama": "ollama/llama3.1",
}

Expand DownExpand Up@@ -160,6 +162,17 @@ def get_model(spec: str, **kwargs: Any) -> BaseChatModel:

 return OpenAIChatModel(model, **kwargs)

 if backend == "novita":
 try:
 from grapharc.gateway.novita import NovitaChatModel
 except ImportError as exc: # pragma: no cover - depends on install extras
 raise UnknownBackendError(
 "The Novita backend needs langchain-openai. "
 "Install it with: uv sync --extra novita"
 ) from exc

 return NovitaChatModel(model, **kwargs)

 if backend == "ollama":
 try:
 from grapharc.gateway.ollama import OllamaChatModel
Expand Down

8 changes: 7 additions & 1 deletion
pyproject.toml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -71,6 +71,12 @@ openrouter = [
openai = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/novita.py. Novita speaks the OpenAI wire format
# against its own endpoint, so this needs the same client and no Novita-specific
# package.
novita = [
 "langchain-openai>=0.2",
]
# Imported by grapharc/gateway/ollama.py. Ollama speaks the OpenAI wire format,
# so the local backend needs the same client and no ollama-specific package.
ollama = [
Expand DownExpand Up@@ -110,7 +116,7 @@ slack = [
]
# Everything above. Self-referential so it cannot drift out of sync.
all = [
 "grapharc[api,ladybug,mcp,ollama,openai,openrouter,otel,server,slack]",
 "grapharc[api,ladybug,mcp,novita,ollama,openai,openrouter,otel,server,slack]",
]

[dependency-groups]
Expand Down

4 changes: 3 additions & 1 deletion
tests/test_cli.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -438,6 +438,7 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 for name in (
 "OPENROUTER_API_KEY", "OPENROUTER_KEY", "open-router-api-key",
 "OPENAI_API_KEY", "OPENAI_KEY", "openai-api-key",
 "NOVITA_API_KEY", "NOVITA_KEY", "novita-api-key",
 "OLLAMA_HOST", "OLLAMA_BASE_URL",
 ):
 monkeypatch.delenv(name, raising=False)
Expand All@@ -452,10 +453,11 @@ def test_models_check_exits_one_when_nothing_is_configured(monkeypatch, capsys):
 "claude-cli": False,
 "openrouter": False,
 "openai": False,
 "novita": False,
 "ollama": False,
 "mock": True,
 }
 for backend in ("openrouter", "openai"):
 for backend in ("openrouter", "openai", "novita"):
 assert next(b for b in payload["backends"] if b["backend"] == backend)[
 "credential"
 ] == "<unset>"
Expand Down
Loading
Loading