Skip to content

feat: support llm_config BYOK on jobs and retrieval - #33

Merged
suguanYang merged 5 commits into
mainfrom
feat/suguanYang/byok-llm-credentials
Jul 15, 2026
Merged

feat: support llm_config BYOK on jobs and retrieval#33
suguanYang merged 5 commits into
mainfrom
feat/suguanYang/byok-llm-credentials

Conversation

@suguanYang

@suguanYangsuguanYang commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add LLMConfig / LLMProviderConfig / LLMModelsConfig and thread llm_config through jobs / parse / retrieval.
  • Flat root {api_key, model, base_url} for multimodal; models for same-endpoint different model ids; text / vision for different endpoints.

Depends on: Ontos-AI/knowhere#217

Usage

# One multimodal modelllm_config= {
"api_key": "sk-...",
"model": "gpt-4o",
"base_url": "https://api.openai.com/v1",
}
# Same endpoint, different modelsllm_config= {
"api_key": "sk-...",
"base_url": "https://api.openai.com/v1",
"models": {"text": "gpt-4o-mini", "vision": "gpt-4o"},
}
# Two different endpointsllm_config= {
"text": {"api_key": "sk-...", "model": "gpt-4o-mini", "base_url": "https://api.openai.com/v1"},
"vision": {"api_key": "sk-ali-...", "model": "qwen-vl-max", "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1"},
}
client.parse(url="https://example.com/doc.pdf", llm_config=llm_config)
client.retrieval.query(query="...", use_agentic=True, llm_config=llm_config)

Test plan

  • pytest tests/test_jobs.py tests/test_retrieval.py tests/test_parse.py

Expose optional text/vision OpenAI-compatible credentials on job create,
parse, and retrieval.query so callers can drive KNOWHERE with their own keys.
Co-authored-by: Cursor <cursoragent@cursor.com>
suguanYangand others added 4 commits July 15, 2026 21:39
Document that missing text/vision slots keep server defaults instead of
acting as a unified multimodal fallback.
Co-authored-by: Cursor <cursoragent@cursor.com>
Document and type the shared provider slot for single multimodal BYOK.
Co-authored-by: Cursor <cursoragent@cursor.com>
Document flat multimodal shorthand and split text/vision endpoints.
Co-authored-by: Cursor <cursoragent@cursor.com>
Support shared auth with different text/vision model ids.
Co-authored-by: Cursor <cursoragent@cursor.com>
@suguanYang
suguanYang merged commit 51adb5a into mainJul 15, 2026
5 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@suguanYang