feat(agent): Ollama tool-call adapter with parameterized timeouts (fixes #205) by mike-011 · Pull Request #210 · VectifyAI/OpenKB

mike-011 · 2026-07-29T12:14:22Z

Summary

When using Ollama models with openkb query and openkb chat, the openai-agents SDK tool-calling loop fails in three ways:

LiteLLM prompt injection — ollama/ prefix causes LiteLLM to strip tools and inject them into prompt text with format: json. Model returns tool-call JSON in content instead of tool_calls.
Tool-name hallucination — small models call non-existent tools, causing ModelBehaviorError.
Timeouts — local models take 60-130s per turn; default timeout too short.

Fix

1. Model rewrite: `ollama/` to `ollama_chat/`

Primary fix — LiteLLM uses native Ollama Chat API with tool support.

2. Retry on hallucination

Catches ModelBehaviorError, appends corrective message, retries (configurable, default 3).

3. Parameterized timeouts and retries

New ollama: config block in config.yaml:

ollama:
timeout: 300tool_call_retries: 3

4. drop_params + api_base propagation

Testing

56 adapter unit tests, all passing
1131 total tests, 0 regressions
Live E2E: gemma4:12b full tool-loop, 132s

See docs/ollama_tool_call_adapter.md for full documentation.

Add openkb/agent/ollama_adapter.py — a resilient wrapper that catches ModelBehaviorError when Ollama models hallucinate tool names and retries with a corrective message listing the actual available tools. - is_ollama_backend(): detect Ollama models (ollama/ and litellm/ollama/ prefixes) - arun_with_retry(): wraps Runner.run() with retry on ModelBehaviorError - run_streamed_with_retry(): wraps Runner.run_streamed() with retry - _RetryableStreamResult: seamless stream retry via generator - 26 new tests in tests/test_ollama_adapter.py (all passing) - query.py patched to use retry wrapper for Ollama backends only - Non-Ollama path unchanged Issue: VectifyAI#205

…I#205

Key improvements over v1: 1. rewrite_ollama_model(): convert ollama/ → ollama_chat/ so LiteLLM uses native Ollama Chat API with tool support (primary fix). Without this, LiteLLM falls back to prompt injection and tool calls never work through the SDK. 2. Configurable timeout: _ensure_ollama_timeout() injects 300s default into ModelSettings.extra_args for Ollama models, preventing premature timeouts on slow local hardware. 3. Updated query.py: rewrite applied in build_query_agent() and build_run_config_from_bundle(). 4. 41 tests (up from 26), all passing. 5. 1116 existing tests, all passing. 6. Live test: gemma4:12b full tool-loop completes in 132s. Issue: VectifyAI#205

Fixes for the full add+query pipeline with Ollama: 1. Module-level OLLAMA_API_BASE propagation in ollama_adapter.py: - OPENAI_API_BASE → OLLAMA_API_BASE (env var) - litellm.api_base module setting So ollama_chat LiteLLM provider finds the endpoint on all paths. 2. compiler.py: pass api_base from env when bundle is None (CLI path) so litellm.completion/acompletion reaches Ollama for add/concepts. 3. cli.py: rewrite_ollama_model applied to all model resolutions (12 sites) so ollama/ → ollama_chat/ everywhere, not just query. 4. add_coordinator.py: import ollama_adapter for side effects. 5. drop_params=True set in _ensure_ollama_settings for ollama_chat (rejects parallel_tool_calls). Tests: 1116 passed, 0 failed. Live: add summary=77s OK; query tool-loop=132s OK (SDK E2E verified). Issue: VectifyAI#205

Add resolve_ollama_settings() to read the 'ollama:' config block: ollama: timeout: 300 # per-request timeout (s), default 300 tool_call_retries: 3 # max retries on hallucinated tool names Timeout precedence: ollama.timeout > litellm.timeout > top-level timeout > 300 Retry precedence: ollama.tool_call_retries > default 3 Changes: - ollama_adapter.py: resolve_ollama_settings() + all retry/timeout functions now resolve from config when not explicitly passed - config.yaml.example: documented ollama: section - docs/ollama_tool_call_adapter.md: full configuration documentation - tests/test_ollama_adapter.py: 15 new tests for resolve_ollama_settings (56 total, all passing; 1131 total suite, 0 failures) Issue: VectifyAI#205

mike-011 · 2026-07-29T12:14:22Z

Summary

Fixes#205

When using Ollama models with openkb query and openkb chat, the openai-agents SDK tool-calling loop fails in three ways:

LiteLLM prompt injection — ollama/ prefix causes LiteLLM to strip tools and inject them into prompt text with format: json. Model returns tool-call JSON in content instead of tool_calls.
Tool-name hallucination — small models call non-existent tools, causing ModelBehaviorError.
Timeouts — local models take 60-130s per turn; default timeout too short.

Fix

1. Model rewrite: `ollama/` to `ollama_chat/`

Primary fix — LiteLLM uses native Ollama Chat API with tool support.

2. Retry on hallucination

Catches ModelBehaviorError, appends corrective message, retries (configurable, default 3).

3. Parameterized timeouts and retries

New ollama: config block in config.yaml:

ollama:
timeout: 300tool_call_retries: 3

4. drop_params + api_base propagation

Testing

56 adapter unit tests, all passing
1131 total tests, 0 regressions
Live E2E: gemma4:12b full tool-loop, 132s

See docs/ollama_tool_call_adapter.md for full documentation.

Add openkb/agent/ollama_adapter.py — a resilient wrapper that catches ModelBehaviorError when Ollama models hallucinate tool names and retries with a corrective message listing the actual available tools. - is_ollama_backend(): detect Ollama models (ollama/ and litellm/ollama/ prefixes) - arun_with_retry(): wraps Runner.run() with retry on ModelBehaviorError - run_streamed_with_retry(): wraps Runner.run_streamed() with retry - _RetryableStreamResult: seamless stream retry via generator - 26 new tests in tests/test_ollama_adapter.py (all passing) - query.py patched to use retry wrapper for Ollama backends only - Non-Ollama path unchanged Issue: VectifyAI#205

…I#205

Key improvements over v1: 1. rewrite_ollama_model(): convert ollama/ → ollama_chat/ so LiteLLM uses native Ollama Chat API with tool support (primary fix). Without this, LiteLLM falls back to prompt injection and tool calls never work through the SDK. 2. Configurable timeout: _ensure_ollama_timeout() injects 300s default into ModelSettings.extra_args for Ollama models, preventing premature timeouts on slow local hardware. 3. Updated query.py: rewrite applied in build_query_agent() and build_run_config_from_bundle(). 4. 41 tests (up from 26), all passing. 5. 1116 existing tests, all passing. 6. Live test: gemma4:12b full tool-loop completes in 132s. Issue: VectifyAI#205

Fixes for the full add+query pipeline with Ollama: 1. Module-level OLLAMA_API_BASE propagation in ollama_adapter.py: - OPENAI_API_BASE → OLLAMA_API_BASE (env var) - litellm.api_base module setting So ollama_chat LiteLLM provider finds the endpoint on all paths. 2. compiler.py: pass api_base from env when bundle is None (CLI path) so litellm.completion/acompletion reaches Ollama for add/concepts. 3. cli.py: rewrite_ollama_model applied to all model resolutions (12 sites) so ollama/ → ollama_chat/ everywhere, not just query. 4. add_coordinator.py: import ollama_adapter for side effects. 5. drop_params=True set in _ensure_ollama_settings for ollama_chat (rejects parallel_tool_calls). Tests: 1116 passed, 0 failed. Live: add summary=77s OK; query tool-loop=132s OK (SDK E2E verified). Issue: VectifyAI#205

Add resolve_ollama_settings() to read the 'ollama:' config block: ollama: timeout: 300 # per-request timeout (s), default 300 tool_call_retries: 3 # max retries on hallucinated tool names Timeout precedence: ollama.timeout > litellm.timeout > top-level timeout > 300 Retry precedence: ollama.tool_call_retries > default 3 Changes: - ollama_adapter.py: resolve_ollama_settings() + all retry/timeout functions now resolve from config when not explicitly passed - config.yaml.example: documented ollama: section - docs/ollama_tool_call_adapter.md: full configuration documentation - tests/test_ollama_adapter.py: 15 new tests for resolve_ollama_settings (56 total, all passing; 1131 total suite, 0 failures) Issue: VectifyAI#205

mike-011 · 2026-07-29T12:14:22Z

Summary

Fixes#205

When using Ollama models with openkb query and openkb chat, the openai-agents SDK tool-calling loop fails in three ways:

LiteLLM prompt injection — ollama/ prefix causes LiteLLM to strip tools and inject them into prompt text with format: json. Model returns tool-call JSON in content instead of tool_calls.
Tool-name hallucination — small models call non-existent tools, causing ModelBehaviorError.
Timeouts — local models take 60-130s per turn; default timeout too short.

Fix

1. Model rewrite: `ollama/` to `ollama_chat/`

Primary fix — LiteLLM uses native Ollama Chat API with tool support.

2. Retry on hallucination

Catches ModelBehaviorError, appends corrective message, retries (configurable, default 3).

3. Parameterized timeouts and retries

New ollama: config block in config.yaml:

ollama:
timeout: 300tool_call_retries: 3

4. drop_params + api_base propagation

Testing

56 adapter unit tests, all passing
1131 total tests, 0 regressions
Live E2E: gemma4:12b full tool-loop, 132s

See docs/ollama_tool_call_adapter.md for full documentation.

Add openkb/agent/ollama_adapter.py — a resilient wrapper that catches ModelBehaviorError when Ollama models hallucinate tool names and retries with a corrective message listing the actual available tools. - is_ollama_backend(): detect Ollama models (ollama/ and litellm/ollama/ prefixes) - arun_with_retry(): wraps Runner.run() with retry on ModelBehaviorError - run_streamed_with_retry(): wraps Runner.run_streamed() with retry - _RetryableStreamResult: seamless stream retry via generator - 26 new tests in tests/test_ollama_adapter.py (all passing) - query.py patched to use retry wrapper for Ollama backends only - Non-Ollama path unchanged Issue: VectifyAI#205

…I#205

Key improvements over v1: 1. rewrite_ollama_model(): convert ollama/ → ollama_chat/ so LiteLLM uses native Ollama Chat API with tool support (primary fix). Without this, LiteLLM falls back to prompt injection and tool calls never work through the SDK. 2. Configurable timeout: _ensure_ollama_timeout() injects 300s default into ModelSettings.extra_args for Ollama models, preventing premature timeouts on slow local hardware. 3. Updated query.py: rewrite applied in build_query_agent() and build_run_config_from_bundle(). 4. 41 tests (up from 26), all passing. 5. 1116 existing tests, all passing. 6. Live test: gemma4:12b full tool-loop completes in 132s. Issue: VectifyAI#205

Fixes for the full add+query pipeline with Ollama: 1. Module-level OLLAMA_API_BASE propagation in ollama_adapter.py: - OPENAI_API_BASE → OLLAMA_API_BASE (env var) - litellm.api_base module setting So ollama_chat LiteLLM provider finds the endpoint on all paths. 2. compiler.py: pass api_base from env when bundle is None (CLI path) so litellm.completion/acompletion reaches Ollama for add/concepts. 3. cli.py: rewrite_ollama_model applied to all model resolutions (12 sites) so ollama/ → ollama_chat/ everywhere, not just query. 4. add_coordinator.py: import ollama_adapter for side effects. 5. drop_params=True set in _ensure_ollama_settings for ollama_chat (rejects parallel_tool_calls). Tests: 1116 passed, 0 failed. Live: add summary=77s OK; query tool-loop=132s OK (SDK E2E verified). Issue: VectifyAI#205

Add resolve_ollama_settings() to read the 'ollama:' config block: ollama: timeout: 300 # per-request timeout (s), default 300 tool_call_retries: 3 # max retries on hallucinated tool names Timeout precedence: ollama.timeout > litellm.timeout > top-level timeout > 300 Retry precedence: ollama.tool_call_retries > default 3 Changes: - ollama_adapter.py: resolve_ollama_settings() + all retry/timeout functions now resolve from config when not explicitly passed - config.yaml.example: documented ollama: section - docs/ollama_tool_call_adapter.md: full configuration documentation - tests/test_ollama_adapter.py: 15 new tests for resolve_ollama_settings (56 total, all passing; 1131 total suite, 0 failures) Issue: VectifyAI#205

mike-011 · 2026-07-29T12:14:22Z

Summary

Fixes#205

When using Ollama models with openkb query and openkb chat, the openai-agents SDK tool-calling loop fails in three ways:

LiteLLM prompt injection — ollama/ prefix causes LiteLLM to strip tools and inject them into prompt text with format: json. Model returns tool-call JSON in content instead of tool_calls.
Tool-name hallucination — small models call non-existent tools, causing ModelBehaviorError.
Timeouts — local models take 60-130s per turn; default timeout too short.

Fix

1. Model rewrite: `ollama/` to `ollama_chat/`

Primary fix — LiteLLM uses native Ollama Chat API with tool support.

2. Retry on hallucination

Catches ModelBehaviorError, appends corrective message, retries (configurable, default 3).

3. Parameterized timeouts and retries

New ollama: config block in config.yaml:

ollama:
timeout: 300tool_call_retries: 3

4. drop_params + api_base propagation

Testing

56 adapter unit tests, all passing
1131 total tests, 0 regressions
Live E2E: gemma4:12b full tool-loop, 132s

See docs/ollama_tool_call_adapter.md for full documentation.

Add openkb/agent/ollama_adapter.py — a resilient wrapper that catches ModelBehaviorError when Ollama models hallucinate tool names and retries with a corrective message listing the actual available tools. - is_ollama_backend(): detect Ollama models (ollama/ and litellm/ollama/ prefixes) - arun_with_retry(): wraps Runner.run() with retry on ModelBehaviorError - run_streamed_with_retry(): wraps Runner.run_streamed() with retry - _RetryableStreamResult: seamless stream retry via generator - 26 new tests in tests/test_ollama_adapter.py (all passing) - query.py patched to use retry wrapper for Ollama backends only - Non-Ollama path unchanged Issue: VectifyAI#205

…I#205

Key improvements over v1: 1. rewrite_ollama_model(): convert ollama/ → ollama_chat/ so LiteLLM uses native Ollama Chat API with tool support (primary fix). Without this, LiteLLM falls back to prompt injection and tool calls never work through the SDK. 2. Configurable timeout: _ensure_ollama_timeout() injects 300s default into ModelSettings.extra_args for Ollama models, preventing premature timeouts on slow local hardware. 3. Updated query.py: rewrite applied in build_query_agent() and build_run_config_from_bundle(). 4. 41 tests (up from 26), all passing. 5. 1116 existing tests, all passing. 6. Live test: gemma4:12b full tool-loop completes in 132s. Issue: VectifyAI#205

Fixes for the full add+query pipeline with Ollama: 1. Module-level OLLAMA_API_BASE propagation in ollama_adapter.py: - OPENAI_API_BASE → OLLAMA_API_BASE (env var) - litellm.api_base module setting So ollama_chat LiteLLM provider finds the endpoint on all paths. 2. compiler.py: pass api_base from env when bundle is None (CLI path) so litellm.completion/acompletion reaches Ollama for add/concepts. 3. cli.py: rewrite_ollama_model applied to all model resolutions (12 sites) so ollama/ → ollama_chat/ everywhere, not just query. 4. add_coordinator.py: import ollama_adapter for side effects. 5. drop_params=True set in _ensure_ollama_settings for ollama_chat (rejects parallel_tool_calls). Tests: 1116 passed, 0 failed. Live: add summary=77s OK; query tool-loop=132s OK (SDK E2E verified). Issue: VectifyAI#205

Add resolve_ollama_settings() to read the 'ollama:' config block: ollama: timeout: 300 # per-request timeout (s), default 300 tool_call_retries: 3 # max retries on hallucinated tool names Timeout precedence: ollama.timeout > litellm.timeout > top-level timeout > 300 Retry precedence: ollama.tool_call_retries > default 3 Changes: - ollama_adapter.py: resolve_ollama_settings() + all retry/timeout functions now resolve from config when not explicitly passed - config.yaml.example: documented ollama: section - docs/ollama_tool_call_adapter.md: full configuration documentation - tests/test_ollama_adapter.py: 15 new tests for resolve_ollama_settings (56 total, all passing; 1131 total suite, 0 failures) Issue: VectifyAI#205

mike-011 · 2026-07-29T12:14:22Z

Summary

Fixes#205

When using Ollama models with openkb query and openkb chat, the openai-agents SDK tool-calling loop fails in three ways:

LiteLLM prompt injection — ollama/ prefix causes LiteLLM to strip tools and inject them into prompt text with format: json. Model returns tool-call JSON in content instead of tool_calls.
Tool-name hallucination — small models call non-existent tools, causing ModelBehaviorError.
Timeouts — local models take 60-130s per turn; default timeout too short.

Fix

1. Model rewrite: `ollama/` to `ollama_chat/`

Primary fix — LiteLLM uses native Ollama Chat API with tool support.

2. Retry on hallucination

Catches ModelBehaviorError, appends corrective message, retries (configurable, default 3).

3. Parameterized timeouts and retries

New ollama: config block in config.yaml:

ollama:
timeout: 300tool_call_retries: 3

4. drop_params + api_base propagation

Testing

56 adapter unit tests, all passing
1131 total tests, 0 regressions
Live E2E: gemma4:12b full tool-loop, 132s

See docs/ollama_tool_call_adapter.md for full documentation.

Add openkb/agent/ollama_adapter.py — a resilient wrapper that catches ModelBehaviorError when Ollama models hallucinate tool names and retries with a corrective message listing the actual available tools. - is_ollama_backend(): detect Ollama models (ollama/ and litellm/ollama/ prefixes) - arun_with_retry(): wraps Runner.run() with retry on ModelBehaviorError - run_streamed_with_retry(): wraps Runner.run_streamed() with retry - _RetryableStreamResult: seamless stream retry via generator - 26 new tests in tests/test_ollama_adapter.py (all passing) - query.py patched to use retry wrapper for Ollama backends only - Non-Ollama path unchanged Issue: VectifyAI#205

…I#205

Key improvements over v1: 1. rewrite_ollama_model(): convert ollama/ → ollama_chat/ so LiteLLM uses native Ollama Chat API with tool support (primary fix). Without this, LiteLLM falls back to prompt injection and tool calls never work through the SDK. 2. Configurable timeout: _ensure_ollama_timeout() injects 300s default into ModelSettings.extra_args for Ollama models, preventing premature timeouts on slow local hardware. 3. Updated query.py: rewrite applied in build_query_agent() and build_run_config_from_bundle(). 4. 41 tests (up from 26), all passing. 5. 1116 existing tests, all passing. 6. Live test: gemma4:12b full tool-loop completes in 132s. Issue: VectifyAI#205

Fixes for the full add+query pipeline with Ollama: 1. Module-level OLLAMA_API_BASE propagation in ollama_adapter.py: - OPENAI_API_BASE → OLLAMA_API_BASE (env var) - litellm.api_base module setting So ollama_chat LiteLLM provider finds the endpoint on all paths. 2. compiler.py: pass api_base from env when bundle is None (CLI path) so litellm.completion/acompletion reaches Ollama for add/concepts. 3. cli.py: rewrite_ollama_model applied to all model resolutions (12 sites) so ollama/ → ollama_chat/ everywhere, not just query. 4. add_coordinator.py: import ollama_adapter for side effects. 5. drop_params=True set in _ensure_ollama_settings for ollama_chat (rejects parallel_tool_calls). Tests: 1116 passed, 0 failed. Live: add summary=77s OK; query tool-loop=132s OK (SDK E2E verified). Issue: VectifyAI#205

Add resolve_ollama_settings() to read the 'ollama:' config block: ollama: timeout: 300 # per-request timeout (s), default 300 tool_call_retries: 3 # max retries on hallucinated tool names Timeout precedence: ollama.timeout > litellm.timeout > top-level timeout > 300 Retry precedence: ollama.tool_call_retries > default 3 Changes: - ollama_adapter.py: resolve_ollama_settings() + all retry/timeout functions now resolve from config when not explicitly passed - config.yaml.example: documented ollama: section - docs/ollama_tool_call_adapter.md: full configuration documentation - tests/test_ollama_adapter.py: 15 new tests for resolve_ollama_settings (56 total, all passing; 1131 total suite, 0 failures) Issue: VectifyAI#205

mike-011 · 2026-07-29T12:14:22Z

Summary

Fixes#205

When using Ollama models with openkb query and openkb chat, the openai-agents SDK tool-calling loop fails in three ways:

LiteLLM prompt injection — ollama/ prefix causes LiteLLM to strip tools and inject them into prompt text with format: json. Model returns tool-call JSON in content instead of tool_calls.
Tool-name hallucination — small models call non-existent tools, causing ModelBehaviorError.
Timeouts — local models take 60-130s per turn; default timeout too short.

Fix

1. Model rewrite: `ollama/` to `ollama_chat/`

Primary fix — LiteLLM uses native Ollama Chat API with tool support.

2. Retry on hallucination

Catches ModelBehaviorError, appends corrective message, retries (configurable, default 3).

3. Parameterized timeouts and retries

New ollama: config block in config.yaml:

ollama:
timeout: 300tool_call_retries: 3

4. drop_params + api_base propagation

Testing

56 adapter unit tests, all passing
1131 total tests, 0 regressions
Live E2E: gemma4:12b full tool-loop, 132s

See docs/ollama_tool_call_adapter.md for full documentation.

Add openkb/agent/ollama_adapter.py — a resilient wrapper that catches ModelBehaviorError when Ollama models hallucinate tool names and retries with a corrective message listing the actual available tools. - is_ollama_backend(): detect Ollama models (ollama/ and litellm/ollama/ prefixes) - arun_with_retry(): wraps Runner.run() with retry on ModelBehaviorError - run_streamed_with_retry(): wraps Runner.run_streamed() with retry - _RetryableStreamResult: seamless stream retry via generator - 26 new tests in tests/test_ollama_adapter.py (all passing) - query.py patched to use retry wrapper for Ollama backends only - Non-Ollama path unchanged Issue: VectifyAI#205

…I#205

Key improvements over v1: 1. rewrite_ollama_model(): convert ollama/ → ollama_chat/ so LiteLLM uses native Ollama Chat API with tool support (primary fix). Without this, LiteLLM falls back to prompt injection and tool calls never work through the SDK. 2. Configurable timeout: _ensure_ollama_timeout() injects 300s default into ModelSettings.extra_args for Ollama models, preventing premature timeouts on slow local hardware. 3. Updated query.py: rewrite applied in build_query_agent() and build_run_config_from_bundle(). 4. 41 tests (up from 26), all passing. 5. 1116 existing tests, all passing. 6. Live test: gemma4:12b full tool-loop completes in 132s. Issue: VectifyAI#205

Fixes for the full add+query pipeline with Ollama: 1. Module-level OLLAMA_API_BASE propagation in ollama_adapter.py: - OPENAI_API_BASE → OLLAMA_API_BASE (env var) - litellm.api_base module setting So ollama_chat LiteLLM provider finds the endpoint on all paths. 2. compiler.py: pass api_base from env when bundle is None (CLI path) so litellm.completion/acompletion reaches Ollama for add/concepts. 3. cli.py: rewrite_ollama_model applied to all model resolutions (12 sites) so ollama/ → ollama_chat/ everywhere, not just query. 4. add_coordinator.py: import ollama_adapter for side effects. 5. drop_params=True set in _ensure_ollama_settings for ollama_chat (rejects parallel_tool_calls). Tests: 1116 passed, 0 failed. Live: add summary=77s OK; query tool-loop=132s OK (SDK E2E verified). Issue: VectifyAI#205

Add resolve_ollama_settings() to read the 'ollama:' config block: ollama: timeout: 300 # per-request timeout (s), default 300 tool_call_retries: 3 # max retries on hallucinated tool names Timeout precedence: ollama.timeout > litellm.timeout > top-level timeout > 300 Retry precedence: ollama.tool_call_retries > default 3 Changes: - ollama_adapter.py: resolve_ollama_settings() + all retry/timeout functions now resolve from config when not explicitly passed - config.yaml.example: documented ollama: section - docs/ollama_tool_call_adapter.md: full configuration documentation - tests/test_ollama_adapter.py: 15 new tests for resolve_ollama_settings (56 total, all passing; 1131 total suite, 0 failures) Issue: VectifyAI#205

mike-011 · 2026-07-29T12:14:22Z

Summary

Fixes#205

When using Ollama models with openkb query and openkb chat, the openai-agents SDK tool-calling loop fails in three ways:

LiteLLM prompt injection — ollama/ prefix causes LiteLLM to strip tools and inject them into prompt text with format: json. Model returns tool-call JSON in content instead of tool_calls.
Tool-name hallucination — small models call non-existent tools, causing ModelBehaviorError.
Timeouts — local models take 60-130s per turn; default timeout too short.

Fix

1. Model rewrite: `ollama/` to `ollama_chat/`

Primary fix — LiteLLM uses native Ollama Chat API with tool support.

2. Retry on hallucination

Catches ModelBehaviorError, appends corrective message, retries (configurable, default 3).

3. Parameterized timeouts and retries

New ollama: config block in config.yaml:

ollama:
timeout: 300tool_call_retries: 3

4. drop_params + api_base propagation

Testing

56 adapter unit tests, all passing
1131 total tests, 0 regressions
Live E2E: gemma4:12b full tool-loop, 132s

See docs/ollama_tool_call_adapter.md for full documentation.

Add openkb/agent/ollama_adapter.py — a resilient wrapper that catches ModelBehaviorError when Ollama models hallucinate tool names and retries with a corrective message listing the actual available tools. - is_ollama_backend(): detect Ollama models (ollama/ and litellm/ollama/ prefixes) - arun_with_retry(): wraps Runner.run() with retry on ModelBehaviorError - run_streamed_with_retry(): wraps Runner.run_streamed() with retry - _RetryableStreamResult: seamless stream retry via generator - 26 new tests in tests/test_ollama_adapter.py (all passing) - query.py patched to use retry wrapper for Ollama backends only - Non-Ollama path unchanged Issue: VectifyAI#205

…I#205

Key improvements over v1: 1. rewrite_ollama_model(): convert ollama/ → ollama_chat/ so LiteLLM uses native Ollama Chat API with tool support (primary fix). Without this, LiteLLM falls back to prompt injection and tool calls never work through the SDK. 2. Configurable timeout: _ensure_ollama_timeout() injects 300s default into ModelSettings.extra_args for Ollama models, preventing premature timeouts on slow local hardware. 3. Updated query.py: rewrite applied in build_query_agent() and build_run_config_from_bundle(). 4. 41 tests (up from 26), all passing. 5. 1116 existing tests, all passing. 6. Live test: gemma4:12b full tool-loop completes in 132s. Issue: VectifyAI#205

Fixes for the full add+query pipeline with Ollama: 1. Module-level OLLAMA_API_BASE propagation in ollama_adapter.py: - OPENAI_API_BASE → OLLAMA_API_BASE (env var) - litellm.api_base module setting So ollama_chat LiteLLM provider finds the endpoint on all paths. 2. compiler.py: pass api_base from env when bundle is None (CLI path) so litellm.completion/acompletion reaches Ollama for add/concepts. 3. cli.py: rewrite_ollama_model applied to all model resolutions (12 sites) so ollama/ → ollama_chat/ everywhere, not just query. 4. add_coordinator.py: import ollama_adapter for side effects. 5. drop_params=True set in _ensure_ollama_settings for ollama_chat (rejects parallel_tool_calls). Tests: 1116 passed, 0 failed. Live: add summary=77s OK; query tool-loop=132s OK (SDK E2E verified). Issue: VectifyAI#205

Add resolve_ollama_settings() to read the 'ollama:' config block: ollama: timeout: 300 # per-request timeout (s), default 300 tool_call_retries: 3 # max retries on hallucinated tool names Timeout precedence: ollama.timeout > litellm.timeout > top-level timeout > 300 Retry precedence: ollama.tool_call_retries > default 3 Changes: - ollama_adapter.py: resolve_ollama_settings() + all retry/timeout functions now resolve from config when not explicitly passed - config.yaml.example: documented ollama: section - docs/ollama_tool_call_adapter.md: full configuration documentation - tests/test_ollama_adapter.py: 15 new tests for resolve_ollama_settings (56 total, all passing; 1131 total suite, 0 failures) Issue: VectifyAI#205

mike-011 · 2026-07-29T12:14:22Z

Summary

Fixes#205

When using Ollama models with openkb query and openkb chat, the openai-agents SDK tool-calling loop fails in three ways:

LiteLLM prompt injection — ollama/ prefix causes LiteLLM to strip tools and inject them into prompt text with format: json. Model returns tool-call JSON in content instead of tool_calls.
Tool-name hallucination — small models call non-existent tools, causing ModelBehaviorError.
Timeouts — local models take 60-130s per turn; default timeout too short.

Fix

1. Model rewrite: `ollama/` to `ollama_chat/`

Primary fix — LiteLLM uses native Ollama Chat API with tool support.

2. Retry on hallucination

Catches ModelBehaviorError, appends corrective message, retries (configurable, default 3).

3. Parameterized timeouts and retries

New ollama: config block in config.yaml:

ollama:
timeout: 300tool_call_retries: 3

4. drop_params + api_base propagation

Testing

56 adapter unit tests, all passing
1131 total tests, 0 regressions
Live E2E: gemma4:12b full tool-loop, 132s

See docs/ollama_tool_call_adapter.md for full documentation.

Add openkb/agent/ollama_adapter.py — a resilient wrapper that catches ModelBehaviorError when Ollama models hallucinate tool names and retries with a corrective message listing the actual available tools. - is_ollama_backend(): detect Ollama models (ollama/ and litellm/ollama/ prefixes) - arun_with_retry(): wraps Runner.run() with retry on ModelBehaviorError - run_streamed_with_retry(): wraps Runner.run_streamed() with retry - _RetryableStreamResult: seamless stream retry via generator - 26 new tests in tests/test_ollama_adapter.py (all passing) - query.py patched to use retry wrapper for Ollama backends only - Non-Ollama path unchanged Issue: VectifyAI#205

…I#205

Key improvements over v1: 1. rewrite_ollama_model(): convert ollama/ → ollama_chat/ so LiteLLM uses native Ollama Chat API with tool support (primary fix). Without this, LiteLLM falls back to prompt injection and tool calls never work through the SDK. 2. Configurable timeout: _ensure_ollama_timeout() injects 300s default into ModelSettings.extra_args for Ollama models, preventing premature timeouts on slow local hardware. 3. Updated query.py: rewrite applied in build_query_agent() and build_run_config_from_bundle(). 4. 41 tests (up from 26), all passing. 5. 1116 existing tests, all passing. 6. Live test: gemma4:12b full tool-loop completes in 132s. Issue: VectifyAI#205

Fixes for the full add+query pipeline with Ollama: 1. Module-level OLLAMA_API_BASE propagation in ollama_adapter.py: - OPENAI_API_BASE → OLLAMA_API_BASE (env var) - litellm.api_base module setting So ollama_chat LiteLLM provider finds the endpoint on all paths. 2. compiler.py: pass api_base from env when bundle is None (CLI path) so litellm.completion/acompletion reaches Ollama for add/concepts. 3. cli.py: rewrite_ollama_model applied to all model resolutions (12 sites) so ollama/ → ollama_chat/ everywhere, not just query. 4. add_coordinator.py: import ollama_adapter for side effects. 5. drop_params=True set in _ensure_ollama_settings for ollama_chat (rejects parallel_tool_calls). Tests: 1116 passed, 0 failed. Live: add summary=77s OK; query tool-loop=132s OK (SDK E2E verified). Issue: VectifyAI#205

Add resolve_ollama_settings() to read the 'ollama:' config block: ollama: timeout: 300 # per-request timeout (s), default 300 tool_call_retries: 3 # max retries on hallucinated tool names Timeout precedence: ollama.timeout > litellm.timeout > top-level timeout > 300 Retry precedence: ollama.tool_call_retries > default 3 Changes: - ollama_adapter.py: resolve_ollama_settings() + all retry/timeout functions now resolve from config when not explicitly passed - config.yaml.example: documented ollama: section - docs/ollama_tool_call_adapter.md: full configuration documentation - tests/test_ollama_adapter.py: 15 new tests for resolve_ollama_settings (56 total, all passing; 1131 total suite, 0 failures) Issue: VectifyAI#205

mike-011 added 5 commits July 29, 2026 15:13

docs: Ollama tool-call retry adapter documentation for issueVectifyA…
3a50868
…I#205

mike-011 changed the title ~~feat(agent): Ollama tool-call retry adapter (fixes #205)~~feat(agent): Ollama tool-call adapter with parameterized timeouts (fixes #205)Jul 29, 2026