Uh oh!
There was an error while loading. Please reload this page.
fix(search): hide unready Tavily WebSearch - #3168
Conversation
📝 WalkthroughProblem solvedThis PR hides Provider-native search remains controlled by the selected model’s hosted-search capability. It does not require Tavily credentials. The call-time fail-closed guard remains in place. Source of truthThe PR extends the existing Readiness flows through Scope and complexityThe solution is the smallest coherent change shown by the diff. The added readiness resolver, composition plumbing, and routing input are required to apply the existing policy before tools reach the model. No behavior or regression coverage is an obvious candidate for deletion. Obsolete child-agent sentinel and validation logic was removed. Existing native, external, disabled, incognito, provider, root, child-scope, and ValidationReported validation includes:
Broader checks reported unrelated UI interface mismatches, sandbox limitations, and unrelated test failures. Required-check status is not independently verified here. Review-relevant risks
The person performing the merge must review the final diff. A maintainer makes the final determination. WalkthroughThe runtime now resolves Tavily readiness before composing web-search tools. Unavailable Tavily configurations omit ChangesWeb-search readiness routing
Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk:⚪ Minimal · up to This change hides Tavily search when its credentials are unavailable while preserving native model search and call-time safeguards. No actionable merge-blocking risk remains after normal checks and review. Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant RuntimePolicyOperationCoordinator
participant ExecutionComposition
participant InteractiveRunComposer
participant Model
RuntimePolicyOperationCoordinator->>ExecutionComposition: Resolve Tavily execution state
ExecutionComposition->>InteractiveRunComposer: Pass runtime policy and tavilyReady
InteractiveRunComposer->>InteractiveRunComposer: Route host, child, and parent-agent tools
InteractiveRunComposer-->>Model: Return filtered tool surface
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
PR Summary by QodoHide Tavily WebSearch when runtime policy says it’s not runnable
AI Description
Diagram
High-Level Assessment
Files changed (7) |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 0256d7d9-107b-4055-9656-37230860edba
📒 Files selected for processing (7)
packages/runtime-host/src/__tests__/execution-model-composition.test.tspackages/runtime-host/src/__tests__/web-search-tool.test.tspackages/runtime-host/src/server/execution-composition.tspackages/runtime-host/src/server/interactive-run-composer.tspackages/runtime-host/src/server/web-search-tool.tspackages/runtime/src/__tests__/native-web-search-tool.test.tspackages/runtime/src/native-web-search-tool.ts
Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Pull request overview
This PR prevents the Tavily-backed WebSearch tool (and dependent child-agent web_research profile) from being advertised to the model when the canonical runtime-policy resolver reports Tavily is unavailable, avoiding wasted model turns on guaranteed-to-fail tool calls.
Changes:
- Adds a
tavilyReadyinput torouteWebSearchTools()and hides the client-executedWebSearchwhen Tavily is not canonicallyready. - Introduces a host-side
resolveHostTavilyWebSearchReadiness()helper and threads readiness through root/bound/child tool routing in the runtime host. - Updates runtime and host tests to validate tool-surface omission for unready Tavily and removal of
web_researchwhen Tavily can’t run.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/runtime/src/native-web-search-tool.ts | Adds tavilyReady gating so Tavily WebSearch is only exposed when canonically ready. |
| packages/runtime/src/tests/native-web-search-tool.test.ts | Adds coverage for Tavily-unready routing behavior. |
| packages/runtime-host/src/server/web-search-tool.ts | Adds a host helper to derive Tavily readiness from the canonical resolver. |
| packages/runtime-host/src/server/interactive-run-composer.ts | Threads Tavily readiness into unified routeWebSearchTools() routing for root/bound/child surfaces. |
| packages/runtime-host/src/server/execution-composition.ts | Applies the same readiness-gated routing to child-agent tool resolution. |
| packages/runtime-host/src/tests/web-search-tool.test.ts | Tests readiness derivation follows resolveWebSearchExecution() states. |
| packages/runtime-host/src/tests/execution-model-composition.test.ts | Ensures effective tool surfaces omit WebSearch/web_research when Tavily cannot run. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Code Review by Qodo
1. |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 2c313c8e-aed7-413a-9a68-94dba2043208
📒 Files selected for processing (5)
packages/runtime-host/src/__tests__/execution-composition.test.tspackages/runtime-host/src/__tests__/web-search-tool.test.tspackages/runtime-host/src/server/execution-composition.tspackages/runtime-host/src/server/interactive-run-composer.tspackages/runtime-host/src/server/web-search-tool.ts
Included review availability: Your plan includes up to 3 reviews per rolling hour; 1 remains after this review.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
Suppressed comments (1)
packages/runtime-host/src/server/interactive-run-composer.ts:352
- Avoid hard-coding the WebSearch tool name here. The canonical tool name already exists as
NATIVE_WEB_SEARCH_TOOL_NAMEin@maka/runtime/native-web-search-tool(used by the router you call below). Importing and reusing that constant reduces drift risk if the tool name ever changes.
const route = (tools: readonly MakaTool[]): MakaTool[] => {
const webFetchTools = routeWebFetchTools(tools, input.runtimePolicy.policy.privacy);
if (!input.connection) {
return webFetchTools.filter((tool) => tool.name !== 'WebSearch');
}
jackwener
commented
Aug 17, 2026
@YayoiNanoka I verified the bug against current main: Tavily-backed WebSearch is still advertised when the canonical execution resolver says the credential path cannot run. The current solution uses the right authority, keeps provider-native search independent, and consistently derives root/bound/child/skill surfaces; I found no implementation blocker on head 88658e4, and all hosted checks are green. Two merge gates remain: this model-visible tool-contract change needs an independent human review under CONTRIBUTING.md, and the PR needs an explicit AI-use declaration (plus Generated-by trailers if generative tooling authored material code). Please complete that provenance declaration while the human review is pending. |
Generated-by: Codex
Generated-by: Codex
88658e4 to
35920ecCompareYayoiNanoka
commented
Aug 17, 2026
@jackwener Provenance is now complete: the PR body explicitly declares substantive OpenAI Codex use and its scope, and both PR commits carry Generated-by: Codex trailers. The trailer-only history rewrite did not change the code tree; the current head is 35920ec. Independent human review remains pending. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
Suppressed comments (1)
packages/runtime-host/src/server/web-search-tool.ts:45
resolveHostTavilyWebSearchReadiness()propagates exceptions fromresolveWebSearchExecution(). Since this function is now called during tool-surface assembly (not just at tool call-time), an unexpected store/vault read failure would fail the entire session/tool-surface resolution rather than failing closed by simply omittingWebSearch. Consider catching errors here and returningfalseso the surface stays usable and remains fail-closed.
export async function resolveHostTavilyWebSearchReadiness(
policy: Pick<RuntimePolicyOperationCoordinator, 'resolveWebSearchExecution'>,
): Promise<boolean> {
return (await policy.resolveWebSearchExecution({ provider: 'tavily' })).kind === 'ready';
}
Astro-Han
left a comment
There was a problem hiding this comment.
No P0–P2 findings on 88658e446.
The problem is correctly defined as a mismatch between the model-visible tool surface and the canonical execution authority. This change fixes it at the right owner: resolveWebSearchExecution() remains the sole Tavily credential/proxy readiness authority, while the synchronous routing layer consumes only the minimal tavilyReady projection. The call-time resolver remains in place as the final fail-closed boundary.
I independently verified that the follow-up commit closes the earlier composition gap: root, bound, child, parent-agent, current-session preview, new-session preview, and durable child surfaces now pass through the shared routing seam. Provider-native search remains independent of Tavily readiness. I found no stale-backend or settings/credential lifecycle path that re-exposes an unavailable tool.
One non-blocking P3 test follow-up remains: the new integration coverage is negative-only. Please add a positive catalog control so an empty catalog cannot satisfy the test, and consider a configured-Tavily composition control proving that ready WebSearch and web_research remain present. The existing unit coverage and direct wiring are sufficient for approval, so this does not need to block the PR.
Verdict: Approve. The implementation is the smallest coherent root fix, does not need a PR split, and should not be replaced by duplicated readiness logic or an async router.
Before merge, please complete the PR's AI-use declaration and preserve Generated-by trailers if material generated code was used. This model-visible contract change still requires the repository's independent human review.
Codex assisted this review by reconstructing the feedback ledger, tracing the current production surfaces and lifecycle, and checking CI evidence. The human reviewer remains responsible for the final judgment and merge decision.
简体中文
当前 head 没有 P0–P2。PR 在正确 owner 修复了模型可见工具表面与实际执行能力不一致的问题：Tavily readiness 继续由权威 resolver 决定，routing 只消费最小布尔投影，调用阶段仍保持 fail closed。
root、bound、child、parent-agent、当前 Session、新 Session 和 durable child preview 已统一经过同一 routing seam；模型原生搜索不依赖 Tavily，没有被误删。
仅剩非阻塞 P3 测试建议：补普通 skill 正向控制，避免空 catalog 也通过；最好再补有效 Tavily credential 的 composition 正向控制。当前实现可以 Approve，无需拆 PR。
合并前仍需完成 AI-use declaration；若存在实质生成代码，应保留 Generated-by trailer。本次模型可见契约变更仍需独立人工 review。
Uh oh!
There was an error while loading. Please reload this page.
Problem
When Web Search selected Tavily, the effective model tool surface kept advertising
WebSearcheven when the canonical runtime-policy resolver reported that the Tavily or required proxy credential was unavailable. The model could therefore spend a turn calling a tool that was guaranteed to fail, and child-agent composition could advertise theweb_researchprofile without a runnable search tool.The existing call-time fail-closed check remains necessary, but it happens after the model has already chosen the tool.
Solution
resolveWebSearchExecution({ provider: 'tavily' })authority.routeWebSearchTools()seam and retain the client-executedWebSearchonly when the result isready.web_researchis omitted when Tavily cannot run.modelstill depends only on the selected model's hosted-search capability and never requires a Tavily credential.Closes#2085.
Verification
npm --workspace @maka/runtime run typechecknpm --workspace @maka/runtime-host run typecheckWebSearchweb_researchwhen Tavily is unavailableBroader checks attempted:
npm run build:testcompiled Code Mode, Core, Storage, MCP, Runtime, Runtime Host, Computer Use, Eval, and the CLI, then stopped in untouched UI files on existing interface mismatches involvingsettledText,conversationKey, andunlockAutoFollow.AI use
Tool(s) and scope: OpenAI Codex assisted with repository analysis, implementation, regression tests, and review revisions. The human contributor reviewed, tested, and accepted the final change.
点击展开中文
问题
当联网搜索选择 Tavily 时，即使 Runtime Policy 的权威解析器已经确认 Tavily 凭证或必需的代理凭证不可用，
WebSearch仍然会出现在模型可见的工具列表中。模型可能因此浪费一轮调用一个必然失败的工具，子代理表面也可能错误地提供无法真正执行搜索的web_research。现有的调用阶段 fail-closed 检查仍然有必要，但它发生在模型选择工具之后，无法避免这次无效调用。
解决方案
resolveWebSearchExecution({ provider: 'tavily' })获取 Tavily 就绪状态。routeWebSearchTools()；只有结果为ready时，才保留由 Runtime 执行的WebSearch。web_research。model时只检查当前模型是否支持原生联网搜索，不要求 Tavily 凭证。验证
WebSearch，子代理也不再暴露不可用的web_research。更大范围检查中，
npm run build:test在完成 Code Mode、Core、Storage、MCP、Runtime、Runtime Host、Computer Use、Eval 和 CLI 编译后，停在未修改的 UI 文件接口不一致；完整 Runtime 测试还受到受管沙箱中的 macOS sandbox/process smoke test 和无关 model-factory streaming 断言影响，因此没有声明全量测试通过。