Haven provider + fix dropped completions on vLLM backends by coelhogonzalo · Pull Request #1 · jan3dev/continue-haven

coelhogonzalo · 2026-08-26T00:17:37Z

Opened against this fork's main to trigger pr-build-upload-vsix.yaml, which
builds the VSIX natively for linux-x64, win32-x64 and darwin-arm64 and uploads
each as an artifact. The darwin-arm64 artifact is the one to share with macOS
users. Not intended for continuedev/continue.

Commits

Add first-class Haven provider — provider: haven, an OpenAI subclass on
the raw-fetch path. Request bodies are HPKE-encrypted in-process to the
Tinfoil enclave via haven-proxy's relay core, so no localhost proxy runs and
Haven never sees prompts or completions.
Make haven-proxy ambient types visible to the gui/binary type-check programs.
Fix streamed completions being dropped when every chunk carries usage.

The streaming fix

chatCompletionStream deferred any chunk carrying a usage field to the end of
the stream, assuming OpenAI's convention where usage arrives once on a final
chunk with no choices. vLLM attaches usage to every chunk — measured 59 of
59 frames against a live endpoint — so each chunk was stored into
lastChunkWithUsage and never yielded. Only the final chunk survived, and its
delta is empty.

The caller assembled an empty completion without raising, so the request logged
as a success with 0 generated tokens and the UI showed no reply at all. This
affected any vLLM-backed provider: openai endpoint, not just Haven.

The fix defers a chunk only when it has no choices, so content chunks pass
through with their usage attached while OpenAI's usage-only chunk is still
emitted last. Three tests cover all three stream shapes; reverting the fix fails
exactly the vLLM one with expected '' to be 'Hi there'.

This commit is independent of the Haven work and cherry-picks cleanly onto
main if it is worth sending upstream.

Verification

packages/openai-adapters: 3/3 new tests pass, tsc --noEmit clean.
core/llm/llms/Haven.vitest.ts: 10/10 pass.
Built and installed the win32-x64 VSIX locally; confirmed the fix is present
in the shipped bundle and that chat returns non-zero generated tokens.

🤖 Generated with Claude Code

…roxy) Adds provider: haven backed by the haven-proxy package's relay core: request bodies are HPKE-encrypted end-to-end to the Tinfoil enclave on this machine, so Haven never sees prompts or completions and no localhost proxy process is needed. - core/llm/llms/Haven.ts: OpenAI subclass on the raw-fetch path (useOpenAIAdapterFor = []); POST chat/completions goes through createSecureRelay, everything else through the normal fetch with key-free headers (the relay injects X-Api-Key itself, only to the Haven origin). Key resolution: config apiKey -> HAVEN_API_KEY -> ~/.haven-proxy/config.json, so the key never has to live in a shareable config file. - Registration, tool-support (per-model tool_call flags from the Haven catalog) and chat-templating entries. - config_schema.json: provider enum entry, description, model enum. - haven-proxy.d.ts: ambient types (the package ships none). - Haven.vitest.ts: relay-mocked coverage of streaming, buffered-SSE synthesis, non-stream JSON, error mapping, abort, key resolution, header hygiene, endpoint shape and context lengths.

gui and binary compile core sources with their own tsconfigs, which do not include core's .d.ts files, so the haven-proxy module declarations were out of scope there and the gui build failed. A triple-slash reference in Haven.ts pulls the declarations into every program that compiles the file.

chatCompletionStream deferred any chunk with a `usage` field to the end of the stream, assuming OpenAI's convention where usage arrives once on a final chunk with no choices. vLLM attaches usage to every chunk, so each one was stored into lastChunkWithUsage and never yielded; only the final chunk survived, and its delta is empty. The caller assembled an empty completion with no error, so the request was logged as a success with 0 generated tokens and the UI showed no reply at all. This hit any vLLM-backed provider: openai endpoint. Defer a chunk only when it has no choices, so content chunks pass through with their usage attached and OpenAI's usage-only chunk is still emitted last. Add tests covering all three stream shapes.

Distinguishes local builds carrying the Haven provider and the streaming fix from the 1.3.40 marketplace release, so an installed VSIX is identifiable and VS Code does not treat the marketplace build as an equal-version replacement. Also records the haven-proxy dependency in the gui lockfile, which the Haven provider commit added to core.

coelhogonzalo · 2026-08-26T00:17:37Z

Opened against this fork's main to trigger pr-build-upload-vsix.yaml, which
builds the VSIX natively for linux-x64, win32-x64 and darwin-arm64 and uploads
each as an artifact. The darwin-arm64 artifact is the one to share with macOS
users. Not intended for continuedev/continue.

Commits

Add first-class Haven provider — provider: haven, an OpenAI subclass on
the raw-fetch path. Request bodies are HPKE-encrypted in-process to the
Tinfoil enclave via haven-proxy's relay core, so no localhost proxy runs and
Haven never sees prompts or completions.
Make haven-proxy ambient types visible to the gui/binary type-check programs.
Fix streamed completions being dropped when every chunk carries usage.

The streaming fix

chatCompletionStream deferred any chunk carrying a usage field to the end of
the stream, assuming OpenAI's convention where usage arrives once on a final
chunk with no choices. vLLM attaches usage to every chunk — measured 59 of
59 frames against a live endpoint — so each chunk was stored into
lastChunkWithUsage and never yielded. Only the final chunk survived, and its
delta is empty.

The caller assembled an empty completion without raising, so the request logged
as a success with 0 generated tokens and the UI showed no reply at all. This
affected any vLLM-backed provider: openai endpoint, not just Haven.

The fix defers a chunk only when it has no choices, so content chunks pass
through with their usage attached while OpenAI's usage-only chunk is still
emitted last. Three tests cover all three stream shapes; reverting the fix fails
exactly the vLLM one with expected '' to be 'Hi there'.

This commit is independent of the Haven work and cherry-picks cleanly onto
main if it is worth sending upstream.

Verification

packages/openai-adapters: 3/3 new tests pass, tsc --noEmit clean.
core/llm/llms/Haven.vitest.ts: 10/10 pass.
Built and installed the win32-x64 VSIX locally; confirmed the fix is present
in the shipped bundle and that chat returns non-zero generated tokens.

🤖 Generated with Claude Code

…roxy) Adds provider: haven backed by the haven-proxy package's relay core: request bodies are HPKE-encrypted end-to-end to the Tinfoil enclave on this machine, so Haven never sees prompts or completions and no localhost proxy process is needed. - core/llm/llms/Haven.ts: OpenAI subclass on the raw-fetch path (useOpenAIAdapterFor = []); POST chat/completions goes through createSecureRelay, everything else through the normal fetch with key-free headers (the relay injects X-Api-Key itself, only to the Haven origin). Key resolution: config apiKey -> HAVEN_API_KEY -> ~/.haven-proxy/config.json, so the key never has to live in a shareable config file. - Registration, tool-support (per-model tool_call flags from the Haven catalog) and chat-templating entries. - config_schema.json: provider enum entry, description, model enum. - haven-proxy.d.ts: ambient types (the package ships none). - Haven.vitest.ts: relay-mocked coverage of streaming, buffered-SSE synthesis, non-stream JSON, error mapping, abort, key resolution, header hygiene, endpoint shape and context lengths.

gui and binary compile core sources with their own tsconfigs, which do not include core's .d.ts files, so the haven-proxy module declarations were out of scope there and the gui build failed. A triple-slash reference in Haven.ts pulls the declarations into every program that compiles the file.

chatCompletionStream deferred any chunk with a `usage` field to the end of the stream, assuming OpenAI's convention where usage arrives once on a final chunk with no choices. vLLM attaches usage to every chunk, so each one was stored into lastChunkWithUsage and never yielded; only the final chunk survived, and its delta is empty. The caller assembled an empty completion with no error, so the request was logged as a success with 0 generated tokens and the UI showed no reply at all. This hit any vLLM-backed provider: openai endpoint. Defer a chunk only when it has no choices, so content chunks pass through with their usage attached and OpenAI's usage-only chunk is still emitted last. Add tests covering all three stream shapes.

Distinguishes local builds carrying the Haven provider and the streaming fix from the 1.3.40 marketplace release, so an installed VSIX is identifiable and VS Code does not treat the marketplace build as an equal-version replacement. Also records the haven-proxy dependency in the gui lockfile, which the Haven provider commit added to core.

coelhogonzalo · 2026-08-26T00:17:37Z

Opened against this fork's main to trigger pr-build-upload-vsix.yaml, which
builds the VSIX natively for linux-x64, win32-x64 and darwin-arm64 and uploads
each as an artifact. The darwin-arm64 artifact is the one to share with macOS
users. Not intended for continuedev/continue.

Commits

Add first-class Haven provider — provider: haven, an OpenAI subclass on
the raw-fetch path. Request bodies are HPKE-encrypted in-process to the
Tinfoil enclave via haven-proxy's relay core, so no localhost proxy runs and
Haven never sees prompts or completions.
Make haven-proxy ambient types visible to the gui/binary type-check programs.
Fix streamed completions being dropped when every chunk carries usage.

The streaming fix

chatCompletionStream deferred any chunk carrying a usage field to the end of
the stream, assuming OpenAI's convention where usage arrives once on a final
chunk with no choices. vLLM attaches usage to every chunk — measured 59 of
59 frames against a live endpoint — so each chunk was stored into
lastChunkWithUsage and never yielded. Only the final chunk survived, and its
delta is empty.

The caller assembled an empty completion without raising, so the request logged
as a success with 0 generated tokens and the UI showed no reply at all. This
affected any vLLM-backed provider: openai endpoint, not just Haven.

The fix defers a chunk only when it has no choices, so content chunks pass
through with their usage attached while OpenAI's usage-only chunk is still
emitted last. Three tests cover all three stream shapes; reverting the fix fails
exactly the vLLM one with expected '' to be 'Hi there'.

This commit is independent of the Haven work and cherry-picks cleanly onto
main if it is worth sending upstream.

Verification

packages/openai-adapters: 3/3 new tests pass, tsc --noEmit clean.
core/llm/llms/Haven.vitest.ts: 10/10 pass.
Built and installed the win32-x64 VSIX locally; confirmed the fix is present
in the shipped bundle and that chat returns non-zero generated tokens.

🤖 Generated with Claude Code

…roxy) Adds provider: haven backed by the haven-proxy package's relay core: request bodies are HPKE-encrypted end-to-end to the Tinfoil enclave on this machine, so Haven never sees prompts or completions and no localhost proxy process is needed. - core/llm/llms/Haven.ts: OpenAI subclass on the raw-fetch path (useOpenAIAdapterFor = []); POST chat/completions goes through createSecureRelay, everything else through the normal fetch with key-free headers (the relay injects X-Api-Key itself, only to the Haven origin). Key resolution: config apiKey -> HAVEN_API_KEY -> ~/.haven-proxy/config.json, so the key never has to live in a shareable config file. - Registration, tool-support (per-model tool_call flags from the Haven catalog) and chat-templating entries. - config_schema.json: provider enum entry, description, model enum. - haven-proxy.d.ts: ambient types (the package ships none). - Haven.vitest.ts: relay-mocked coverage of streaming, buffered-SSE synthesis, non-stream JSON, error mapping, abort, key resolution, header hygiene, endpoint shape and context lengths.

gui and binary compile core sources with their own tsconfigs, which do not include core's .d.ts files, so the haven-proxy module declarations were out of scope there and the gui build failed. A triple-slash reference in Haven.ts pulls the declarations into every program that compiles the file.

chatCompletionStream deferred any chunk with a `usage` field to the end of the stream, assuming OpenAI's convention where usage arrives once on a final chunk with no choices. vLLM attaches usage to every chunk, so each one was stored into lastChunkWithUsage and never yielded; only the final chunk survived, and its delta is empty. The caller assembled an empty completion with no error, so the request was logged as a success with 0 generated tokens and the UI showed no reply at all. This hit any vLLM-backed provider: openai endpoint. Defer a chunk only when it has no choices, so content chunks pass through with their usage attached and OpenAI's usage-only chunk is still emitted last. Add tests covering all three stream shapes.

Distinguishes local builds carrying the Haven provider and the streaming fix from the 1.3.40 marketplace release, so an installed VSIX is identifiable and VS Code does not treat the marketplace build as an equal-version replacement. Also records the haven-proxy dependency in the gui lockfile, which the Haven provider commit added to core.

coelhogonzalo · 2026-08-26T00:17:37Z

Opened against this fork's main to trigger pr-build-upload-vsix.yaml, which
builds the VSIX natively for linux-x64, win32-x64 and darwin-arm64 and uploads
each as an artifact. The darwin-arm64 artifact is the one to share with macOS
users. Not intended for continuedev/continue.

Commits

Add first-class Haven provider — provider: haven, an OpenAI subclass on
the raw-fetch path. Request bodies are HPKE-encrypted in-process to the
Tinfoil enclave via haven-proxy's relay core, so no localhost proxy runs and
Haven never sees prompts or completions.
Make haven-proxy ambient types visible to the gui/binary type-check programs.
Fix streamed completions being dropped when every chunk carries usage.

The streaming fix

chatCompletionStream deferred any chunk carrying a usage field to the end of
the stream, assuming OpenAI's convention where usage arrives once on a final
chunk with no choices. vLLM attaches usage to every chunk — measured 59 of
59 frames against a live endpoint — so each chunk was stored into
lastChunkWithUsage and never yielded. Only the final chunk survived, and its
delta is empty.

The caller assembled an empty completion without raising, so the request logged
as a success with 0 generated tokens and the UI showed no reply at all. This
affected any vLLM-backed provider: openai endpoint, not just Haven.

The fix defers a chunk only when it has no choices, so content chunks pass
through with their usage attached while OpenAI's usage-only chunk is still
emitted last. Three tests cover all three stream shapes; reverting the fix fails
exactly the vLLM one with expected '' to be 'Hi there'.

This commit is independent of the Haven work and cherry-picks cleanly onto
main if it is worth sending upstream.

Verification

packages/openai-adapters: 3/3 new tests pass, tsc --noEmit clean.
core/llm/llms/Haven.vitest.ts: 10/10 pass.
Built and installed the win32-x64 VSIX locally; confirmed the fix is present
in the shipped bundle and that chat returns non-zero generated tokens.

🤖 Generated with Claude Code

…roxy) Adds provider: haven backed by the haven-proxy package's relay core: request bodies are HPKE-encrypted end-to-end to the Tinfoil enclave on this machine, so Haven never sees prompts or completions and no localhost proxy process is needed. - core/llm/llms/Haven.ts: OpenAI subclass on the raw-fetch path (useOpenAIAdapterFor = []); POST chat/completions goes through createSecureRelay, everything else through the normal fetch with key-free headers (the relay injects X-Api-Key itself, only to the Haven origin). Key resolution: config apiKey -> HAVEN_API_KEY -> ~/.haven-proxy/config.json, so the key never has to live in a shareable config file. - Registration, tool-support (per-model tool_call flags from the Haven catalog) and chat-templating entries. - config_schema.json: provider enum entry, description, model enum. - haven-proxy.d.ts: ambient types (the package ships none). - Haven.vitest.ts: relay-mocked coverage of streaming, buffered-SSE synthesis, non-stream JSON, error mapping, abort, key resolution, header hygiene, endpoint shape and context lengths.

gui and binary compile core sources with their own tsconfigs, which do not include core's .d.ts files, so the haven-proxy module declarations were out of scope there and the gui build failed. A triple-slash reference in Haven.ts pulls the declarations into every program that compiles the file.

chatCompletionStream deferred any chunk with a `usage` field to the end of the stream, assuming OpenAI's convention where usage arrives once on a final chunk with no choices. vLLM attaches usage to every chunk, so each one was stored into lastChunkWithUsage and never yielded; only the final chunk survived, and its delta is empty. The caller assembled an empty completion with no error, so the request was logged as a success with 0 generated tokens and the UI showed no reply at all. This hit any vLLM-backed provider: openai endpoint. Defer a chunk only when it has no choices, so content chunks pass through with their usage attached and OpenAI's usage-only chunk is still emitted last. Add tests covering all three stream shapes.

Distinguishes local builds carrying the Haven provider and the streaming fix from the 1.3.40 marketplace release, so an installed VSIX is identifiable and VS Code does not treat the marketplace build as an equal-version replacement. Also records the haven-proxy dependency in the gui lockfile, which the Haven provider commit added to core.

coelhogonzalo · 2026-08-26T00:17:37Z

Opened against this fork's main to trigger pr-build-upload-vsix.yaml, which
builds the VSIX natively for linux-x64, win32-x64 and darwin-arm64 and uploads
each as an artifact. The darwin-arm64 artifact is the one to share with macOS
users. Not intended for continuedev/continue.

Commits

Add first-class Haven provider — provider: haven, an OpenAI subclass on
the raw-fetch path. Request bodies are HPKE-encrypted in-process to the
Tinfoil enclave via haven-proxy's relay core, so no localhost proxy runs and
Haven never sees prompts or completions.
Make haven-proxy ambient types visible to the gui/binary type-check programs.
Fix streamed completions being dropped when every chunk carries usage.

The streaming fix

chatCompletionStream deferred any chunk carrying a usage field to the end of
the stream, assuming OpenAI's convention where usage arrives once on a final
chunk with no choices. vLLM attaches usage to every chunk — measured 59 of
59 frames against a live endpoint — so each chunk was stored into
lastChunkWithUsage and never yielded. Only the final chunk survived, and its
delta is empty.

The caller assembled an empty completion without raising, so the request logged
as a success with 0 generated tokens and the UI showed no reply at all. This
affected any vLLM-backed provider: openai endpoint, not just Haven.

The fix defers a chunk only when it has no choices, so content chunks pass
through with their usage attached while OpenAI's usage-only chunk is still
emitted last. Three tests cover all three stream shapes; reverting the fix fails
exactly the vLLM one with expected '' to be 'Hi there'.

This commit is independent of the Haven work and cherry-picks cleanly onto
main if it is worth sending upstream.

Verification

packages/openai-adapters: 3/3 new tests pass, tsc --noEmit clean.
core/llm/llms/Haven.vitest.ts: 10/10 pass.
Built and installed the win32-x64 VSIX locally; confirmed the fix is present
in the shipped bundle and that chat returns non-zero generated tokens.

🤖 Generated with Claude Code

…roxy) Adds provider: haven backed by the haven-proxy package's relay core: request bodies are HPKE-encrypted end-to-end to the Tinfoil enclave on this machine, so Haven never sees prompts or completions and no localhost proxy process is needed. - core/llm/llms/Haven.ts: OpenAI subclass on the raw-fetch path (useOpenAIAdapterFor = []); POST chat/completions goes through createSecureRelay, everything else through the normal fetch with key-free headers (the relay injects X-Api-Key itself, only to the Haven origin). Key resolution: config apiKey -> HAVEN_API_KEY -> ~/.haven-proxy/config.json, so the key never has to live in a shareable config file. - Registration, tool-support (per-model tool_call flags from the Haven catalog) and chat-templating entries. - config_schema.json: provider enum entry, description, model enum. - haven-proxy.d.ts: ambient types (the package ships none). - Haven.vitest.ts: relay-mocked coverage of streaming, buffered-SSE synthesis, non-stream JSON, error mapping, abort, key resolution, header hygiene, endpoint shape and context lengths.

gui and binary compile core sources with their own tsconfigs, which do not include core's .d.ts files, so the haven-proxy module declarations were out of scope there and the gui build failed. A triple-slash reference in Haven.ts pulls the declarations into every program that compiles the file.

chatCompletionStream deferred any chunk with a `usage` field to the end of the stream, assuming OpenAI's convention where usage arrives once on a final chunk with no choices. vLLM attaches usage to every chunk, so each one was stored into lastChunkWithUsage and never yielded; only the final chunk survived, and its delta is empty. The caller assembled an empty completion with no error, so the request was logged as a success with 0 generated tokens and the UI showed no reply at all. This hit any vLLM-backed provider: openai endpoint. Defer a chunk only when it has no choices, so content chunks pass through with their usage attached and OpenAI's usage-only chunk is still emitted last. Add tests covering all three stream shapes.

Distinguishes local builds carrying the Haven provider and the streaming fix from the 1.3.40 marketplace release, so an installed VSIX is identifiable and VS Code does not treat the marketplace build as an equal-version replacement. Also records the haven-proxy dependency in the gui lockfile, which the Haven provider commit added to core.

coelhogonzalo · 2026-08-26T00:17:37Z

Opened against this fork's main to trigger pr-build-upload-vsix.yaml, which
builds the VSIX natively for linux-x64, win32-x64 and darwin-arm64 and uploads
each as an artifact. The darwin-arm64 artifact is the one to share with macOS
users. Not intended for continuedev/continue.

Commits

Add first-class Haven provider — provider: haven, an OpenAI subclass on
the raw-fetch path. Request bodies are HPKE-encrypted in-process to the
Tinfoil enclave via haven-proxy's relay core, so no localhost proxy runs and
Haven never sees prompts or completions.
Make haven-proxy ambient types visible to the gui/binary type-check programs.
Fix streamed completions being dropped when every chunk carries usage.

The streaming fix

chatCompletionStream deferred any chunk carrying a usage field to the end of
the stream, assuming OpenAI's convention where usage arrives once on a final
chunk with no choices. vLLM attaches usage to every chunk — measured 59 of
59 frames against a live endpoint — so each chunk was stored into
lastChunkWithUsage and never yielded. Only the final chunk survived, and its
delta is empty.

The caller assembled an empty completion without raising, so the request logged
as a success with 0 generated tokens and the UI showed no reply at all. This
affected any vLLM-backed provider: openai endpoint, not just Haven.

The fix defers a chunk only when it has no choices, so content chunks pass
through with their usage attached while OpenAI's usage-only chunk is still
emitted last. Three tests cover all three stream shapes; reverting the fix fails
exactly the vLLM one with expected '' to be 'Hi there'.

This commit is independent of the Haven work and cherry-picks cleanly onto
main if it is worth sending upstream.

Verification

packages/openai-adapters: 3/3 new tests pass, tsc --noEmit clean.
core/llm/llms/Haven.vitest.ts: 10/10 pass.
Built and installed the win32-x64 VSIX locally; confirmed the fix is present
in the shipped bundle and that chat returns non-zero generated tokens.

🤖 Generated with Claude Code

…roxy) Adds provider: haven backed by the haven-proxy package's relay core: request bodies are HPKE-encrypted end-to-end to the Tinfoil enclave on this machine, so Haven never sees prompts or completions and no localhost proxy process is needed. - core/llm/llms/Haven.ts: OpenAI subclass on the raw-fetch path (useOpenAIAdapterFor = []); POST chat/completions goes through createSecureRelay, everything else through the normal fetch with key-free headers (the relay injects X-Api-Key itself, only to the Haven origin). Key resolution: config apiKey -> HAVEN_API_KEY -> ~/.haven-proxy/config.json, so the key never has to live in a shareable config file. - Registration, tool-support (per-model tool_call flags from the Haven catalog) and chat-templating entries. - config_schema.json: provider enum entry, description, model enum. - haven-proxy.d.ts: ambient types (the package ships none). - Haven.vitest.ts: relay-mocked coverage of streaming, buffered-SSE synthesis, non-stream JSON, error mapping, abort, key resolution, header hygiene, endpoint shape and context lengths.

gui and binary compile core sources with their own tsconfigs, which do not include core's .d.ts files, so the haven-proxy module declarations were out of scope there and the gui build failed. A triple-slash reference in Haven.ts pulls the declarations into every program that compiles the file.

chatCompletionStream deferred any chunk with a `usage` field to the end of the stream, assuming OpenAI's convention where usage arrives once on a final chunk with no choices. vLLM attaches usage to every chunk, so each one was stored into lastChunkWithUsage and never yielded; only the final chunk survived, and its delta is empty. The caller assembled an empty completion with no error, so the request was logged as a success with 0 generated tokens and the UI showed no reply at all. This hit any vLLM-backed provider: openai endpoint. Defer a chunk only when it has no choices, so content chunks pass through with their usage attached and OpenAI's usage-only chunk is still emitted last. Add tests covering all three stream shapes.

Distinguishes local builds carrying the Haven provider and the streaming fix from the 1.3.40 marketplace release, so an installed VSIX is identifiable and VS Code does not treat the marketplace build as an equal-version replacement. Also records the haven-proxy dependency in the gui lockfile, which the Haven provider commit added to core.

coelhogonzalo · 2026-08-26T00:17:37Z

Opened against this fork's main to trigger pr-build-upload-vsix.yaml, which
builds the VSIX natively for linux-x64, win32-x64 and darwin-arm64 and uploads
each as an artifact. The darwin-arm64 artifact is the one to share with macOS
users. Not intended for continuedev/continue.

Commits

Add first-class Haven provider — provider: haven, an OpenAI subclass on
the raw-fetch path. Request bodies are HPKE-encrypted in-process to the
Tinfoil enclave via haven-proxy's relay core, so no localhost proxy runs and
Haven never sees prompts or completions.
Make haven-proxy ambient types visible to the gui/binary type-check programs.
Fix streamed completions being dropped when every chunk carries usage.

The streaming fix

chatCompletionStream deferred any chunk carrying a usage field to the end of
the stream, assuming OpenAI's convention where usage arrives once on a final
chunk with no choices. vLLM attaches usage to every chunk — measured 59 of
59 frames against a live endpoint — so each chunk was stored into
lastChunkWithUsage and never yielded. Only the final chunk survived, and its
delta is empty.

The caller assembled an empty completion without raising, so the request logged
as a success with 0 generated tokens and the UI showed no reply at all. This
affected any vLLM-backed provider: openai endpoint, not just Haven.

The fix defers a chunk only when it has no choices, so content chunks pass
through with their usage attached while OpenAI's usage-only chunk is still
emitted last. Three tests cover all three stream shapes; reverting the fix fails
exactly the vLLM one with expected '' to be 'Hi there'.

This commit is independent of the Haven work and cherry-picks cleanly onto
main if it is worth sending upstream.

Verification

packages/openai-adapters: 3/3 new tests pass, tsc --noEmit clean.
core/llm/llms/Haven.vitest.ts: 10/10 pass.
Built and installed the win32-x64 VSIX locally; confirmed the fix is present
in the shipped bundle and that chat returns non-zero generated tokens.

🤖 Generated with Claude Code

…roxy) Adds provider: haven backed by the haven-proxy package's relay core: request bodies are HPKE-encrypted end-to-end to the Tinfoil enclave on this machine, so Haven never sees prompts or completions and no localhost proxy process is needed. - core/llm/llms/Haven.ts: OpenAI subclass on the raw-fetch path (useOpenAIAdapterFor = []); POST chat/completions goes through createSecureRelay, everything else through the normal fetch with key-free headers (the relay injects X-Api-Key itself, only to the Haven origin). Key resolution: config apiKey -> HAVEN_API_KEY -> ~/.haven-proxy/config.json, so the key never has to live in a shareable config file. - Registration, tool-support (per-model tool_call flags from the Haven catalog) and chat-templating entries. - config_schema.json: provider enum entry, description, model enum. - haven-proxy.d.ts: ambient types (the package ships none). - Haven.vitest.ts: relay-mocked coverage of streaming, buffered-SSE synthesis, non-stream JSON, error mapping, abort, key resolution, header hygiene, endpoint shape and context lengths.

gui and binary compile core sources with their own tsconfigs, which do not include core's .d.ts files, so the haven-proxy module declarations were out of scope there and the gui build failed. A triple-slash reference in Haven.ts pulls the declarations into every program that compiles the file.

chatCompletionStream deferred any chunk with a `usage` field to the end of the stream, assuming OpenAI's convention where usage arrives once on a final chunk with no choices. vLLM attaches usage to every chunk, so each one was stored into lastChunkWithUsage and never yielded; only the final chunk survived, and its delta is empty. The caller assembled an empty completion with no error, so the request was logged as a success with 0 generated tokens and the UI showed no reply at all. This hit any vLLM-backed provider: openai endpoint. Defer a chunk only when it has no choices, so content chunks pass through with their usage attached and OpenAI's usage-only chunk is still emitted last. Add tests covering all three stream shapes.

Distinguishes local builds carrying the Haven provider and the streaming fix from the 1.3.40 marketplace release, so an installed VSIX is identifiable and VS Code does not treat the marketplace build as an equal-version replacement. Also records the haven-proxy dependency in the gui lockfile, which the Haven provider commit added to core.

coelhogonzalo · 2026-08-26T00:17:37Z

Opened against this fork's main to trigger pr-build-upload-vsix.yaml, which
builds the VSIX natively for linux-x64, win32-x64 and darwin-arm64 and uploads
each as an artifact. The darwin-arm64 artifact is the one to share with macOS
users. Not intended for continuedev/continue.

Commits

Add first-class Haven provider — provider: haven, an OpenAI subclass on
the raw-fetch path. Request bodies are HPKE-encrypted in-process to the
Tinfoil enclave via haven-proxy's relay core, so no localhost proxy runs and
Haven never sees prompts or completions.
Make haven-proxy ambient types visible to the gui/binary type-check programs.
Fix streamed completions being dropped when every chunk carries usage.

The streaming fix

chatCompletionStream deferred any chunk carrying a usage field to the end of
the stream, assuming OpenAI's convention where usage arrives once on a final
chunk with no choices. vLLM attaches usage to every chunk — measured 59 of
59 frames against a live endpoint — so each chunk was stored into
lastChunkWithUsage and never yielded. Only the final chunk survived, and its
delta is empty.

The caller assembled an empty completion without raising, so the request logged
as a success with 0 generated tokens and the UI showed no reply at all. This
affected any vLLM-backed provider: openai endpoint, not just Haven.

The fix defers a chunk only when it has no choices, so content chunks pass
through with their usage attached while OpenAI's usage-only chunk is still
emitted last. Three tests cover all three stream shapes; reverting the fix fails
exactly the vLLM one with expected '' to be 'Hi there'.

This commit is independent of the Haven work and cherry-picks cleanly onto
main if it is worth sending upstream.

Verification

packages/openai-adapters: 3/3 new tests pass, tsc --noEmit clean.
core/llm/llms/Haven.vitest.ts: 10/10 pass.
Built and installed the win32-x64 VSIX locally; confirmed the fix is present
in the shipped bundle and that chat returns non-zero generated tokens.

🤖 Generated with Claude Code

…roxy) Adds provider: haven backed by the haven-proxy package's relay core: request bodies are HPKE-encrypted end-to-end to the Tinfoil enclave on this machine, so Haven never sees prompts or completions and no localhost proxy process is needed. - core/llm/llms/Haven.ts: OpenAI subclass on the raw-fetch path (useOpenAIAdapterFor = []); POST chat/completions goes through createSecureRelay, everything else through the normal fetch with key-free headers (the relay injects X-Api-Key itself, only to the Haven origin). Key resolution: config apiKey -> HAVEN_API_KEY -> ~/.haven-proxy/config.json, so the key never has to live in a shareable config file. - Registration, tool-support (per-model tool_call flags from the Haven catalog) and chat-templating entries. - config_schema.json: provider enum entry, description, model enum. - haven-proxy.d.ts: ambient types (the package ships none). - Haven.vitest.ts: relay-mocked coverage of streaming, buffered-SSE synthesis, non-stream JSON, error mapping, abort, key resolution, header hygiene, endpoint shape and context lengths.

gui and binary compile core sources with their own tsconfigs, which do not include core's .d.ts files, so the haven-proxy module declarations were out of scope there and the gui build failed. A triple-slash reference in Haven.ts pulls the declarations into every program that compiles the file.

chatCompletionStream deferred any chunk with a `usage` field to the end of the stream, assuming OpenAI's convention where usage arrives once on a final chunk with no choices. vLLM attaches usage to every chunk, so each one was stored into lastChunkWithUsage and never yielded; only the final chunk survived, and its delta is empty. The caller assembled an empty completion with no error, so the request was logged as a success with 0 generated tokens and the UI showed no reply at all. This hit any vLLM-backed provider: openai endpoint. Defer a chunk only when it has no choices, so content chunks pass through with their usage attached and OpenAI's usage-only chunk is still emitted last. Add tests covering all three stream shapes.

Distinguishes local builds carrying the Haven provider and the streaming fix from the 1.3.40 marketplace release, so an installed VSIX is identifiable and VS Code does not treat the marketplace build as an equal-version replacement. Also records the haven-proxy dependency in the gui lockfile, which the Haven provider commit added to core.

coelhogonzalo added 4 commits August 24, 2026 22:16