Skip to content

fix: fallback adapters report the active instance's model/provider - #6690

Open
Fyko wants to merge 2 commits into
livekit:mainfrom
petsinc:fallback-adapter-model-labels
Open

fix: fallback adapters report the active instance's model/provider#6690
Fyko wants to merge 2 commits into
livekit:mainfrom
petsinc:fallback-adapter-model-labels

Conversation

@Fyko

@FykoFyko commented Aug 3, 2026

Copy link
Copy Markdown

What

STT/TTS/LLM/realtime fallback adapters now report the model/provider of the underlying instance that most recently served a request, instead of the hardcoded "FallbackAdapter"/"livekit".

Why

Turn metrics and spans read model/provider off the top-level component, so with a fallback adapter every lk.agents.turn.* histogram collapses into one model_name="FallbackAdapter" series — you can't tell your primary's ttfb from your fallback's. Discussed with the LiveKit team in slack; surfacing the actual provider was agreed on.

Semantics: the primary is reported before any traffic, the last instance to produce output after that. Background recovery probes never relabel — the caller never saw their output. The serving streams mark the active instance directly (same delegation pattern as stt.StreamAdapter and the realtime adapter's capabilities), no event listening needed.

Tests added for all four adapters, including probe-doesn't-relabel and label-follows-recovery cases.

@Fyko
Fyko requested a review from a team as a code ownerAugust 3, 2026 22:45
@CLAassistant

CLAassistant commented Aug 3, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

devin-ai-integration[bot]

This comment was marked as resolved.

def model(self) -> str:
return "FallbackAdapter"
"""The model of the instance that most recently served a request (the primary before any traffic).""" # noqa: E501
return self._active_instance.model

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nested llm_request span already carries the real gen_ai.request.model, so model can stay "FallbackAdapter".

The only gap is lk.agents.turn.*, whose labels come from MetricsReport at three sites (agent_activity:3380, :3385, :4457). Maybe add a single metrics_metadata property, read at those three sites, splits the histograms and keeps model, ErrorEvent, and the user_turn span stable.

ifTYPE_CHECKING:
from ..llm.chat_contextimportMetricsMetadata@propertydefmetrics_metadata(self) ->MetricsMetadata: # base classes return their own model/providerreturnself._active_instance.metrics_metadata

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@Fyko@CLAassistant@longcw