fix(proxy): record messages stream telemetry

[jarvis9443]

Fixes#359.

When an Anthropic /v1/messages streaming request is routed to an OpenAI-compatible upstream, telemetry now waits for the stream to finish before emitting usage. That lets the proxy record TTFT, token counts, provider request metadata, canonical finish-reason labels, and inbound_protocol = "anthropic" for the protocol-conversion path.

TTFT histogram samples are only recorded when the first-token measurement is known, so non-streaming/default paths do not add synthetic 0ms samples.

This keeps Anthropic-upstream byte-stream passthrough behavior unchanged.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds time-to-first-token (TTFT) measurement for cross-provider streaming by threading a started timestamp through the dispatch chain. Cross-provider streaming now computes TTFT when the first content chunk arrives and emits the Anthropic usage event from the stream completion callback instead of the outer wrapper, controlled by a new usage_handled_by_stream flag on DispatchOutcome.

Changes

Cross-provider Streaming TTFT and Conditional Usage Event Emission

Layer / File(s)	Summary
Data structures for TTFT and conditional emission crates/aisix-proxy/src/messages.rs	DispatchOutcome gains usage_handled_by_stream boolean field; AnthropicUsageMetrics gains ttft_ms: u32 field; new AnthropicStreamCompletion data type and CompleteAnthropicStreamOnDrop drop-guard to ensure callback invocation.
Request entry point and timestamp threading crates/aisix-proxy/src/messages.rs	Handler passes started: Instant to dispatch() and destructures usage_handled_by_stream from DispatchOutcome; dispatch function signature extended to accept started parameter.
Cross-provider streaming with TTFT computation and completion callback crates/aisix-proxy/src/messages.rs	cross_provider_dispatch signature accepts started and identity fields; streaming path calls build_anthropic_sse_stream with on_complete callback that constructs AnthropicUsageMetrics with computed ttft_ms and emits usage event on stream completion; streaming outcome marks usage_handled_by_stream: true.
Conditional usage event emission and TTFT metric recording crates/aisix-proxy/src/messages.rs	Wrapper's usage event emission is gated to run only when usage_handled_by_stream is false; emit_anthropic_usage_event now writes ttft_ms into the UsageEvent and records a corresponding record_time_to_first_token metric.
Anthropic upstream and non-streaming paths crates/aisix-proxy/src/messages.rs	Anthropic upstream and cross-provider non-streaming paths set usage_handled_by_stream: false to keep usage emission in wrapper; non-streaming metrics initialized with ttft_ms: 0.
Cross-provider streaming TTFT test coverage crates/aisix-proxy/src/messages.rs	New test validates that non-Anthropic streaming emits usage event with ttft_ms > 0 and that event is emitted exactly once.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: recording telemetry for messages stream handling when protocol conversion occurs.
Linked Issues check	✅ Passed	The PR addresses all coding requirements from issue #359: recording TTFT for Anthropic→OpenAI streaming, using correct inbound_protocol, and capturing token/usage data.
Out of Scope Changes check	✅ Passed	All changes in messages.rs are directly related to implementing telemetry recording for protocol-converted streaming requests, staying within the scope of issue #359.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3cc5057b-a0df-4ad1-8003-50e10ee4a07f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d68d31 and e9525c9.

📒 Files selected for processing (1)

• crates/aisix-proxy/src/messages.rs

[Copilot]

Pull request overview

This PR fixes telemetry gaps for Anthropic /v1/messagesstreaming requests that are protocol-converted and routed to an OpenAI-compatible upstream. It defers usage/telemetry emission until the upstream stream completes so the proxy can record TTFT, token counts, provider request metadata, and correctly tag inbound_protocol = "anthropic" for the conversion path (while keeping Anthropic-upstream byte-stream passthrough unchanged).

Changes:

• Plumbs request start time and identity metadata into the cross-provider streaming path and emits usage on stream completion.
• Adds TTFT tracking for /v1/messages protocol-conversion streaming by observing the first upstream chunk that carries generated output.
• Adds an E2E-style unit test verifying that cross-provider streaming emits a single usage event with TTFT and upstream metadata.

Comments suppressed due to low confidence (1)

crates/aisix-proxy/src/messages.rs:785

• finish_reason is derived via format!("{fr:?}").to_lowercase(), which collapses variants like ContentFilter/ToolCalls into contentfilter/toolcalls (missing underscores) and can diverge from the canonical wire labels used elsewhere (e.g., chat.rs’ finish_reason_label). This will produce inconsistent UsageEvent.finish_reason values for /v1/messages protocol-conversion streams. Consider reusing the same mapping (stop, length, content_filter, tool_calls, other) here.

 if let Some(fr) = chunk.finish_reason.as_ref() {
comp.finish_reason = format!("{fr:?}").to_lowercase();
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[jarvis9443]

Fixes#359.

When an Anthropic /v1/messages streaming request is routed to an OpenAI-compatible upstream, telemetry now waits for the stream to finish before emitting usage. That lets the proxy record TTFT, token counts, provider request metadata, canonical finish-reason labels, and inbound_protocol = "anthropic" for the protocol-conversion path.

TTFT histogram samples are only recorded when the first-token measurement is known, so non-streaming/default paths do not add synthetic 0ms samples.

This keeps Anthropic-upstream byte-stream passthrough behavior unchanged.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds time-to-first-token (TTFT) measurement for cross-provider streaming by threading a started timestamp through the dispatch chain. Cross-provider streaming now computes TTFT when the first content chunk arrives and emits the Anthropic usage event from the stream completion callback instead of the outer wrapper, controlled by a new usage_handled_by_stream flag on DispatchOutcome.

Changes

Cross-provider Streaming TTFT and Conditional Usage Event Emission

Layer / File(s)	Summary
Data structures for TTFT and conditional emission crates/aisix-proxy/src/messages.rs	DispatchOutcome gains usage_handled_by_stream boolean field; AnthropicUsageMetrics gains ttft_ms: u32 field; new AnthropicStreamCompletion data type and CompleteAnthropicStreamOnDrop drop-guard to ensure callback invocation.
Request entry point and timestamp threading crates/aisix-proxy/src/messages.rs	Handler passes started: Instant to dispatch() and destructures usage_handled_by_stream from DispatchOutcome; dispatch function signature extended to accept started parameter.
Cross-provider streaming with TTFT computation and completion callback crates/aisix-proxy/src/messages.rs	cross_provider_dispatch signature accepts started and identity fields; streaming path calls build_anthropic_sse_stream with on_complete callback that constructs AnthropicUsageMetrics with computed ttft_ms and emits usage event on stream completion; streaming outcome marks usage_handled_by_stream: true.
Conditional usage event emission and TTFT metric recording crates/aisix-proxy/src/messages.rs	Wrapper's usage event emission is gated to run only when usage_handled_by_stream is false; emit_anthropic_usage_event now writes ttft_ms into the UsageEvent and records a corresponding record_time_to_first_token metric.
Anthropic upstream and non-streaming paths crates/aisix-proxy/src/messages.rs	Anthropic upstream and cross-provider non-streaming paths set usage_handled_by_stream: false to keep usage emission in wrapper; non-streaming metrics initialized with ttft_ms: 0.
Cross-provider streaming TTFT test coverage crates/aisix-proxy/src/messages.rs	New test validates that non-Anthropic streaming emits usage event with ttft_ms > 0 and that event is emitted exactly once.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: recording telemetry for messages stream handling when protocol conversion occurs.
Linked Issues check	✅ Passed	The PR addresses all coding requirements from issue #359: recording TTFT for Anthropic→OpenAI streaming, using correct inbound_protocol, and capturing token/usage data.
Out of Scope Changes check	✅ Passed	All changes in messages.rs are directly related to implementing telemetry recording for protocol-converted streaming requests, staying within the scope of issue #359.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3cc5057b-a0df-4ad1-8003-50e10ee4a07f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d68d31 and e9525c9.

📒 Files selected for processing (1)

• crates/aisix-proxy/src/messages.rs

[Copilot]

Pull request overview

This PR fixes telemetry gaps for Anthropic /v1/messagesstreaming requests that are protocol-converted and routed to an OpenAI-compatible upstream. It defers usage/telemetry emission until the upstream stream completes so the proxy can record TTFT, token counts, provider request metadata, and correctly tag inbound_protocol = "anthropic" for the conversion path (while keeping Anthropic-upstream byte-stream passthrough unchanged).

Changes:

• Plumbs request start time and identity metadata into the cross-provider streaming path and emits usage on stream completion.
• Adds TTFT tracking for /v1/messages protocol-conversion streaming by observing the first upstream chunk that carries generated output.
• Adds an E2E-style unit test verifying that cross-provider streaming emits a single usage event with TTFT and upstream metadata.

Comments suppressed due to low confidence (1)

crates/aisix-proxy/src/messages.rs:785

• finish_reason is derived via format!("{fr:?}").to_lowercase(), which collapses variants like ContentFilter/ToolCalls into contentfilter/toolcalls (missing underscores) and can diverge from the canonical wire labels used elsewhere (e.g., chat.rs’ finish_reason_label). This will produce inconsistent UsageEvent.finish_reason values for /v1/messages protocol-conversion streams. Consider reusing the same mapping (stop, length, content_filter, tool_calls, other) here.

 if let Some(fr) = chunk.finish_reason.as_ref() {
comp.finish_reason = format!("{fr:?}").to_lowercase();
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[jarvis9443]

Fixes#359.

When an Anthropic /v1/messages streaming request is routed to an OpenAI-compatible upstream, telemetry now waits for the stream to finish before emitting usage. That lets the proxy record TTFT, token counts, provider request metadata, canonical finish-reason labels, and inbound_protocol = "anthropic" for the protocol-conversion path.

TTFT histogram samples are only recorded when the first-token measurement is known, so non-streaming/default paths do not add synthetic 0ms samples.

This keeps Anthropic-upstream byte-stream passthrough behavior unchanged.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds time-to-first-token (TTFT) measurement for cross-provider streaming by threading a started timestamp through the dispatch chain. Cross-provider streaming now computes TTFT when the first content chunk arrives and emits the Anthropic usage event from the stream completion callback instead of the outer wrapper, controlled by a new usage_handled_by_stream flag on DispatchOutcome.

Changes

Cross-provider Streaming TTFT and Conditional Usage Event Emission

Layer / File(s)	Summary
Data structures for TTFT and conditional emission crates/aisix-proxy/src/messages.rs	DispatchOutcome gains usage_handled_by_stream boolean field; AnthropicUsageMetrics gains ttft_ms: u32 field; new AnthropicStreamCompletion data type and CompleteAnthropicStreamOnDrop drop-guard to ensure callback invocation.
Request entry point and timestamp threading crates/aisix-proxy/src/messages.rs	Handler passes started: Instant to dispatch() and destructures usage_handled_by_stream from DispatchOutcome; dispatch function signature extended to accept started parameter.
Cross-provider streaming with TTFT computation and completion callback crates/aisix-proxy/src/messages.rs	cross_provider_dispatch signature accepts started and identity fields; streaming path calls build_anthropic_sse_stream with on_complete callback that constructs AnthropicUsageMetrics with computed ttft_ms and emits usage event on stream completion; streaming outcome marks usage_handled_by_stream: true.
Conditional usage event emission and TTFT metric recording crates/aisix-proxy/src/messages.rs	Wrapper's usage event emission is gated to run only when usage_handled_by_stream is false; emit_anthropic_usage_event now writes ttft_ms into the UsageEvent and records a corresponding record_time_to_first_token metric.
Anthropic upstream and non-streaming paths crates/aisix-proxy/src/messages.rs	Anthropic upstream and cross-provider non-streaming paths set usage_handled_by_stream: false to keep usage emission in wrapper; non-streaming metrics initialized with ttft_ms: 0.
Cross-provider streaming TTFT test coverage crates/aisix-proxy/src/messages.rs	New test validates that non-Anthropic streaming emits usage event with ttft_ms > 0 and that event is emitted exactly once.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: recording telemetry for messages stream handling when protocol conversion occurs.
Linked Issues check	✅ Passed	The PR addresses all coding requirements from issue #359: recording TTFT for Anthropic→OpenAI streaming, using correct inbound_protocol, and capturing token/usage data.
Out of Scope Changes check	✅ Passed	All changes in messages.rs are directly related to implementing telemetry recording for protocol-converted streaming requests, staying within the scope of issue #359.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3cc5057b-a0df-4ad1-8003-50e10ee4a07f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d68d31 and e9525c9.

📒 Files selected for processing (1)

• crates/aisix-proxy/src/messages.rs

[Copilot]

Pull request overview

This PR fixes telemetry gaps for Anthropic /v1/messagesstreaming requests that are protocol-converted and routed to an OpenAI-compatible upstream. It defers usage/telemetry emission until the upstream stream completes so the proxy can record TTFT, token counts, provider request metadata, and correctly tag inbound_protocol = "anthropic" for the conversion path (while keeping Anthropic-upstream byte-stream passthrough unchanged).

Changes:

• Plumbs request start time and identity metadata into the cross-provider streaming path and emits usage on stream completion.
• Adds TTFT tracking for /v1/messages protocol-conversion streaming by observing the first upstream chunk that carries generated output.
• Adds an E2E-style unit test verifying that cross-provider streaming emits a single usage event with TTFT and upstream metadata.

Comments suppressed due to low confidence (1)

crates/aisix-proxy/src/messages.rs:785

• finish_reason is derived via format!("{fr:?}").to_lowercase(), which collapses variants like ContentFilter/ToolCalls into contentfilter/toolcalls (missing underscores) and can diverge from the canonical wire labels used elsewhere (e.g., chat.rs’ finish_reason_label). This will produce inconsistent UsageEvent.finish_reason values for /v1/messages protocol-conversion streams. Consider reusing the same mapping (stop, length, content_filter, tool_calls, other) here.

 if let Some(fr) = chunk.finish_reason.as_ref() {
comp.finish_reason = format!("{fr:?}").to_lowercase();
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[jarvis9443]

Fixes#359.

When an Anthropic /v1/messages streaming request is routed to an OpenAI-compatible upstream, telemetry now waits for the stream to finish before emitting usage. That lets the proxy record TTFT, token counts, provider request metadata, canonical finish-reason labels, and inbound_protocol = "anthropic" for the protocol-conversion path.

TTFT histogram samples are only recorded when the first-token measurement is known, so non-streaming/default paths do not add synthetic 0ms samples.

This keeps Anthropic-upstream byte-stream passthrough behavior unchanged.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds time-to-first-token (TTFT) measurement for cross-provider streaming by threading a started timestamp through the dispatch chain. Cross-provider streaming now computes TTFT when the first content chunk arrives and emits the Anthropic usage event from the stream completion callback instead of the outer wrapper, controlled by a new usage_handled_by_stream flag on DispatchOutcome.

Changes

Cross-provider Streaming TTFT and Conditional Usage Event Emission

Layer / File(s)	Summary
Data structures for TTFT and conditional emission crates/aisix-proxy/src/messages.rs	DispatchOutcome gains usage_handled_by_stream boolean field; AnthropicUsageMetrics gains ttft_ms: u32 field; new AnthropicStreamCompletion data type and CompleteAnthropicStreamOnDrop drop-guard to ensure callback invocation.
Request entry point and timestamp threading crates/aisix-proxy/src/messages.rs	Handler passes started: Instant to dispatch() and destructures usage_handled_by_stream from DispatchOutcome; dispatch function signature extended to accept started parameter.
Cross-provider streaming with TTFT computation and completion callback crates/aisix-proxy/src/messages.rs	cross_provider_dispatch signature accepts started and identity fields; streaming path calls build_anthropic_sse_stream with on_complete callback that constructs AnthropicUsageMetrics with computed ttft_ms and emits usage event on stream completion; streaming outcome marks usage_handled_by_stream: true.
Conditional usage event emission and TTFT metric recording crates/aisix-proxy/src/messages.rs	Wrapper's usage event emission is gated to run only when usage_handled_by_stream is false; emit_anthropic_usage_event now writes ttft_ms into the UsageEvent and records a corresponding record_time_to_first_token metric.
Anthropic upstream and non-streaming paths crates/aisix-proxy/src/messages.rs	Anthropic upstream and cross-provider non-streaming paths set usage_handled_by_stream: false to keep usage emission in wrapper; non-streaming metrics initialized with ttft_ms: 0.
Cross-provider streaming TTFT test coverage crates/aisix-proxy/src/messages.rs	New test validates that non-Anthropic streaming emits usage event with ttft_ms > 0 and that event is emitted exactly once.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: recording telemetry for messages stream handling when protocol conversion occurs.
Linked Issues check	✅ Passed	The PR addresses all coding requirements from issue #359: recording TTFT for Anthropic→OpenAI streaming, using correct inbound_protocol, and capturing token/usage data.
Out of Scope Changes check	✅ Passed	All changes in messages.rs are directly related to implementing telemetry recording for protocol-converted streaming requests, staying within the scope of issue #359.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3cc5057b-a0df-4ad1-8003-50e10ee4a07f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d68d31 and e9525c9.

📒 Files selected for processing (1)

• crates/aisix-proxy/src/messages.rs

[Copilot]

Pull request overview

This PR fixes telemetry gaps for Anthropic /v1/messagesstreaming requests that are protocol-converted and routed to an OpenAI-compatible upstream. It defers usage/telemetry emission until the upstream stream completes so the proxy can record TTFT, token counts, provider request metadata, and correctly tag inbound_protocol = "anthropic" for the conversion path (while keeping Anthropic-upstream byte-stream passthrough unchanged).

Changes:

• Plumbs request start time and identity metadata into the cross-provider streaming path and emits usage on stream completion.
• Adds TTFT tracking for /v1/messages protocol-conversion streaming by observing the first upstream chunk that carries generated output.
• Adds an E2E-style unit test verifying that cross-provider streaming emits a single usage event with TTFT and upstream metadata.

Comments suppressed due to low confidence (1)

crates/aisix-proxy/src/messages.rs:785

• finish_reason is derived via format!("{fr:?}").to_lowercase(), which collapses variants like ContentFilter/ToolCalls into contentfilter/toolcalls (missing underscores) and can diverge from the canonical wire labels used elsewhere (e.g., chat.rs’ finish_reason_label). This will produce inconsistent UsageEvent.finish_reason values for /v1/messages protocol-conversion streams. Consider reusing the same mapping (stop, length, content_filter, tool_calls, other) here.

 if let Some(fr) = chunk.finish_reason.as_ref() {
comp.finish_reason = format!("{fr:?}").to_lowercase();
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[jarvis9443]

Fixes#359.

When an Anthropic /v1/messages streaming request is routed to an OpenAI-compatible upstream, telemetry now waits for the stream to finish before emitting usage. That lets the proxy record TTFT, token counts, provider request metadata, canonical finish-reason labels, and inbound_protocol = "anthropic" for the protocol-conversion path.

TTFT histogram samples are only recorded when the first-token measurement is known, so non-streaming/default paths do not add synthetic 0ms samples.

This keeps Anthropic-upstream byte-stream passthrough behavior unchanged.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds time-to-first-token (TTFT) measurement for cross-provider streaming by threading a started timestamp through the dispatch chain. Cross-provider streaming now computes TTFT when the first content chunk arrives and emits the Anthropic usage event from the stream completion callback instead of the outer wrapper, controlled by a new usage_handled_by_stream flag on DispatchOutcome.

Changes

Cross-provider Streaming TTFT and Conditional Usage Event Emission

Layer / File(s)	Summary
Data structures for TTFT and conditional emission crates/aisix-proxy/src/messages.rs	DispatchOutcome gains usage_handled_by_stream boolean field; AnthropicUsageMetrics gains ttft_ms: u32 field; new AnthropicStreamCompletion data type and CompleteAnthropicStreamOnDrop drop-guard to ensure callback invocation.
Request entry point and timestamp threading crates/aisix-proxy/src/messages.rs	Handler passes started: Instant to dispatch() and destructures usage_handled_by_stream from DispatchOutcome; dispatch function signature extended to accept started parameter.
Cross-provider streaming with TTFT computation and completion callback crates/aisix-proxy/src/messages.rs	cross_provider_dispatch signature accepts started and identity fields; streaming path calls build_anthropic_sse_stream with on_complete callback that constructs AnthropicUsageMetrics with computed ttft_ms and emits usage event on stream completion; streaming outcome marks usage_handled_by_stream: true.
Conditional usage event emission and TTFT metric recording crates/aisix-proxy/src/messages.rs	Wrapper's usage event emission is gated to run only when usage_handled_by_stream is false; emit_anthropic_usage_event now writes ttft_ms into the UsageEvent and records a corresponding record_time_to_first_token metric.
Anthropic upstream and non-streaming paths crates/aisix-proxy/src/messages.rs	Anthropic upstream and cross-provider non-streaming paths set usage_handled_by_stream: false to keep usage emission in wrapper; non-streaming metrics initialized with ttft_ms: 0.
Cross-provider streaming TTFT test coverage crates/aisix-proxy/src/messages.rs	New test validates that non-Anthropic streaming emits usage event with ttft_ms > 0 and that event is emitted exactly once.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: recording telemetry for messages stream handling when protocol conversion occurs.
Linked Issues check	✅ Passed	The PR addresses all coding requirements from issue #359: recording TTFT for Anthropic→OpenAI streaming, using correct inbound_protocol, and capturing token/usage data.
Out of Scope Changes check	✅ Passed	All changes in messages.rs are directly related to implementing telemetry recording for protocol-converted streaming requests, staying within the scope of issue #359.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3cc5057b-a0df-4ad1-8003-50e10ee4a07f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d68d31 and e9525c9.

📒 Files selected for processing (1)

• crates/aisix-proxy/src/messages.rs

[Copilot]

Pull request overview

This PR fixes telemetry gaps for Anthropic /v1/messagesstreaming requests that are protocol-converted and routed to an OpenAI-compatible upstream. It defers usage/telemetry emission until the upstream stream completes so the proxy can record TTFT, token counts, provider request metadata, and correctly tag inbound_protocol = "anthropic" for the conversion path (while keeping Anthropic-upstream byte-stream passthrough unchanged).

Changes:

• Plumbs request start time and identity metadata into the cross-provider streaming path and emits usage on stream completion.
• Adds TTFT tracking for /v1/messages protocol-conversion streaming by observing the first upstream chunk that carries generated output.
• Adds an E2E-style unit test verifying that cross-provider streaming emits a single usage event with TTFT and upstream metadata.

Comments suppressed due to low confidence (1)

crates/aisix-proxy/src/messages.rs:785

• finish_reason is derived via format!("{fr:?}").to_lowercase(), which collapses variants like ContentFilter/ToolCalls into contentfilter/toolcalls (missing underscores) and can diverge from the canonical wire labels used elsewhere (e.g., chat.rs’ finish_reason_label). This will produce inconsistent UsageEvent.finish_reason values for /v1/messages protocol-conversion streams. Consider reusing the same mapping (stop, length, content_filter, tool_calls, other) here.

 if let Some(fr) = chunk.finish_reason.as_ref() {
comp.finish_reason = format!("{fr:?}").to_lowercase();
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[jarvis9443]

Fixes#359.

When an Anthropic /v1/messages streaming request is routed to an OpenAI-compatible upstream, telemetry now waits for the stream to finish before emitting usage. That lets the proxy record TTFT, token counts, provider request metadata, canonical finish-reason labels, and inbound_protocol = "anthropic" for the protocol-conversion path.

TTFT histogram samples are only recorded when the first-token measurement is known, so non-streaming/default paths do not add synthetic 0ms samples.

This keeps Anthropic-upstream byte-stream passthrough behavior unchanged.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds time-to-first-token (TTFT) measurement for cross-provider streaming by threading a started timestamp through the dispatch chain. Cross-provider streaming now computes TTFT when the first content chunk arrives and emits the Anthropic usage event from the stream completion callback instead of the outer wrapper, controlled by a new usage_handled_by_stream flag on DispatchOutcome.

Changes

Cross-provider Streaming TTFT and Conditional Usage Event Emission

Layer / File(s)	Summary
Data structures for TTFT and conditional emission crates/aisix-proxy/src/messages.rs	DispatchOutcome gains usage_handled_by_stream boolean field; AnthropicUsageMetrics gains ttft_ms: u32 field; new AnthropicStreamCompletion data type and CompleteAnthropicStreamOnDrop drop-guard to ensure callback invocation.
Request entry point and timestamp threading crates/aisix-proxy/src/messages.rs	Handler passes started: Instant to dispatch() and destructures usage_handled_by_stream from DispatchOutcome; dispatch function signature extended to accept started parameter.
Cross-provider streaming with TTFT computation and completion callback crates/aisix-proxy/src/messages.rs	cross_provider_dispatch signature accepts started and identity fields; streaming path calls build_anthropic_sse_stream with on_complete callback that constructs AnthropicUsageMetrics with computed ttft_ms and emits usage event on stream completion; streaming outcome marks usage_handled_by_stream: true.
Conditional usage event emission and TTFT metric recording crates/aisix-proxy/src/messages.rs	Wrapper's usage event emission is gated to run only when usage_handled_by_stream is false; emit_anthropic_usage_event now writes ttft_ms into the UsageEvent and records a corresponding record_time_to_first_token metric.
Anthropic upstream and non-streaming paths crates/aisix-proxy/src/messages.rs	Anthropic upstream and cross-provider non-streaming paths set usage_handled_by_stream: false to keep usage emission in wrapper; non-streaming metrics initialized with ttft_ms: 0.
Cross-provider streaming TTFT test coverage crates/aisix-proxy/src/messages.rs	New test validates that non-Anthropic streaming emits usage event with ttft_ms > 0 and that event is emitted exactly once.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: recording telemetry for messages stream handling when protocol conversion occurs.
Linked Issues check	✅ Passed	The PR addresses all coding requirements from issue #359: recording TTFT for Anthropic→OpenAI streaming, using correct inbound_protocol, and capturing token/usage data.
Out of Scope Changes check	✅ Passed	All changes in messages.rs are directly related to implementing telemetry recording for protocol-converted streaming requests, staying within the scope of issue #359.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3cc5057b-a0df-4ad1-8003-50e10ee4a07f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d68d31 and e9525c9.

📒 Files selected for processing (1)

• crates/aisix-proxy/src/messages.rs

[Copilot]

Pull request overview

This PR fixes telemetry gaps for Anthropic /v1/messagesstreaming requests that are protocol-converted and routed to an OpenAI-compatible upstream. It defers usage/telemetry emission until the upstream stream completes so the proxy can record TTFT, token counts, provider request metadata, and correctly tag inbound_protocol = "anthropic" for the conversion path (while keeping Anthropic-upstream byte-stream passthrough unchanged).

Changes:

• Plumbs request start time and identity metadata into the cross-provider streaming path and emits usage on stream completion.
• Adds TTFT tracking for /v1/messages protocol-conversion streaming by observing the first upstream chunk that carries generated output.
• Adds an E2E-style unit test verifying that cross-provider streaming emits a single usage event with TTFT and upstream metadata.

Comments suppressed due to low confidence (1)

crates/aisix-proxy/src/messages.rs:785

• finish_reason is derived via format!("{fr:?}").to_lowercase(), which collapses variants like ContentFilter/ToolCalls into contentfilter/toolcalls (missing underscores) and can diverge from the canonical wire labels used elsewhere (e.g., chat.rs’ finish_reason_label). This will produce inconsistent UsageEvent.finish_reason values for /v1/messages protocol-conversion streams. Consider reusing the same mapping (stop, length, content_filter, tool_calls, other) here.

 if let Some(fr) = chunk.finish_reason.as_ref() {
comp.finish_reason = format!("{fr:?}").to_lowercase();
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[jarvis9443]

Fixes#359.

When an Anthropic /v1/messages streaming request is routed to an OpenAI-compatible upstream, telemetry now waits for the stream to finish before emitting usage. That lets the proxy record TTFT, token counts, provider request metadata, canonical finish-reason labels, and inbound_protocol = "anthropic" for the protocol-conversion path.

TTFT histogram samples are only recorded when the first-token measurement is known, so non-streaming/default paths do not add synthetic 0ms samples.

This keeps Anthropic-upstream byte-stream passthrough behavior unchanged.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds time-to-first-token (TTFT) measurement for cross-provider streaming by threading a started timestamp through the dispatch chain. Cross-provider streaming now computes TTFT when the first content chunk arrives and emits the Anthropic usage event from the stream completion callback instead of the outer wrapper, controlled by a new usage_handled_by_stream flag on DispatchOutcome.

Changes

Cross-provider Streaming TTFT and Conditional Usage Event Emission

Layer / File(s)	Summary
Data structures for TTFT and conditional emission crates/aisix-proxy/src/messages.rs	DispatchOutcome gains usage_handled_by_stream boolean field; AnthropicUsageMetrics gains ttft_ms: u32 field; new AnthropicStreamCompletion data type and CompleteAnthropicStreamOnDrop drop-guard to ensure callback invocation.
Request entry point and timestamp threading crates/aisix-proxy/src/messages.rs	Handler passes started: Instant to dispatch() and destructures usage_handled_by_stream from DispatchOutcome; dispatch function signature extended to accept started parameter.
Cross-provider streaming with TTFT computation and completion callback crates/aisix-proxy/src/messages.rs	cross_provider_dispatch signature accepts started and identity fields; streaming path calls build_anthropic_sse_stream with on_complete callback that constructs AnthropicUsageMetrics with computed ttft_ms and emits usage event on stream completion; streaming outcome marks usage_handled_by_stream: true.
Conditional usage event emission and TTFT metric recording crates/aisix-proxy/src/messages.rs	Wrapper's usage event emission is gated to run only when usage_handled_by_stream is false; emit_anthropic_usage_event now writes ttft_ms into the UsageEvent and records a corresponding record_time_to_first_token metric.
Anthropic upstream and non-streaming paths crates/aisix-proxy/src/messages.rs	Anthropic upstream and cross-provider non-streaming paths set usage_handled_by_stream: false to keep usage emission in wrapper; non-streaming metrics initialized with ttft_ms: 0.
Cross-provider streaming TTFT test coverage crates/aisix-proxy/src/messages.rs	New test validates that non-Anthropic streaming emits usage event with ttft_ms > 0 and that event is emitted exactly once.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: recording telemetry for messages stream handling when protocol conversion occurs.
Linked Issues check	✅ Passed	The PR addresses all coding requirements from issue #359: recording TTFT for Anthropic→OpenAI streaming, using correct inbound_protocol, and capturing token/usage data.
Out of Scope Changes check	✅ Passed	All changes in messages.rs are directly related to implementing telemetry recording for protocol-converted streaming requests, staying within the scope of issue #359.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3cc5057b-a0df-4ad1-8003-50e10ee4a07f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d68d31 and e9525c9.

📒 Files selected for processing (1)

• crates/aisix-proxy/src/messages.rs

[Copilot]

Pull request overview

This PR fixes telemetry gaps for Anthropic /v1/messagesstreaming requests that are protocol-converted and routed to an OpenAI-compatible upstream. It defers usage/telemetry emission until the upstream stream completes so the proxy can record TTFT, token counts, provider request metadata, and correctly tag inbound_protocol = "anthropic" for the conversion path (while keeping Anthropic-upstream byte-stream passthrough unchanged).

Changes:

• Plumbs request start time and identity metadata into the cross-provider streaming path and emits usage on stream completion.
• Adds TTFT tracking for /v1/messages protocol-conversion streaming by observing the first upstream chunk that carries generated output.
• Adds an E2E-style unit test verifying that cross-provider streaming emits a single usage event with TTFT and upstream metadata.

Comments suppressed due to low confidence (1)

crates/aisix-proxy/src/messages.rs:785

• finish_reason is derived via format!("{fr:?}").to_lowercase(), which collapses variants like ContentFilter/ToolCalls into contentfilter/toolcalls (missing underscores) and can diverge from the canonical wire labels used elsewhere (e.g., chat.rs’ finish_reason_label). This will produce inconsistent UsageEvent.finish_reason values for /v1/messages protocol-conversion streams. Consider reusing the same mapping (stop, length, content_filter, tool_calls, other) here.

 if let Some(fr) = chunk.finish_reason.as_ref() {
comp.finish_reason = format!("{fr:?}").to_lowercase();
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

[jarvis9443]

Fixes#359.

When an Anthropic /v1/messages streaming request is routed to an OpenAI-compatible upstream, telemetry now waits for the stream to finish before emitting usage. That lets the proxy record TTFT, token counts, provider request metadata, canonical finish-reason labels, and inbound_protocol = "anthropic" for the protocol-conversion path.

TTFT histogram samples are only recorded when the first-token measurement is known, so non-streaming/default paths do not add synthetic 0ms samples.

This keeps Anthropic-upstream byte-stream passthrough behavior unchanged.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR adds time-to-first-token (TTFT) measurement for cross-provider streaming by threading a started timestamp through the dispatch chain. Cross-provider streaming now computes TTFT when the first content chunk arrives and emits the Anthropic usage event from the stream completion callback instead of the outer wrapper, controlled by a new usage_handled_by_stream flag on DispatchOutcome.

Changes

Cross-provider Streaming TTFT and Conditional Usage Event Emission

Layer / File(s)	Summary
Data structures for TTFT and conditional emission crates/aisix-proxy/src/messages.rs	DispatchOutcome gains usage_handled_by_stream boolean field; AnthropicUsageMetrics gains ttft_ms: u32 field; new AnthropicStreamCompletion data type and CompleteAnthropicStreamOnDrop drop-guard to ensure callback invocation.
Request entry point and timestamp threading crates/aisix-proxy/src/messages.rs	Handler passes started: Instant to dispatch() and destructures usage_handled_by_stream from DispatchOutcome; dispatch function signature extended to accept started parameter.
Cross-provider streaming with TTFT computation and completion callback crates/aisix-proxy/src/messages.rs	cross_provider_dispatch signature accepts started and identity fields; streaming path calls build_anthropic_sse_stream with on_complete callback that constructs AnthropicUsageMetrics with computed ttft_ms and emits usage event on stream completion; streaming outcome marks usage_handled_by_stream: true.
Conditional usage event emission and TTFT metric recording crates/aisix-proxy/src/messages.rs	Wrapper's usage event emission is gated to run only when usage_handled_by_stream is false; emit_anthropic_usage_event now writes ttft_ms into the UsageEvent and records a corresponding record_time_to_first_token metric.
Anthropic upstream and non-streaming paths crates/aisix-proxy/src/messages.rs	Anthropic upstream and cross-provider non-streaming paths set usage_handled_by_stream: false to keep usage emission in wrapper; non-streaming metrics initialized with ttft_ms: 0.
Cross-provider streaming TTFT test coverage crates/aisix-proxy/src/messages.rs	New test validates that non-Anthropic streaming emits usage event with ttft_ms > 0 and that event is emitted exactly once.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: recording telemetry for messages stream handling when protocol conversion occurs.
Linked Issues check	✅ Passed	The PR addresses all coding requirements from issue #359: recording TTFT for Anthropic→OpenAI streaming, using correct inbound_protocol, and capturing token/usage data.
Out of Scope Changes check	✅ Passed	All changes in messages.rs are directly related to implementing telemetry recording for protocol-converted streaming requests, staying within the scope of issue #359.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3cc5057b-a0df-4ad1-8003-50e10ee4a07f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d68d31 and e9525c9.

📒 Files selected for processing (1)

• crates/aisix-proxy/src/messages.rs

[Copilot]

Pull request overview

This PR fixes telemetry gaps for Anthropic /v1/messagesstreaming requests that are protocol-converted and routed to an OpenAI-compatible upstream. It defers usage/telemetry emission until the upstream stream completes so the proxy can record TTFT, token counts, provider request metadata, and correctly tag inbound_protocol = "anthropic" for the conversion path (while keeping Anthropic-upstream byte-stream passthrough unchanged).

Changes:

• Plumbs request start time and identity metadata into the cross-provider streaming path and emits usage on stream completion.
• Adds TTFT tracking for /v1/messages protocol-conversion streaming by observing the first upstream chunk that carries generated output.
• Adds an E2E-style unit test verifying that cross-provider streaming emits a single usage event with TTFT and upstream metadata.

Comments suppressed due to low confidence (1)

crates/aisix-proxy/src/messages.rs:785

• finish_reason is derived via format!("{fr:?}").to_lowercase(), which collapses variants like ContentFilter/ToolCalls into contentfilter/toolcalls (missing underscores) and can diverge from the canonical wire labels used elsewhere (e.g., chat.rs’ finish_reason_label). This will produce inconsistent UsageEvent.finish_reason values for /v1/messages protocol-conversion streams. Consider reusing the same mapping (stop, length, content_filter, tool_calls, other) here.

 if let Some(fr) = chunk.finish_reason.as_ref() {
comp.finish_reason = format!("{fr:?}").to_lowercase();
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.