Skip to content
This repository was archived by the owner on May 15, 2026. It is now read-only.

Fix Groq context window display - #7839

Merged
mrubens merged 2 commits into
mainfrom
fix_groq_input_token_counting
Sep 10, 2025
Merged

Fix Groq context window display#7839
mrubens merged 2 commits into
mainfrom
fix_groq_input_token_counting

Conversation

@mrubens

@mrubensmrubens commented Sep 10, 2025

Copy link
Copy Markdown
Collaborator

Important

Fix GroqHandler to report full input token count without subtracting cached tokens.

  • Behavior:
    • GroqHandler in groq.ts now reports full inputTokens without subtracting cached tokens in yieldUsage().
    • Updates test in groq.spec.ts to expect full inputTokens count in createMessage test.
  • Tests:
    • Modify createMessage should handle cached tokens in usage data test in groq.spec.ts to expect inputTokens: 100 instead of 70.

This description was created by Ellipsis for 417a40f. You can customize this summary. It will automatically update as commits are pushed.

@mrubens
mrubens requested review from cte and jr as code ownersSeptember 10, 2025 04:20
@dosubotdosubotBot added size:S This PR changes 10-29 lines, ignoring generated files. bug Something isn't working labels Sep 10, 2025
@mrubens
mrubensforce-pushed the fix_groq_input_token_counting branch from 2b96d53 to 417a40fCompareSeptember 10, 2025 04:24

ghost left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution! I've reviewed the changes and have some questions and suggestions for improvement.

yield {
type: "usage",
inputTokens: nonCachedInputTokens,
inputTokens,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to see the test was updated to match this change! However, could you clarify what specific issue this fixes? The PR title mentions "Fix Groq context window display" but it's not clear what was broken.

Also, is this token reporting approach intentional? This makes Groq's behavior different from how it previously worked (subtracting cached tokens from input tokens). The base OpenAI provider reports total tokens, but some other providers handle cached tokens differently. Could you explain the reasoning behind this change?

Comment threadsrc/api/providers/__tests__/groq.spec.ts Outdated
@hannesrudolphhannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Sep 10, 2025
@dosubotdosubotBot added the lgtm This PR has been approved by a maintainer label Sep 10, 2025
@daniel-lxsdaniel-lxs moved this from Triage to PR [Needs Review] in Roo Code RoadmapSep 10, 2025
Comment threadsrc/api/providers/__tests__/groq.spec.ts Outdated
@hannesrudolphhannesrudolph added PR - Needs Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Sep 10, 2025
@mrubens
mrubens merged commit 7cd6520 into mainSep 10, 2025
@mrubens
mrubens deleted the fix_groq_input_token_counting branch September 10, 2025 04:50
@github-project-automationgithub-project-automationBot moved this from PR [Needs Review] to Done in Roo Code RoadmapSep 10, 2025
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

bugSomething isn't workinglgtmThis PR has been approved by a maintainerPR - Needs Reviewsize:SThis PR changes 10-29 lines, ignoring generated files.

Projects

No open projects
Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants

@mrubens@daniel-lxs@hannesrudolph