Uh oh!
There was an error while loading. Please reload this page.
feat(headless): add DeepSeek Flash harness A/B runtime - #1730
Merged
Conversation
Astro-Hanforce-pushed
the
feat/headless-deepseek-flash-runtime
branch
from
July 31, 2026 19:49
2ef8469 to
5c6f437CompareUh oh!
There was an error while loading. Please reload this page.
This was referenced Aug 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the
deepseek-v4-flash-maxruntime profile onto main, cherry-picked from the local experiment branch that produced the published Terminal-Bench 2.1 comparison (#1719). Main currently cannot re-run that comparison: the report is merged but the runtime that produced it is not.deepseek-v4-flashatmaxreasoning, metered billing against the checked-inDEEPSEEK_V4_FLASH_PRICINGidentityMAKA_HARNESS_AB_DEEPSEEK_KEY_FILE, defaulting to~/.maka/secrets/deepseek.key; each runtime ignores other runtimes' key-file env so a stale export cannot misroute credentialsterminal-bench-2.1 | deepseek-v4-flash-max | opencode; DeepSeek OpenCode routes through the per-cell host proxy capability, never a raw keyThis is the runtime layer only. The DeepSeek-metered DeepSWE composition and the OpenCode Pier arm stack on top in a child PR.
Verification
packages/headlesssuite: 1449 pass / 0 fail (includes the cherry-picked DeepSeek runtime CLI/adapter cases)benchmark=terminal-bench-2.1 (89 frozen tasks via harbor); runtime=deepseek-v4-flash-max (deepseek/deepseek-v4-flash, max, metered); competitor=opencode@1.17.18docs/eval/terminal-bench-2.1-deepseek-v4-flash-maka-vs-opencode.md(run iddeepseek-v4-flash-maka-vs-opencode-tbench-2.1-full-v7)npm run format/npm run lintcleanReview focus
Cherry-pick fidelity: the only conflict against main was the README compositions paragraph (main gained the
deep-swe-1.1-fullsentence in #1726); resolution keeps main's wording and adds the DeepSeek composition. Everything else applied byte-clean.