Spawn and drive DeepSeek Harness workers from the codeman agent skill - #341
Merged
Conversation
`GET /api/sessions/:id/last-response` is how an agent (and the Response Viewer) reads what a worker said. DeepSeek was falling through to the pane segmenter with the other external CLIs, which for this mode is not merely coarse but wrong: dsh-TUI paints a full-screen splash, so a `last-response` call on a fresh dsh session answered with its ASCII-art logo -- and anything polling for a worker's first reply reads that as a reply. dsh does not belong in that group. It writes a structured JSONL transcript per session, so read it. Four things in that file shaped the reader, all measured against real transcripts on disk: 1. dsh appends ONE ZSTD FRAME PER WRITE, and Node's zlib zstd decoder (one-shot and streaming alike) stops at the first frame end: a real 56-line transcript decoded as 1 line / 158 bytes -- the session header alone, i.e. a silent truncation that reads as "nothing said yet" forever. `zstdFrameRanges()` walks frame and block headers to find exact boundaries; splitting on the 4-byte magic would corrupt everything after a magic sequence occurring inside compressed data. zstd is resolved at RUNTIME because it landed in Node 22.15 while the project floor is 22.0, so an older Node keeps the pane behaviour. 2. Every turn also records a plugin-sourced `user/message` (the runtime context snapshot), which must not render as the user's own words. 3. A turn that ends in an error carries the provider's message; it is surfaced as `Turn error: …` (and a non-error early stop as `Turn ended: …`) rather than as an empty string, which an agent reads as "still thinking" through fifteen polls. 4. Reply text is assembled per (turn, step): a finalized message wins and the streamed deltas fill in only for a step that never finalized, so a partial answer is readable mid-turn and never doubled. "Finalized" is tracked as a set of steps rather than as non-empty text, because a step whose whole reply was reasoning strips to '' at the `</think>` boundary and would otherwise resurrect the raw deltas in its place. Session-to-transcript pairing is by the transcript's own header `cwd` plus a boot window against the session's createdAt, never by reproducing dsh's directory mangling (already two forms on disk) and never by newest-mtime alone -- mtime alone handed a freshly spawned worker its predecessor's answer in the same case directory. An empty result still wins over the pane; only a Node that cannot decode zstd falls back to it.
The agent skill could spawn a worker in any mode, but it could only DRIVE a claude one: every other CLI has neither a real end-of-turn signal nor an answer to read, so the recipes route them through output markers. dsh has both halves now -- its harness reports idle/working/blocked to Codeman, and the previous commit reads its transcript -- so it joins claude as a mode the four verbs work on unchanged. `spawn_workers alpha beta:deepseek` is a mixed fleet in one call, and `sendwait` / `last_text` / `delete_session` need no per-mode variant. Preamble 1.20.0 (SKILL.md's §0 heredoc regenerated from it): - `spawn_worker` grows a deepseek branch that gates on the harness composer.⚠️ Readiness there is NOT the stop signal: the harness reports idle at BOOT ~300 ms before its composer paints (measured 2.26 s vs 2.56 s after spawn), so a send-and-wait fired straight after quick-start resolves on the boot edge, reports a turn that never ran, and strands the prompt in a pane not yet taking input. Waiting for the composer also spends that edge, since signals are edge-triggered. - `spawn_workers` takes `name[:mode]`, so a mixed fleet stays one concurrent call. Case names still have to be unique -- the mode never disambiguates two workers that would share a directory. - `sendwait` asks for `wait:"stop,exit"` instead of the `wait:true` default set. That set also carries `idle`, which for an external CLI is inferred from output stabilization: on a dsh worker whose TUI repaints rarely, the re-wait resolved in 0 ms with `signal:"idle"` on a turn with three minutes left to run. It also makes a wrong mode loud -- the modes that cannot deliver `stop` answer 400 before writing anything, instead of resolving on a flap. - The self-heal resend carries `delivered:true` forward. The resend is a tagged duplicate, so the server truthfully reports `delivered:false` about a write it skipped, and §1's cleanup then read a completed turn as an undelivered one and kept a finished worker forever. - dsh workers spawn with the permission posture the Run button sends, because the harness default still asks and a worker parked on an approval row cannot finish a fan-out. The multi-user clamp still applies. Docs: a worked dsh flow in recipes.md, readiness and the signal rules in verbs.md, and the corrections this makes necessary -- `stop`/`blocked` are no longer claude-only, and `last-response` is no longer permanently empty for deepseek. The integration guide gains a section on reading a session back and driving one as a worker; its web-UI section was also stale (that server moved out of a shell session). The static guard that keeps those lists from naming some external CLIs but not others is extended rather than exempted: it now knows the three real classes inside that family (no transcript, no hook signals, and the positive twin -- the modes whose answers can be read), with the hook class derived from `hooksAvailableForMode()` so the predicate and the prose cannot drift apart. Any other partial list still fails, and a new backend belongs to none of the classes until someone says so.
… gate, poll memo, honest pairing docs Four review findings on the worker-transcript feature: - Docker and remote-SSH dsh sessions now keep the pane segmenter: their transcripts live in the container's / remote host's own ~/.dsh, which the local reader can never see, so the transcript path returned 'nothing said yet' forever and an agent polling such a worker starved on an answer that existed. Gated on !session.docker && !session.remote (statically pinned) and documented in the integration guide. - last-response reads are memoized on (path, mtime, size, blocks): the skill's last_text polls once per second, and each poll decompressed and reparsed the whole file on the event loop even when nothing had been appended. An unchanged poll now costs one stat. - The pairing ladder's comment claimed /new is served by step 2; in truth the boot-window transcript wins for as long as it exists (deliberately: preferring newest-eligible would hand a worker its busier sibling's reply). The comment now states the real tradeoff instead of the aspirational one. Same for decodeZstdFrames' 'skipped' wording — a corrupt frame truncates the decode there, which is the safe behavior. - stripReasoningPrefix no longer runs on user prompt text, so a prompt containing a literal </think> renders whole in blocks view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…/deepseek-agent-workers # Conflicts: # CLAUDE.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Codeman ships an agent skill (
skills/codeman) that lets an agent running inside a Codeman session drive the server over HTTP: spawn worker sessions, hand them tasks, block until a worker's turn ends, read its answer, clean up. Until now that only worked properly forclaudeworkers. Every other CLI backend has neither a real end-of-turn signal (Codeman infers "idle" from the pane going quiet, which flaps mid-turn) nor an answer Codeman can read, so the skill drives them the hard way: make the worker print a unique marker and poll the terminal for it.DeepSeek Harness (
dsh, the ninth run mode added in #337) is the one exception on both counts, and this PR turns that into a usable capability. A dsh worker is now spawned, tasked, waited on and read with exactly the same four verbs as a claude worker:Why dsh can do this and the other backends cannot
Two halves have to be true, and dsh is the only non-Claude CLI where both are:
idle/working/blockedto a supervising process over a generic env-gated contract, and Codeman is that supervisor (deepseek-status-shim.ts, from feat(deepseek): add DeepSeek Harness (dsh) as a ninth CLI run mode #337). Sowait?until=stopon a dsh session is a definitive signal, not a guess.Server:
last-responsereads the dsh transcriptDeepSeek was falling through to the terminal-buffer segmenter used for the other external CLIs. For this mode that is not merely coarse, it is wrong: dsh-TUI paints a full-screen splash, so
last-responseon a fresh dsh session returned its ASCII-art logo, which anything polling for a worker's first reply reads as a reply.src/deepseek-transcript.tsreads$DSH_HOME/sessions/<mangled-cwd>/<id>/session.jsonl.zstdinstead. Four things in that file shaped it, all found against real transcripts on disk:zlibzstd decoder (one-shot and streaming alike) stops at the first frame end. A real 56-line transcript decoded as 1 line / 158 bytes: the session header alone, a silent truncation that reads forever as "the worker never answered". The module walks zstd frame and block headers itself to find exact boundaries. Splitting on the 4-byte magic instead would corrupt everything after a magic sequence that happens to occur inside compressed data. zstd is resolved at runtime, because it landed in Node 22.15 while this project's floor is 22.0 and@types/nodestill does not declare it: an older Node keeps the previous pane behaviour rather than throwing.user/messageis the user. Each turn also records a plugin-sourced runtime-context snapshot (sandbox policy, approval policy, cwd). Rendering those shows the agent its own boilerplate back as the user's words.turn/endcarries the provider's error, surfaced asTurn error: …(and a non-error early stop such asmax-tokensasTurn ended: …) instead of an empty string that reads as "still thinking" through fifteen polls.assistant/messagewins; the streamed deltas fill in only for a step that never finalized, so a partial answer is readable mid-turn and never doubled. "Finalized" is tracked as a set of steps rather than as non-empty text: a step whose whole reply was reasoning strips to''at the</think>boundary, and without that distinction the raw unstripped deltas were resurrected in its place, putting the model's monologue in front of the caller.Session-to-transcript pairing is by the transcript's own header
cwdplus a boot window against the Codeman session'screatedAt— never by reproducing dsh's directory mangling (which already has two forms on disk), and never by newest-mtime alone. Mtime alone handed a freshly spawned worker its predecessor's answer from the same case directory, which is worse than saying nothing, because an agent cannot tell a stale answer from a fresh one.Session.deepSeekHomeOverrideis a deliberately narrow getter:envOverridescan hold provider credentials, and the reader needs the profile tree's location and nothing else.Skill: dsh workers, driven like claude workers
Preamble 1.20.0 (SKILL.md's §0 heredoc is regenerated from
preamble.sh, byte-identical as the existing test requires):spawn_workergrows a deepseek branch that gates on the harness composer. Readiness is not the stop signal: the harness reportsidleat boot ~300 ms before its composer paints (measured 2.26 s vs 2.56 s after spawn, twice), so a send-and-wait fired straight afterquick-startresolves on that boot edge, reports a turn that never ran, and strands the prompt in a pane that was not yet taking input. Waiting for the composer also spends that edge, since signals are edge-triggered with no history.spawn_workersacceptsname[:mode], so a mixed fleet is one concurrent call. Case names still have to be unique — the mode never disambiguates two workers that would share a directory.sendwaitasks forwait:"stop,exit"rather than thewait:truedefault set. That set also carriesidle, which for an external CLI is inferred from output stabilization: on a dsh worker whose TUI repaints rarely, the re-wait resolved in 0 ms withsignal:"idle"on a turn that had another three minutes to run. It also makes a wrong mode loud — the modes that cannot deliverstopanswer 400 before writing anything, instead of resolving on a flap.delivered:trueforward. That resend is a tagged duplicate, so the server truthfully reportsdelivered:falseabout a write it skipped — but about the wrong send, and the fast path then read a completed turn as an undelivered one and kept a finished worker forever.Docs: a worked dsh flow in
recipes.md, readiness and signal rules inverbs.md, and the corrections this makes necessary —stop/blockedare no longer claude-only, andlast-responseis no longer permanently empty for deepseek. The integration guide gains a section on reading a session back and driving one as a worker; its web-UI section was also stale (that server moved out of a shell session in an earlier commit).Measured against a live server
Verified end to end against a real dsh install (
dsh 0.1.1-rc.2,@deepseek-harness-tui/dsh-tui 0.8.8) on an isolated Codeman instance:alphaclaude +beta:deepseek) spawned and both ready in 6 s; both tasked concurrently, both resolved on a realstop(dsh 555 ms, claude 3894 ms), both answers read, 12 s total.stop, each answer read back distinctly (the second read correctly refuses to return the first answer), full prompt/response/tool blocks via?context=full, then a clean delete — 5 s total.stopafter 172 s, answer read from the transcript. Against a provider that rejected the request, the reader surfacedTurn error: 400: … does not support toolsinstead of an empty string.zstd -dcon all 13 real transcripts on the box, including one with 2470 frames.Tests
test/deepseek-transcript.test.ts(21 tests) covers the frame walker (multi-frame, single-frame, uncompressed passthrough, torn tail, non-zstd), the parser (plugin-context filtering, reasoning strip, tool blocks, streamed-vs-finalized steps, last-turn semantics, turn errors) and the pairing rules (predecessor excluded, boot window vs a busier sibling, foreign workspace,/newmid-session, missing home).test/agent-skill-mode-lists.test.ts— the static guard that stops the skill's prose from listing some external CLIs but not others — is extended rather than exempted: it now knows the three real classes inside that family (no transcript, no hook signals, and the positive twin: modes whose answers can be read), with the hook class derived fromhooksAvailableForMode()so predicate and prose cannot drift. Any other partial list still fails, and a new backend belongs to none of the classes until someone says so.Full suite green (
npm test).Note on the base
This is stacked on #337 (the DeepSeek Harness run mode), which it needs and which is not merged yet, so the diff here is only the new work. If #337 lands first, GitHub retargets this to master.