feat: add a questions node for asking a human a set of questions

[Jason Robert (jrob5756)]

Closes#376.

human_gate handles one decision well. It has no answer to "an agent has N questions and wants a human to work through them" — and three workflows in microsoft/conductor-workflows independently invented the same workaround.

Workflow	Strategy	Cost
sdd/plan.yaml	hand-rolled ask_question → answer_log loop	~45 lines of nested Jinja
sdd/design.yaml	the same loop, copy-pasted	~45 more
document/review.yaml	gave up on iteration — one prompt_for blob	loses per-question attribution

That loop has two structural problems, not just missing features. It cannot support going back — a workflow step cannot be un-executed, and a concatenated transcript string has no addressable per-question answer to overwrite — and it costs 2 iterations per question against limits.max_iterations.

Three commits

1. feat(gates): opt-in multi-line text input — stands alone

prompt_for was single-line on both surfaces (Prompt.ask, <input type="text">), so a multi-paragraph answer was silently truncated at the first newline. Adds GateOption.multiline, defaulting to False so every existing gate is byte-identical.

A sentinel reader (. on its own line, or EOF) rather than a prompt-toolkit dependency: _handle_gate_with_web cancels the losing CLI task when the web arm wins, and an abandoned asyncio.to_thread never actually dies — a cancelled prompt-toolkit session would strand the terminal in raw mode, where Rich only leaks a blocked thread. Non-TTY stdin falls back to the single-line path unconditionally, keeping the EOFError behaviour _handle_gate_with_web is built around.

2. refactor(gates): extract GatePrompt — behaviour-preserving

_handle_gate_with_web already took an AgentDef and never consulted the workflow graph — it read only name, prompt, options. The seam existed; it was just expressed in terms of user config. Introduces GatePrompt / GateChoice / GateResponse and moves the environment policy (the #286EOFError fix, the unconditionally-started web task, the non-TTY arm, the bg-without-dashboard raise, the #245 killable parked gate) into a graph-agnostic _resolve_human_prompt.

GateChoice deliberately has no route. Routing is a graph concern, and its absence is what stops questions inventing a sentinel route for every synthesized choice.

Landed separately so a gate regression bisects to the refactor rather than to the feature.

3. feat(engine): add a questions node

- name: ask_questionstype: questionssource: architect.output.open_questions # or inline `questions:`routes:
- to: finalizewhen: "{{ ask_questions.output.answered_any }}"

N prompts inside one engine step. Answers are a keyed dict, which is what makes revisiting question 3 overwrite answers.q3 rather than append.

Source entries may be plain strings or objects, so an agent already emitting array of string migrates untouched, and adding choices later is a backward-compatible upgrade that turns "answer this" into "pick one, or write your own".

Details worth reviewing

• --skip-gates never selects a suggested answer. Those come from the upstream agent, so auto-selecting options[0] would feed invented input back as though a human had provided it — worse than the current loop, which skips honestly. Questions with a default take it; the rest are skipped.
• Stale responses would have resolved the wrong question. Every prompt shares the node's name, so a click meant for Q3 landing after Q4 opened would resolve Q4 with Q3's answer. Each presentation carries a prompt_id token, matching the pattern already proven for iteration-limit gates. A response without one is still accepted, so conductor gate respond keeps working.
• A closing review offers Finish / Back. Without it, answering the last question ends the node instantly, making Back unusable exactly where a user is most likely to want it. Skipped when allow_back: false, where it would be pure friction.
• Partial answers survive a checkpoint for free. Answers are committed to the context after each response, and checkpoints already serialize WorkflowContext.to_dict() — so no checkpoint schema change. Resume continues at the first unanswered question; answers to questions that no longer exist are dropped rather than resurrected.
• Nav flags are tri-state (bool | None) so the schema can reject them on other step types. A plain bool default makes "set to False" and "not set" indistinguishable. (I first tried model_fields_set — the existing round-trip tests correctly caught that model_dump() emits defaults.)
• _workflow_has_human_gate now covers questions, or a questions-only workflow launched with --web-bg would park silently with no notice explaining why.

Verification

• 4925 passing, 37 skipped (-m "not performance"); make check and make test-frontend clean; make validate-examples passes.
• Commit 2's acceptance criterion held: the only test changes are three engine tests that patched gate_handler.handle_gate as a stand-in for the CLI arm, repointed to .prompt. Two of them are assert_not_called()--web-bg blocks workflows with human_gate even though the dashboard fully supports gate resolution — extend the #198 policy to human gates #286 regression guards that would otherwise have passed vacuously — verified by sabotaging the web-only shortcut and confirming the guard fails.
• tests/test_performance.py::TestForEachPerformance::test_batching_scalability fails on this branch and identically on unmodified origin/main — pre-existing, unrelated.

Follow-up (separate repo)

Migrate sdd/plan.yaml and sdd/design.yaml — deletes ~90 lines of duplicated Jinja and both answer_log nodes — and consider promoting document/review.yaml from the single-blob gate to real per-question answers.

---

Review round

Seven specialist agents reviewed this PR. They found two silent-correctness bugs I'd missed, several unenforced invariants, and a thread leak the node amplifies — all fixed in c63bd04.

Correctness

• Loop-back replayed the previous pass's answers. Ids default to positional q1..qN, so a second pass over a different question set silently inherited the first pass's answers; with allow_back: false it presented zero prompts and reported completed. Restore is now consumed once and set only by resume().
• Restore matched on id alone, so editing a question's text or order misfiled the old answer onto a different question. Text must now match too, and every drop is logged.
• source: text was Jinja-rendered — Should this use {{ user.id }}? is an ordinary question for a developer tool, and it aborted the step with a TemplateError exactly when the human was about to be asked. Source text is now verbatim (matching for_each); inline questions keep full rendering.
• required was bypassable in one click via skip-all, which runs before the per-question check.
• conductor gate respond --input silently discarded the answer (sent as a bare string, coerced to {}) while printing success, and never echoed prompt_id, so an answer could land on whichever question opened between its status GET and its POST.
• Back at question 1 popped an answer without moving; mid-node commits added N entries to execution_history.

Validation — abort_route, source format, questions in for-each groups, and unanswerable questions were all accepted and then failed at runtime in front of a human.

Thread leak — cancelling the losing CLI arm doesn't stop its asyncio.to_thread worker, which keeps a slot in the shared default executor. One leaks per question, so the executor drains, every unrelated to_thread in the process blocks forever, and loop.shutdown_default_executor() hangs at exit. Verified: 12 abandoned reads previously hung asyncio.run(); they are now daemon threads, the default executor stays usable, and the interpreter exits cleanly.

Tests — two tests passed vacuously (proven by mutating production code and watching them still pass) and were rewritten. Added coverage for the prompt_id staleness guard, the frontend store handlers, and the schema/validator branches — all previously untested. The frontend gap had already hidden a real bug: Back emits a second questions_answered, so the monotonic counter reached "3 of 2" and rendered a 150%-wide progress bar.

Final: 4967 passed, 37 skipped (-m "not performance"); 79 frontend tests; make check and make validate-examples clean.