feat(workflow-executor): add AI-assisted and Full AI execution to the guidance step (PRD-148 pattern, PRD-18)

[Scra3]

What

Adds the 3-way Execution mode (Manual / AI-assisted / Full AI) to the Guidance step. Part of PRD-18 — 3-PR set (executor + `ForestAdmin/forestadmin-server` orchestrator + `ForestAdmin/forestadmin` editor).

• Manual — human types + submits (unchanged); AI never called.
• AI-assisted (automated-with-confirmation) — AI pre-fills the free-text response from the step prompt + workflow context; persisted as pendingData.userInput with an aiGenerated flag (drives the front "AI" badge); the human edits/submits.
• Full AI (fully-automated) — AI writes and submits automatically (executionResult.userInput + generatedByAi); the response becomes a variable reusable by downstream steps.

Degrade / resilience (product-confirmed): on AI failure/timeout, or an empty draft, the step degrades to Manual (empty field, pendingData: {}) — the run never fails, the step never auto-skips. Full AI always submits on a valid draft (no low-confidence degrade — a text generator, unlike Load Related Record). The AI is never re-run on re-dispatch of the same (runId, stepIndex).

withAiAssist + logAiDegrade are hoisted from LoadRelatedRecordStepExecutor into BaseStepExecutor (2nd consumer); load-related behavior is unchanged.

Tests

Full suite green (1412). Added: Manual never calls AI; submit path never calls AI; AI-assisted pre-fill + badge flag + re-dispatch no-regen + degrade (AI failure / empty draft) + no-retry-after-degrade; Full AI submit + replay + degrade; schema (3-way, no .catch, rejects unknown); mapper (3 values + default manual). Load-related suite unchanged (hoist is behavior-neutral).

Relates to PRD-18.

🤖 Generated with Claude Code

Note

Add AI-assisted and fully automated execution modes to guidance steps

• GuidanceStepExecutor now supports three modes: Manual (unchanged), AutomatedWithConfirmation (AI drafts a response for user review), and FullyAutomated (AI completes the step directly with generatedByAi=true).
• AI prompts include trigger record context (non-relationship fields, capped at 40) fetched via buildTriggerRecordMessages, improving model grounding.
• On AI failure or empty/non-string draft, the executor degrades to manual by persisting empty pendingData, preventing repeated AI attempts on re-dispatch.
• GuidanceStepExecutionData gains aiGenerated and generatedByAi flags; the guidance summary formatter now attributes AI-generated responses to the AI instead of the operator.
• BaseStepExecutor gains shared withAiAssist and logAiDegrade helpers, removing duplicated logic from LoadRelatedRecordStepExecutor.
• Behavioral Change: GuidanceStepDefinitionSchema now rejects invalid executionType values instead of coercing them; defaults to manual when the field is absent.

^{Macroscope summarized 719bfd3.}

[linear-code]

PRD-148

PRD-18

[qltysh]

2 new issues

Tool	Category	Rule	Count
qlty	Structure	Function with many returns (count = 6): mapTask	2

[qltysh]

Coverage Impact

⬆️ Merging this pull request will increase total coverage on main by 0.04%.

Modified Files with Diff Coverage (3)

Rating	File	% Diff	Uncovered Line #s
	packages/workflow-executor/src/executors/base-step-executor.ts	100.0%
	...ow-executor/src/executors/summary/step-execution-formatters.ts	100.0%
	...ages/workflow-executor/src/executors/guidance-step-executor.ts	100.0%
	Total	100.0%

🚦 See full report on Qlty Cloud »

🛟 Help

• Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.

• Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.

• File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)

  • Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

[hercemer42]

Manual test finding — AI-assisted / Full AI guidance draft is not grounded in the trigger record

Found while manually testing the guidance AI modes end-to-end (editor → orchestrator → executor) on the executor dev project.

Symptom. A first-step free-text Guidance set to AI-assisted or Full AI, instruction "Write a one-sentence internal note summarizing this account, using the account holder's first and last name," run on account 6a01a17e4fca9c347ccff875 (Godfried Kunstmann), drafts from the operator's identity instead of the record:

"This account belongs to Brian Fox and is managed by the Operations team."

Brian Fox / Operations are the logged-in operator + team, not the account holder. In Full AI this is auto-submitted with no human review, so the wrong-subject text ships silently.

Root cause. For guidance, the AI receives only buildContextMessage() (packages/workflow-executor/src/executors/base-step-executor.ts:233) + buildPreviousStepsMessages(). On a first step there are no previous steps, so the entire context sent to the model is:

Step executed by: Brian Fox (brian.fox@forestadmin.com, id: 3)
Role: | Team: Operations
Current date and time: 2026-07-09T…Z (UTC)

No trigger record — no id, no fields. Yet GUIDANCE_RESPONSE_SYSTEM_PROMPT (guidance-step-executor.ts) tells the model it has "the workflow context (trigger record, previous steps)." The prompt promises a trigger record the executor never supplies, so the only name in scope is the operator's.

Confirmed via the AI guidance-response: context debug log (executor at LOG_LEVEL=Debug) — workflowContext was the single string above.

Why only guidance. Record executors (update-record, trigger-action, load-related-record) append their own record data afterbuildContextMessage(). Guidance doesn't, so a standalone guidance step has nothing about the record. Adding a Get Data step before it works around it (its summary lands in previous-steps context).

Repro (short). Account collection → new workflow → first step Guidance, Input = Free text, Execution = AI-assisted or Full AI, instruction referencing the account holder's name → End → run on any account. Draft uses the operator name, never the account holder.

Options.

1. Include the trigger record in the guidance context so drafts are grounded (most useful; executor may hold only a record reference → may need a fetch).
2. Align the system prompt to what's actually provided (drop the "trigger record" promise; author adds a Get Data step to reference fields).

Everything else in the feature checked out end-to-end: editor Execution dropdown (default AI-assisted, all 3 modes, Full AI hides the button), executionMode persists through save+reload, AI-assisted draft/badge/edit-clears-badge/submit/advance, Full AI auto-complete, and idempotent re-dispatch (no second AI call).

[hercemer42]

Follow-up: the context gap is a PRD-148 degrade-safety violation, not just a missing field

The context gap reported above (guidance AI never receives the trigger record) breaks a core, product-signed-off rule of the PRD-148 pattern this PR is built on:

Full AI, when it can't confidently resolve, degrades to human confirmation — it never acts autonomously on weak input, and never silently skips.

PRD-148 makes this explicit for Load Related Record:

• "Full AI: if more than one candidate is viable and the AI cannot justify a single best choice … it degrades (pre-selecting its best guess for a human to confirm) instead of auto-loading an arbitrary pick."
• "No source record … Full AI does NOT silently auto-advance; it surfaces the same error + 'Continue without a X' and waits for an explicit action."

Guidance has no equivalent guard. With no record in context, Full AI has nothing to ground "the account holder's first and last name" on — yet it auto-commits an ungrounded draft ("…Brian Fox…", the operator) and advances. That's exactly the autonomous-action-on-weak-input that PRD-148 forbids for LRR.

Suggested guidance AC (mirroring PRD-148):

• Full AI must degrade to AI-assisted (human review) rather than autonomously submit when it can't ground the response in the available context — instead of emitting a low-confidence / wrong-subject draft.
• Pair this with the context-gap fix (include the trigger record) so the common case doesn't degrade unnecessarily.

LRR's "can't ground" triggers are crisp (empty candidate list / no source record). Guidance is free-text so "can't ground" is fuzzier to detect, but the operator-name substitution shows the risk is real and currently unguarded.

[hercemer42]

Test-coverage note: the grounding bug is invisible to the unit suite

The guidance executor suite (packages/workflow-executor/test/executors/guidance-step-executor.test.ts) is genuinely strong on the state machine — all three modes, both degrade triggers (AI failure and empty/whitespace draft) for AI-assisted and Full AI, idempotent re-dispatch, and submit-path-never-calls-AI. Those paths are low-risk and I corroborated the failure/degrade ones manually.

The gap is the single test that inspects what's actually sent to the model — "sends the step prompt and workflow context to the AI" (L230). It asserts only the prompt:

constallContent=messages.map(m=>m.content).join('\n');expect(allContent).toContain('Summarize the customer situation');

It runs on a mocked context and never asserts the trigger record (id / field values) is in allContent. So the suite stays green while the AI receives no record to ground on — which is precisely why the "draft uses the operator's name, not the account holder" bug reached manual testing instead of being caught here.

Suggested regression guard (add with the context fix): assert the trigger record reaches the AI, e.g.

expect(allContent).toContain(triggerRecord.id);// or a known field value

Today that assertion fails — which is the point: it pins the fix and prevents silent regression of context completeness. Without it, a green suite gives false confidence about response grounding.

[Scra3]

Thanks for the thorough end-to-end catch — this was a real gap. Addressed in 84e577052:

1. Context gap (grounding) — fixed.askAiForResponse now builds a buildTriggerRecordMessages() block: it resolves the base collection schema, fetches the trigger record's non-relation fields via agent.getRecord (capped at 40 fields / 200 chars each), and injects a Trigger record — the <Collection> this workflow is running on: system message before the previous-steps context. So a first-step guidance is now grounded on the record (the account holder), not just the operator identity. The fetch is non-fatal: a schema/record failure logs a warning and the AI proceeds without it, rather than failing the step. The system prompt's "trigger record" promise is now actually honored.

3. Test gap — fixed. Added grounds the AI in the trigger record field values: it wires a schema + getRecord returning Godfried Kunstmann, asserts only non-relation fields are fetched, and asserts the field values reach the model messages. This is exactly the regression guard you suggested — it fails without the fix.

2. Degrade-safety — intentional product divergence, not an oversight. Full AI guidance auto-submitting with no low-confidence degrade is a signed-off product decision (Brice, on PRD-18): unlike Load Related Record (a record pick with hard downstream consequences and a crisp "can't ground" signal — empty candidate list / no source), guidance is free-text generation where "can't ground" isn't reliably detectable. The decision was: Full AI guidance always submits; the safety net is the default being AI-assisted (human review) + the AI-failure→Manual degrade, not a confidence gate. The reported failure was really the missing record (now fixed) + the prompt already instructing "use only facts in context, never invent names". I'd rather not add a heuristic confidence-degrade that contradicts the signed-off call — happy to take it back to product if you think the free-text grounding risk warrants revisiting that decision.

Everything else you validated end-to-end (editor modes, persistence, AI-assisted draft/badge/edit, Full AI auto-complete, idempotent re-dispatch) — thanks for the detailed pass.

[hercemer42]

Approving — review threads resolved. Grounding fix verified end-to-end (trigger record now in the AI context; draft names the account holder, not the operator), the regression guard test passes, and the degrade-safety point is product-approved (Brice). Editor modes / persistence / AI-assisted / Full AI / idempotent re-dispatch all validated in a live run.