fix: resolve lint failure and defer human-only ledger tasks (#1454 follow-up)

[stranske]

Source: Issue #1454

Automated Status Summary

Scope

PR #1447 addressed issue #1437, but verification failed because the actual workflow files were not updated to use pinned LLM dependency installs and proper pip caching, and the enforcement tests were effectively skipped in normal CI. This follow-up closes the remaining gaps by updating the live GitHub Actions workflows, enabling non-skipped enforcement tests, and hardening check_prompt_injection() to avoid runtime errors on malformed detector outputs.

Context for Agent

Related Issues/PRs

• #1447
• #1437

Context for Agent

Related Issues/PRs

• stranske/Workflows#1447
• stranske/Workflows#1437
• #1453
• #1447
• #1437

Tasks

Test Updates (Agent-Executable)

• Remove the unconditional skip logic from tests/workflows/test_workflow_llm_installs.py so tests run in normal CI
• Add assertion to tests/workflows/test_workflow_llm_installs.py to verify agents-auto-pilot workflow contains the pinned install step and lacks unpinned langchain commands
• Add assertions to tests/workflows/test_workflow_llm_installs.py to verify reusable-agents-verifier workflow contains pinned install steps in both evaluate and compare modes
• Add assertions to tests/workflows/test_workflow_llm_installs.py to verify both workflows include actions/cache@v4 with correct cache keys containing Python version and requirements hash
• Add unit tests to validate check_prompt_injection() handles malformed detector outputs without raising exceptions
• Add unit tests to validate check_prompt_injection() correctly handles known-good reason code formats

Code Hardening (Agent-Executable)

• Update check_prompt_injection() to validate detector output format before parsing (check prefix/delimiters and guard integer conversion)
• Implement safe fallback path in check_prompt_injection() for unexpected formats (no exception, defined behavior)

Workflow Update Snippets (Agent-Executable)

• Create docs/workflow-updates/agents-auto-pilot-changes.yml with complete YAML snippet for pip install step and cache configuration
• Create docs/workflow-updates/reusable-agents-verifier-changes.yml with complete YAML snippets for evaluate/compare mode pip install steps and cache configuration
• Add docs/workflow-updates/README.md with instructions for applying the YAML snippets to protected workflow files

Acceptance criteria

Workflow Files (Human-Applied)

• .github/workflows/agents-auto-pilot.yml contains a step that executes exactly pip install -r tools/requirements-llm.txt (as a run command in the workflow)
• .github/workflows/agents-auto-pilot.yml does NOT contain any pip install commands that install langchain via an unpinned specifier, including (but not limited to) pip install langchain, pip install langchain*, pip install langchain== (missing version), or pip install git+...langchain...
• .github/workflows/reusable-agents-verifier.yml includes a step that executes exactly pip install -r .workflows-lib/tools/requirements-llm.txt in the evaluate mode execution path
• .github/workflows/reusable-agents-verifier.yml includes a step that executes exactly pip install -r .workflows-lib/tools/requirements-llm.txt in the compare mode execution path
• .github/workflows/reusable-agents-verifier.yml does NOT contain any pip install commands that install langchain via an unpinned specifier in the evaluate or compare execution paths (e.g., pip install langchain*, pip install langchain, pip install langchain-community without a version pin)
• .github/workflows/agents-auto-pilot.yml contains an actions/cache@v4 step that caches pip (cache path includes pip cache directory) and uses a cache key that includes BOTH the Python version and ${{ hashFiles('tools/requirements-llm.txt') }}
• .github/workflows/reusable-agents-verifier.yml contains an actions/cache@v4 step that caches pip (cache path includes pip cache directory) and uses a cache key that includes BOTH the Python version and ${{ hashFiles('.workflows-lib/tools/requirements-llm.txt') }}

Test Enforcement

• tests/workflows/test_workflow_llm_installs.py is executed under normal CI (i.e., it is not unconditionally skipped) and will fail if the agents-auto-pilot workflow lacks the pinned requirements install step
• tests/workflows/test_workflow_llm_installs.py asserts that .github/workflows/agents-auto-pilot.yml contains pip install -r tools/requirements-llm.txt AND asserts it does not contain any floating pip install langchain*/unversioned langchain install commands
• tests/workflows/test_workflow_llm_installs.py asserts that .github/workflows/reusable-agents-verifier.yml contains pip install -r .workflows-lib/tools/requirements-llm.txt for BOTH evaluate and compare modes (separate assertions per mode/path, not a single generic check)
• tests/workflows/test_workflow_llm_installs.py asserts that BOTH workflows include an actions/cache@v4 step and that each cache key includes Python version + the correct hashFiles(...) call for the correct requirements path

Code Hardening

• check_prompt_injection() does not raise an exception when detect_prompt_injection returns an unexpected or malformed reason code (e.g., empty string, missing delimiter/prefix, non-integer suffix); instead it follows a defined fallback path (e.g., returns a safe default or logs and continues)
• check_prompt_injection() continues to correctly handle valid reason code formats by producing the same outcome as before for at least one known-good reason code value
• Unit tests pass that verify check_prompt_injection() behavior with both malformed and known-good detector outputs

Documentation Deliverables

• docs/workflow-updates/agents-auto-pilot-changes.yml exists and contains valid YAML snippet for pip install and cache steps

• docs/workflow-updates/reusable-agents-verifier-changes.yml exists and contains valid YAML snippets for evaluate/compare mode pip install and cache steps

• docs/workflow-updates/README.md exists and provides clear instructions for applying the YAML snippets to the protected workflow files

• Head SHA: 8d75b3e

• Latest Runs: ❔ in progress — Agents PR meta manager

• Required: gate: ⏸️ not started

• | Workflow / Job | Result | Logs |

• |----------------|--------|------|

• | Agents PR meta manager | ❔ in progress | View run |

Head SHA: 0cfc951
Latest Runs: ✅ success — Gate
Required: gate: ✅ success

Workflow / Job	Result	Logs
Agents PR meta manager	❔ in progress	View run
CI Autofix Loop	✅ success	View run
Gate	✅ success	View run
Health 40 Sweep	✅ success	View run
Health 44 Gate Branch Protection	✅ success	View run
Health 45 Agents Guard	✅ success	View run
Health 50 Security Scan	✅ success	View run
Maint 52 Validate Workflows	✅ success	View run
PR 11 - Minimal invariant CI	✅ success	View run
Selftest CI	✅ success	View run

[agents-workflows-bot]

Automated Status Summary

Head SHA: 2ed70e9
Latest Runs: ⏳ pending — Gate
Required contexts: Gate / gate, Health 45 Agents Guard / guard
Required: core tests (3.11): ⏳ pending, core tests (3.12): ⏳ pending, docker smoke: ⏳ pending, gate: ⏳ pending

Workflow / Job	Result	Logs
(no jobs reported)	⏳ pending	—

Coverage Overview

• Coverage history entries: 1

Coverage Trend

Metric	Value
Current	93.12%
Baseline	85.00%
Delta	+8.12%
Minimum	70.00%
Status	✅ Pass

Top Coverage Hotspots (lowest coverage)

File	Coverage	Missing
src/cli_parser.py	81.8%	4
src/percentile_calculator.py	95.0%	1
src/aggregator.py	95.0%	2
src/__init__.py	100.0%	0
src/ndjson_parser.py	100.0%	0

Updated automatically; will refresh on subsequent CI/Docker completions.

---

Keepalive checklist

Scope

PR #1447 addressed issue #1437, but verification failed because the actual workflow files were not updated to use pinned LLM dependency installs and proper pip caching, and the enforcement tests were effectively skipped in normal CI. This follow-up closes the remaining gaps by updating the live GitHub Actions workflows, enabling non-skipped enforcement tests, and hardening check_prompt_injection() to avoid runtime errors on malformed detector outputs.

Context for Agent

Related Issues/PRs

• #1447
• #1437

Context for Agent

Related Issues/PRs

• stranske/Workflows#1447
• stranske/Workflows#1437
• #1453
• #1447
• #1437

Tasks

Test Updates (Agent-Executable)

• Remove the unconditional skip logic from tests/workflows/test_workflow_llm_installs.py so tests run in normal CI
• Add assertion to tests/workflows/test_workflow_llm_installs.py to verify agents-auto-pilot workflow contains the pinned install step and lacks unpinned langchain commands
• Add assertions to tests/workflows/test_workflow_llm_installs.py to verify reusable-agents-verifier workflow contains pinned install steps in both evaluate and compare modes
• Add assertions to tests/workflows/test_workflow_llm_installs.py to verify both workflows include actions/cache@v4 with correct cache keys containing Python version and requirements hash
• Add unit tests to validate check_prompt_injection() handles malformed detector outputs without raising exceptions
• Add unit tests to validate check_prompt_injection() correctly handles known-good reason code formats

Code Hardening (Agent-Executable)

• Update check_prompt_injection() to validate detector output format before parsing (check prefix/delimiters and guard integer conversion)
• Implement safe fallback path in check_prompt_injection() for unexpected formats (no exception, defined behavior)

Workflow Update Snippets (Agent-Executable)

• Create docs/workflow-updates/agents-auto-pilot-changes.yml with complete YAML snippet for pip install step and cache configuration
• Create docs/workflow-updates/reusable-agents-verifier-changes.yml with complete YAML snippets for evaluate/compare mode pip install steps and cache configuration
• Add docs/workflow-updates/README.md with instructions for applying the YAML snippets to protected workflow files

Acceptance criteria

Workflow Files (Human-Applied)

• .github/workflows/agents-auto-pilot.yml contains a step that executes exactly pip install -r tools/requirements-llm.txt (as a run command in the workflow)
• .github/workflows/agents-auto-pilot.yml does NOT contain any pip install commands that install langchain via an unpinned specifier, including (but not limited to) pip install langchain, pip install langchain*, pip install langchain== (missing version), or pip install git+...langchain...
• .github/workflows/reusable-agents-verifier.yml includes a step that executes exactly pip install -r .workflows-lib/tools/requirements-llm.txt in the evaluate mode execution path
• .github/workflows/reusable-agents-verifier.yml includes a step that executes exactly pip install -r .workflows-lib/tools/requirements-llm.txt in the compare mode execution path
• .github/workflows/reusable-agents-verifier.yml does NOT contain any pip install commands that install langchain via an unpinned specifier in the evaluate or compare execution paths (e.g., pip install langchain*, pip install langchain, pip install langchain-community without a version pin)
• .github/workflows/agents-auto-pilot.yml contains an actions/cache@v4 step that caches pip (cache path includes pip cache directory) and uses a cache key that includes BOTH the Python version and ${{ hashFiles('tools/requirements-llm.txt') }}
• .github/workflows/reusable-agents-verifier.yml contains an actions/cache@v4 step that caches pip (cache path includes pip cache directory) and uses a cache key that includes BOTH the Python version and ${{ hashFiles('.workflows-lib/tools/requirements-llm.txt') }}

Test Enforcement

• tests/workflows/test_workflow_llm_installs.py is executed under normal CI (i.e., it is not unconditionally skipped) and will fail if the agents-auto-pilot workflow lacks the pinned requirements install step
• tests/workflows/test_workflow_llm_installs.py asserts that .github/workflows/agents-auto-pilot.yml contains pip install -r tools/requirements-llm.txt AND asserts it does not contain any floating pip install langchain*/unversioned langchain install commands
• tests/workflows/test_workflow_llm_installs.py asserts that .github/workflows/reusable-agents-verifier.yml contains pip install -r .workflows-lib/tools/requirements-llm.txt for BOTH evaluate and compare modes (separate assertions per mode/path, not a single generic check)
• tests/workflows/test_workflow_llm_installs.py asserts that BOTH workflows include an actions/cache@v4 step and that each cache key includes Python version + the correct hashFiles(...) call for the correct requirements path

Code Hardening

• check_prompt_injection() does not raise an exception when detect_prompt_injection returns an unexpected or malformed reason code (e.g., empty string, missing delimiter/prefix, non-integer suffix); instead it follows a defined fallback path (e.g., returns a safe default or logs and continues)
• check_prompt_injection() continues to correctly handle valid reason code formats by producing the same outcome as before for at least one known-good reason code value
• Unit tests pass that verify check_prompt_injection() behavior with both malformed and known-good detector outputs

Documentation Deliverables

• docs/workflow-updates/agents-auto-pilot-changes.yml exists and contains valid YAML snippet for pip install and cache steps

• docs/workflow-updates/reusable-agents-verifier-changes.yml exists and contains valid YAML snippets for evaluate/compare mode pip install and cache steps

• docs/workflow-updates/README.md exists and provides clear instructions for applying the YAML snippets to the protected workflow files

• [ ]

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c1cafa00b6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[agents-workflows-bot]

🤖 Keepalive Loop Status

PR #1455 | Agent: Codex | Iteration 0/5

Current State

Metric	Value
Iteration progress	[----------] 0/5
Action	wait (missing-agent-label)
Disposition	skipped (transient)
Gate	success
Tasks	21/34 complete
Timeout	45 min (default)
Timeout usage	3m elapsed (8%, 42m remaining)
Keepalive	❌ disabled
Autofix	❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | resource |
| Suggested recovery | Confirm the referenced resource exists (repo, PR, branch, workflow, or file). |

[Copilot]

Pull request overview

Follow-up to PR #1454 to restore CI signal and prevent the Codex worker loop from repeatedly attempting human-only work.

Changes:

• Fixes a Python lint (E501) by splitting a long acceptance-criteria string via implicit string literal concatenation.
• Updates the issue ledger to mark protected-workflow edits as deferred (human-only) to avoid the worker repeatedly picking and failing those tasks.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
scripts/langchain/followup_issue_generator.py	Splits a long bullet string to satisfy line-length linting without changing output text.
.agents/issue-1453-ledger.yml	Defers human-only tasks involving protected workflow files to prevent worker stalls.