[Follow-up] Implement unit tests for the FallbackChainProvider (PR #1263)

[stranske]

Why

PR #1263 addressed issue #1253 but verification identified concerns (verdict: CONCERNS).
This follow-up adds targeted tests and cleanup to close those gaps.

Source

Original PR: #1263
Parent issue: #1253

Scope

Add tests to verify quality_context propagation into provider calls.
Add regression tests for confidence caps with short analysis text.
Audit provider implementations for optional quality_context handling and add
legacy-compat integration tests where needed.
Isolate keepalive_loop.js behavior changes into a dedicated commit with tests.
Clean up duplicate/conflicting tasks in .agents/issue-1253-ledger.yml.

Non-Goals

Redesign keepalive scoring logic beyond the targeted regression tests.
Broader refactors of provider implementations unrelated to quality_context.

Tasks

• Add unit tests for FallbackChainProvider that pass quality_context and
• assert it reaches the target provider.
• Add regression tests for analysis_text_length < 50 with has_work_evidence
• enabled so confidence does not exceed the cap.
• Audit provider implementations for optional quality_context handling and
• add integration tests that call providers without quality_context to
• confirm legacy behavior.
• Isolate keepalive_loop.js behavior changes into a separate commit and
• add dedicated tests.
• Clean up .agents/issue-1253-ledger.yml to remove duplicate/conflicting
• task entries (manual).

Acceptance Criteria

• Tests verify quality_context is passed to active providers in
• FallbackChainProvider.
• Regression tests confirm confidence cap behavior for short analysis text
• with has_work_evidence enabled.
• Integration tests confirm providers tolerate missing quality_context.
• keepalive_loop.js changes are isolated with dedicated tests.
• .agents/issue-1253-ledger.yml has no duplicate/conflicting tasks (manual).

Implementation Notes

Focus on targeted tests and keep behavioral changes isolated.

Background (previous attempt context)

The previous attempt did not run any iterations and did not attempt tasks, leaving
quality_context integration unverified. Verification logs noted tests were missing
and unrelated keepalive_loop.js changes increased regression risk.

Original Issue

Focus on targeted tests and keep behavioral changes isolated.

<details>
<summary>Background (previous attempt context)</summary>

The previous attempt did not run any iterations and did not attempt tasks, leaving
quality_context integration unverified. Verification logs noted tests were missing
and unrelated keepalive_loop.js changes increased regression risk.

</details>

Deferred Tasks (Requires Human)

[ ] Clean up the .agents/issue-1253-ledger.yml file by removing duplicate and conflicting task entries. (AGENT_LIMITATIONS: Cannot modify .github/workflows/*.yml (protected). | Flag this task for manual review by a repository maintainer.)

[stranske-keepalive]

🤖 Auto-pilot step $((STEP_COUNT + 1)): Analyzing issue for improvements...

Running optimizer inline (not via label trigger).

[stranske-keepalive]

Issue Optimization Suggestions

Review the suggestions below. If you want the agent to apply them, add the
agents:apply-suggestions label to this issue.

Task splitting

• Audit all provider implementations to ensure they handle the optional quality_context parameter appropriately. Add integration tests if needed that call these providers without the parameter to simulate legacy behavior. (This task combines two distinct activities: auditing provider implementations and adding integration tests. These should be split to ensure clarity and focus.). Suggested split: Define scope for: Audit all provider implementations to ensure they handle the optional quality_context parameter appropriately. (verify: confirm completion in repo), Implement focused slice for: Audit all provider implementations to ensure they handle the optional quality_context parameter appropriately. (verify: confirm completion in repo), Validate focused slice for: Audit all provider implementations to ensure they handle the optional quality_context parameter appropriately. (verify: confirm completion in repo), Define scope for: Add integration tests that call providers without the quality_context parameter to simulate legacy behavior. (verify: tests pass), Implement focused slice for: Add integration tests that call providers without the quality_context parameter to simulate legacy behavior. (verify: tests pass), Validate focused slice for: Add integration tests that call providers without the quality_context parameter to simulate legacy behavior. (verify: tests pass).
• Revert or isolate the behavioral changes in keepalive_loop.js into a separate commit or section with dedicated tests. (This task involves two separate actions: reverting or isolating changes and creating dedicated tests. These should be split to avoid conflating tasks.). Suggested split: Define scope for: Revert or isolate the behavioral changes in keepalive_loop.js into a separate commit or section. (verify: confirm completion in repo), Implement focused slice for: Revert or isolate the behavioral changes in keepalive_loop.js into a separate commit or section. (verify: confirm completion in repo), Validate focused slice for: Revert or isolate the behavioral changes in keepalive_loop.js into a separate commit or section. (verify: confirm completion in repo), Create dedicated tests for the behavioral changes in keepalive_loop.js. (verify: tests pass).

Blocked tasks

• Clean up the .agents/issue-1253-ledger.yml file by removing duplicate and conflicting task entries. (AGENT_LIMITATIONS: Cannot modify .github/workflows/*.yml (protected). | Flag this task for manual review by a repository maintainer.)

Objective acceptance criteria

• The unit tests in FallbackChainProviderTests.js must verify that the quality_context is passed to the active provider by capturing the argument in a mock or stub. (The acceptance criterion is subjective because it specifies the use of a mock or stub, which is a design decision. | Rephrase to: 'The unit tests in FallbackChainProviderTests.js must verify that the quality_context is passed to the active provider and is correctly utilized.')
• The changes in keepalive_loop.js must be isolated in a separate commit with dedicated tests that verify the intended behavior without affecting quality_context propagation. (The requirement to isolate changes in a separate commit is subjective and not directly testable. | Rephrase to: 'The changes in keepalive_loop.js must include dedicated tests that verify the intended behavior without affecting quality_context propagation.')

Missing sections

• Scope
• Non-goals

Formatting issues

• The 'Source' section uses bullets for non-task items. Consider using a numbered list or plain text for clarity.
• The 'Implementation Notes' section uses bullets, but these are not tasks. Consider using a numbered list or plain text for better formatting.

Notes

• The issue is well-structured but could benefit from splitting some tasks for clarity and addressing subjective acceptance criteria. Additionally, the inclusion of a 'Scope' section to define the boundaries of the work and a 'Non-goals' section to clarify what is out of scope would improve the issue's comprehensiveness. Formatting improvements in the 'Source' and 'Implementation Notes' sections would enhance readability.

[agents-workflows-bot]

🤖 Auto-pilot step $((STEP_COUNT + 1)): Applying optimization suggestions...

Running apply inline (not via label trigger).

[agents-workflows-bot]

🤖 Auto-pilot step 1: Issue prepared! Assigning to agent...

Adding agent:codex label. The capability check will run automatically.

⏳ Agent will create a PR shortly.

[stranske-automation-bot]

Codex belt dispatcher queued this issue and created branch codex/issue-1267.

[stranske-keepalive]

🤖 Auto-pilot step $((STEP_COUNT + 1)): Applying optimization suggestions...

Running apply inline (not via label trigger).

[github-actions]

Branch codex/issue-1267 created from main.

Option 1 (Invite) is enforced on issue events. PR creation is disabled by design so you are the PR author. Codex only engages on human-authored PRs.

Please open the PR as the author so Codex can work on it:

• Compare link: https://github.com/stranske/Workflows/compare/main...codex/issue-1267?expand=1
• Suggested title: Codex bootstrap for #1267
• Suggested body (copy/paste):

<!-- meta:issue:1267 -->

@codex Your objective is to satisfy the **Acceptance Criteria** by completing each **Task** within the defined **Scope**.

## Keepalive Next Task

**This round you MUST:**
1. Implement actual code or test changes that advance at least one incomplete task toward acceptance.
2. Commit meaningful source code (.py, .yml, .js, etc.)—not just status/docs updates.
3. Mark a task checkbox complete ONLY after verifying the implementation works.
4. Focus on the FIRST unchecked task unless blocked, then move to the next.

**Guidelines:**
- Keep edits scoped to the current task rather than reshaping the entire PR.
- Use repository instructions, conventions, and tests to validate work.
- Prefer small, reviewable commits; leave clear notes when follow-up is required.
- Do NOT work on unrelated improvements until all PR tasks are complete.

**COVERAGE TASKS - SPECIAL RULES:**
If a task mentions "coverage" or a percentage target (e.g., "≥95%", "to 95%"), you MUST:
1. After adding tests, run TARGETED coverage verification to avoid timeouts:
   - For a specific script like `scripts/foo.py`, run:
     `pytest tests/scripts/test_foo.py --cov=scripts/foo --cov-report=term-missing -m "not slow"`
   - If no matching test file exists, run:
     `pytest tests/ --cov=scripts/foo --cov-report=term-missing -m "not slow" -x`
2. Find the specific script in the coverage output table
3. Verify the `Cover` column shows the target percentage or higher
4. Only mark the task complete if the actual coverage meets the target
5. If coverage is below target, add more tests until it meets the target

IMPORTANT: Always use `-m "not slow"` to skip slow integration tests that may timeout.
IMPORTANT: Use targeted `--cov=scripts/specific_module` instead of `--cov=scripts` for faster feedback.

A coverage task is NOT complete just because you added tests. It is complete ONLY when the coverage command output confirms the target is met.

**The Tasks and Acceptance Criteria are provided in the appendix below.** Work through them in order.

<!-- 
  DO NOT ADD TASK CONTENT BELOW THIS LINE
  
  Task content is dynamically injected from the PR body via the `appendix` input.
  This template file should only contain instructions, not PR-specific tasks.
  
  If you see task content in this file, it was incorrectly committed and should
  be removed. The guard script will flag this as an error.
-->

[stranske-keepalive]

🤖 Auto-pilot step 1: Issue prepared! Assigning to agent...

Adding agent:codex label. The capability check will run automatically.

⏳ Agent will create a PR shortly.

[stranske-automation-bot]

Codex belt dispatcher queued this issue and created branch codex/issue-1267.