[Keepalive] Add Codex log analyzer for automatic progress detection

[stranske]

Why

The keepalive automation relies on Codex updating PR body checkboxes to track progress. However, Codex frequently completes work without updating checkboxes, causing the keepalive to see no progress and either stall or repeat instructions. This creates a disconnect between actual work done and detected progress.

Scope

Implement a tool that analyzes PR file changes and/or Codex session logs to infer task completion, then automatically suggests or applies checkbox updates.

Non-Goals

• Replacing the existing checkbox-based tracking entirely
• Modifying how Codex itself operates
• Changes to the PR body format

Tasks

• Port prototype from stranske/Trend_Model_Project/tools/codex_log_analyzer.py
• Add integration point in keepalive_loop.js after Codex round completes
• Implement PR body checkbox auto-update based on analysis
• Add tests for the analyzer integration
• Document the new progress detection flow

Acceptance Criteria

• Analyzer can identify task completion from PR file changes
• Checkbox updates are suggested/applied when evidence found
• No false positives (low confidence results are flagged, not auto-applied)
• Integration does not break existing keepalive flow
• Works without external API dependencies (LLM optional enhancement)

Implementation Notes

Prototype implementation exists at:

• stranske/Trend_Model_Project/tools/codex_log_analyzer.py

Key functions to port:

• analyze_pr_changes() - analyzes PR file diff for task evidence
• find_evidence_for_task() - matches task text against file changes
• format_analysis_report() - produces structured output

Integration point in keepalive:

// After Codex round, before next instruction
const analysis = await analyzeCodexProgress(prNumber, fileChanges, tasks);
if (analysis.checkbox_updates_needed.length > 0) {
  await updatePRCheckboxes(prNumber, analysis.checkbox_updates_needed);
}

Related: Addresses keepalive stalling issue observed in Trend_Model_Project PRs #4111, #4112, #4113, #4114.

[stranske]

Progress Update: LangChain Integration Foundation

Branch feature/langchain-analysis has been created with the core infrastructure:

Implemented

• ✅ tools/llm_provider.py - Provider abstraction with fallback chain:
  • GitHub Models API (primary, uses existing GITHUB_TOKEN)
  • OpenAI API (fallback)
  • Regex patterns (last resort, no API)
• ✅ tools/codex_jsonl_parser.py - Parser for Codex --json event streams
• ✅ tools/codex_session_analyzer.py - Task completion detection combining parsing + LLM
• ✅ 38 tests covering all new modules
• ✅ langchain optional dependency added to pyproject.toml

Documentation

• 📄 Integration Plan - Detailed implementation plan with:
  • Data source options (summary vs JSONL vs filtered)
  • Community tools research (codex-session-view, etc.)
  • JSONL schema reference
  • Testing phases

Key Discovery: --json Mode

The Codex CLI has a --json flag that streams detailed JSONL events including:

• agent_message - What Codex says it accomplished
• reasoning - Model thinking summaries
• todo_list - Direct task mapping (if emitted)
• command_execution - Shell commands with exit codes
• file_change - Files modified

This is significantly richer than the current --output-last-message summary.

Next Steps

1. Modify workflow to capture --json output
2. Capture sample session to verify event structure
3. Add workflow integration step
4. Test in Portable Alpha repo

Related issues: #455, #456, #457 (can use same LLM infrastructure)

[github-actions]

Branch codex/issue-454 created from main.

Option 1 (Invite) is enforced on issue events. PR creation is disabled by design so you are the PR author. Codex only engages on human-authored PRs.

Please open the PR as the author so Codex can work on it:

• Compare link: https://github.com/stranske/Workflows/compare/main...codex/issue-454?expand=1
• Suggested title: Codex bootstrap for #454
• Suggested body (copy/paste):

<!-- meta:issue:454 -->

@codex Your objective is to satisfy the **Acceptance Criteria** by completing each **Task** within the defined **Scope**.

**This round you MUST:**
1. Implement actual code or test changes that advance at least one incomplete task toward acceptance.
2. Commit meaningful source code (.py, .yml, .js, etc.)—not just status/docs updates.
3. **UPDATE THE CHECKBOXES** in the Tasks and Acceptance Criteria sections below to mark completed items.
4. Change `- [ ]` to `- [x]` for items you have completed and verified.
5. **In your final summary**, list completed tasks using the format: `✅ Completed: [exact task text]`

**CRITICAL - Checkbox Updates:**
When you complete a task or acceptance criterion, update its checkbox directly in this prompt file. Change the `[ ]` to `[x]` for completed items. The automation will read these checkboxes and update the PR's status summary.

**CRITICAL - Summary Format:**
At the end of your work, include explicit completion markers for each task you finished:

✅ Completed: Add validation for user input
✅ Completed: Write unit tests for validator module

This helps the automation accurately track which tasks were addressed in this round.

**Example:**
Before: `- [ ] Add validation for user input`
After:  `- [x] Add validation for user input`

**DO NOT:**
- Commit only status files, markdown summaries, or documentation when tasks require code.
- Mark checkboxes complete without actually implementing and verifying the work.
- Close the round without source-code changes when acceptance criteria require them.
- Change the text of checkboxes—only change `[ ]` to `[x]`.

**COVERAGE TASKS - SPECIAL RULES:**
If a task mentions "coverage" or a percentage target (e.g., "≥95%", "to 95%"), you MUST:
1. After adding tests, run TARGETED coverage verification to avoid timeouts:
   - For a specific script like `scripts/foo.py`, run:
     `pytest tests/scripts/test_foo.py --cov=scripts/foo --cov-report=term-missing -m "not slow"`
   - If no matching test file exists, run:
     `pytest tests/ --cov=scripts/foo --cov-report=term-missing -m "not slow" -x`
2. Find the specific script in the coverage output table
3. Verify the `Cover` column shows the target percentage or higher
4. Only mark the task complete if the actual coverage meets the target
5. If coverage is below target, add more tests until it meets the target

IMPORTANT: Always use `-m "not slow"` to skip slow integration tests that may timeout.
IMPORTANT: Use targeted `--cov=scripts/specific_module` instead of `--cov=scripts` for faster feedback.

A coverage task is NOT complete just because you added tests. It is complete ONLY when the coverage command output confirms the target is met.

**CONTEXT TIP:**
If the PR body includes a **Source** section with links to a parent issue or original PR, those contain additional context about the problem being solved. Check the linked issue/PR for background information, related discussions, or details not captured in the Scope section.

Review the Scope/Tasks/Acceptance below, identify the next incomplete task that requires code, implement it, then **update the checkboxes** to mark completed items.

[stranske]

Codex belt dispatcher queued this issue and created branch codex/issue-454.

[stranske]

Codex belt dispatcher queued this issue and created branch codex/issue-454.