Skip to content

Add test coverage for untested GitHub Scripts #153

Description

@stranske

Why

Seven core GitHub Action scripts in .github/scripts/ lack dedicated test coverage:

  1. agents-guard.js - Critical security guardrail for agents surface
  2. agents_pr_meta_orchestrator.js - PR metadata coordination
  3. keepalive_guard_utils.js - Keepalive safety utilities
  4. keepalive_instruction_template.js - Prompt generation for keepalive
  5. keepalive_orchestrator_gate_runner.js - Gate integration for orchestrator
  6. keepalive_post_work.js - Post-keepalive cleanup operations
  7. merge_manager.js - PR merge automation

These scripts handle critical workflow orchestration and security checks. Without tests, regressions can slip through undetected. The recent API retry integration (PR #151) highlighted how untested code paths can harbor subtle bugs.

Scope

Add comprehensive Node.js test files for the 7 untested scripts listed above. Each test file should:

  • Live in .github/scripts/__tests__/ following existing naming conventions
  • Use Node.js built-in test runner (node:test)
  • Mock GitHub API calls and context objects
  • Cover happy path, error handling, and edge cases
  • Achieve >80% line coverage for the target script

Non-Goals

  • Refactoring the scripts themselves (unless blocking testability)
  • Adding Python test coverage (separate effort)
  • Integration testing with live GitHub API
  • Performance benchmarking

Tasks

Round 1: Security-critical scripts

  • Create agents-guard.test.js with tests for label validation, immutable surface checks, and bypass detection
  • Create keepalive-guard-utils.test.js covering pause label detection and guard state management

Round 2: Orchestration scripts

  • Create agents-pr-meta-orchestrator.test.js testing body section updates and conflict resolution
  • Create keepalive-orchestrator-gate-runner.test.js for gate status evaluation and dispatch logic

Round 3: Keepalive utilities

  • Create keepalive-instruction-template.test.js validating prompt generation and variable substitution
  • Create keepalive-post-work.test.js testing cleanup operations and state transitions

Round 4: Merge automation

  • Create merge-manager.test.js covering merge eligibility, conflict detection, and squash behavior
  • Run full test suite and verify all new tests pass
  • Update test documentation if needed

Acceptance criteria

  • All 7 test files exist in .github/scripts/__tests__/
  • Each test file has at least 5 test cases covering core functionality
  • node --test .github/scripts/__tests__/*.test.js passes with 0 failures
  • No regressions in existing tests (Selftest CI workflow passes)
  • New tests follow patterns established in existing test files (e.g., api-helpers.test.js)

Implementation notes

Files to create:

  • .github/scripts/__tests__/agents-guard.test.js
  • .github/scripts/__tests__/keepalive-guard-utils.test.js
  • .github/scripts/__tests__/agents-pr-meta-orchestrator.test.js
  • .github/scripts/__tests__/keepalive-orchestrator-gate-runner.test.js
  • .github/scripts/__tests__/keepalive-instruction-template.test.js
  • .github/scripts/__tests__/keepalive-post-work.test.js
  • .github/scripts/__tests__/merge-manager.test.js

Reference files for test patterns:

  • .github/scripts/__tests__/api-helpers.test.js (mocking patterns)
  • .github/scripts/__tests__/agents-belt-scan.test.js (context mocking)
  • .github/scripts/__tests__/detect-changes.test.js (GitHub API mocking)

Scripts to analyze for test design:

# View exports and public API of each script
grep -E "^(module\.exports|exports\.)" .github/scripts/agents-guard.js
grep -E "^(module\.exports|exports\.)" .github/scripts/keepalive_guard_utils.js
# ... etc

Testing

After each round of implementation:

  1. Run new tests in isolation:

    node --test .github/scripts/__tests__/<new-test>.test.js
  2. Run full test suite:

    node --test .github/scripts/__tests__/*.test.js
  3. Verify CI passes on the PR


Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:codexAgent-created issues from CodextestingSynthesized from ChatGPT import for testing

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions