Skip to content

feat(config): expose supported runner options - #151

Open
BenSheridanEdwards wants to merge 2 commits into
mainfrom
feat/supported-runner-config
Open

BenSheridanEdwards wants to merge 2 commits into
mainfrom
feat/supported-runner-config

Conversation

@BenSheridanEdwards

@BenSheridanEdwards BenSheridanEdwards commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Summary

  • expose runner-native mochaOpts, spec-file retry settings, and reporter configuration through nativeproof.config.ts
  • add --grepNATIVEPROOF_GREP forwarding so one invocation can filter title/tag matches across the selected spec files
  • compose the public afterTest hook after NativeProof's best-effort failure capture, instead of allowing consumer configuration to replace capture
  • use WebdriverIO's supported reporter-entry and Test/TestResult hook types, including reporter classes and retry/error metadata
  • preserve explicit empty --grep '' through CLI, environment, and generated-config precedence so it clears inherited/config filters
  • document the supported single-config path; no new NativeProof runner DSL

Fixes #148

Proof

  • npm run check — pass at 5ab8af8c6d0d04fb907cc527d3903d4004fdde37
  • npm test — 232 passed, 0 failed at 5ab8af8c6d0d04fb907cc527d3903d4004fdde37
  • npm run build — pass at 5ab8af8c6d0d04fb907cc527d3903d4004fdde37
  • Generated project/device proof, or N/A: a fresh packed consumer installed the exact npm pack tarball, compiled a public nativeproof.config.ts with a custom reporter class, all three spec retry fields, and an afterTest callback reading test.file, test.fullName, result.retries.attempts/limit, duration, status, and error?.message; packaged nativeproof --help also passed. No device was required because this changes config translation and CLI/env forwarding, not a device interaction.

Measured baseline → fixed comparison

Source baseline: dfa57e72047c155a97e512121ed8608368108597
Fixed head: 5ab8af8c6d0d04fb907cc527d3903d4004fdde37

Same proof Baseline Fixed
Regression tests for runner retries/reporters/Mocha config, CLI grep, and composed hooks 205 pass, 5 fail 232 pass, 0 fail (full suite)
Fresh packed consumer typecheck failed with five TS2353 errors: mochaOpts, specFileRetries, reporters, and afterTest were not public NativeProofConfig fields (the full consumer config failed too) build/load passed from the fresh packed tarball
Across-file tagged selection file globs existed, but --grep was rejected and no supported config/env grep reached Mocha --spec 'tests/{login,settings}.spec.ts' --grep '@smoke' keeps the selected file set and overrides only mochaOpts.grep
Failure capture with a consumer hook no public hook could be configured without replacing/mutating private generated config regression verifies capture → consumer, and a capture exception remains best-effort while the consumer hook still runs

Review-finding regression proof

Review baseline: ecb7f3cd752f605aa629f8c32aedadb543ef3372
Repaired head: 5ab8af8c6d0d04fb907cc527d3903d4004fdde37

Same proof Review baseline Repaired
Public reporter class + typed hook consumer tsc --noEmit failed with TS2322, TS18046, and TS2339 for the reporter class and normal test/result fields exact packed consumer compiles the reporter class, retry fields, duration/status, and error message
Explicit empty grep precedence three focused regressions failed: CLI retained @inherited, env selection omitted grep, generated Mocha config retained @regression CLI → env → generated config preserves defined ''; all three regressions pass

Impact controls:

  • pre-edit gitnexus impact classified RunnerConfig, RunnerEnv, CliArgs, and runnerEnvFromProcess as HIGH because they fan into the public config, CLI, runTests, and runInspect; edits are additive optional fields and covered at each path
  • repair pre-edit impact was HIGH for RunnerAfterTestHook and runnerEnvFromProcess, LOW for buildWdioConfig and runnerEnv, and UNKNOWN for the type-only reporter alias; direct dependants are the public config/hook, generated runner, tests, runTests, and runInspect
  • final gitnexus detect-changes --scope all: 8 tracked files, 16 symbols, 7 affected flows, HIGH; affected flows are the expected buildWdioConfig, runTests, and runInspect paths (package metadata and the new type-only test are outside its symbol count)

Reviewer view

  • Smallest source-verified view for a non-trivial PR, or Not applicable with the reason for a trivial PR:
nativeproof.config.ts ──defineConfig──> buildWdioConfig
  mochaOpts/retries/reporters              │
  afterTest                                ├─ failure: captureState (best-effort)
                                           └─ consumer afterTest

nativeproof --grep ──> NATIVEPROOF_GREP ──> runnerEnvFromProcess ──> mochaOpts.grep
  • Source files, symbols, states, or commands that verify the view:
    • src/config.ts: RunnerConfig, composeAfterTest, buildWdioConfig
    • src/cli.ts: parseArgs, runnerEnv, helpText
    • src/runner-env.ts: runnerEnvFromProcess
    • test/config.test.ts, test/cli.test.ts, test/runner-env.test.ts, test/public-runner-types.test.ts
    • npm run check && npm test && npm run build

Risk

  • Public interfaces feed the generated runner, so impact is HIGH; defaults remain unchanged when fields are omitted, CLI grep overwrites only grep, and regression tests pin reporter/timeout compatibility and evidence-hook ordering.
  • Optional reporter implementations remain consumer-owned. @wdio/types is now a direct dependency because the exported declarations reference its public contracts; this PR does not bundle Allure/JUnit or add a report UI.
  • A consumer afterTest exception follows runner-native hook behavior and can fail the hook; only NativeProof evidence capture is deliberately best-effort.

Notes

  • Full device coverage not run: no device behavior changed, and no screenshot/device proof was fabricated. The generated config path, CLI/env path, hook ordering, package type surface, and fresh packed consumer were exercised without device dependencies.
  • No merge or npm release performed.

PR Proof Law

  • I followed .agents/DEFINITION_OF_DONE.md and .agents/skills/pr-inline-screenshot-proof/SKILL.md.
  • Screenshots are committed and embedded inline with ![alt](...png?raw=1), or the proof section says Not applicable with the technical reason.
  • The PR body has no bare screenshot links, local paths, relative paths, or proof placeholders.

Screenshot proof: Not applicable — this is a config/type/CLI forwarding change with no rendered UI surface. The source-verified reviewer view and executable before/after proof above are the appropriate evidence.

BenSheridanEdwards added a commit that referenced this pull request Sep 15, 2026
# Conflicts:
#	src/config.ts
#	test/config.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(config): expose supported runner retries, grep, reporters, and hooks

1 participant