feat(evidence): expose failure artifact records - #153
Open
BenSheridanEdwards wants to merge 4 commits into
Open
BenSheridanEdwards wants to merge 4 commits into
BenSheridanEdwards wants to merge 4 commits into
Conversation
# Conflicts: # src/config.ts # test/config.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
onFailureEvidenceconfig callback withproject,file,fullName, retryattempt, and exact successfully produced PNG/XML pathscaptureState(prefix): Promise<string>runtime compatibility: a source-read failure warns, degrades to"", then writes the screenshot/XML pair best-effort as beforecaptureStatePathsstrict: a source-read failure rejects before screenshot/XML writes, so passing tests emit no callback and failed or incomplete captures emit no fictitious complete record or replacement failureafterTestbehaviorFixes #152
Depends on #151. This PR now contains repaired parent head
5ab8af8c6d0d04fb907cc527d3903d4004fdde37via merge commite244b8b5f0338cc977989a762bee36793925fab6; merge #151 first.Proof
npm run check— passnpm test— 237 passed, 0 failednpm run build— pass/tmp/nativeproof-failure-evidence-consumerinstalled the exactnpm packtarball, typechecked a publicnativeproof.config.tsusingonFailureEvidence, loaded and invoked the callback, and ran packagednativeproof --help; output includedpacked failure-evidence config proof: PASSandpacked nativeproof --help: PASS. No device was required because this changes config translation, artifact identity, and hook association rather than device interaction.EventEmitterreporter class, retry fields, and typedafterTesttest/result fields includingerror; packaged CLI help passed.1b60928c45068c9dd46f5db239aa5d4c236917ade2edc16361a7ba688a2f3f53, typechecked publiccaptureState(prefix): Promise<string>, simulated rejectedgetPageSource()through the packed runtime, and proved: public capture resolved""plus PNG/empty XML and one warning; strict failure capture emitted zero structured records, skipped screenshot, preserved the original failed result by identity, and invoked consumerafterTest; packagednativeproof --helppassed.Measured baseline → fixed comparison
Source baseline: PR #151 exact head
ecb7f3cd752f605aa629f8c32aedadb543ef3372Original feature head:
4f9fac8cf47b973297aba8ec7cabff449393ab14Restacked fixed head:
e244b8b5f0338cc977989a762bee36793925fab6Acceptance fixed head:
f79f0c15588d3ce878fa874198691760b2906592Compatibility fixed head:
73c691b909153de7c81c271edb14412de64e9294failure-Login-rejects_bad_password-d3b3f935(uniqueNames: 1)capturePathsVisibleToConsumer: false)test/config.test.ts; 235/235 full suiteafterTestcase-64394/case-84739)failure-Login-rejects_bad_password-6333da18; exact regression fails...14898824…0718b2/...47c770c8…f903da), remain deterministic, and are 99 charscaptureStatewith rejectedgetPageSource()f79f0c1…rejects; new compatibility regression fails 23/2473c691b…warns, resolves"", and writes PNG/empty XML; focused regression passes 25/25 (five consecutive runs)getPageSource()rejectioncaptureStatewarned/degraded to empty XML; no structured callback existedcaptureStatepreserves that established fallback; internalcaptureStatePathsrejects before any file is reported, so the structured callback is skipped while the original failed result and consumerafterTestare preservedImpact controls:
composeAfterTest,failureEvidenceName,captureState, andbuildWdioConfig; direct dependants were the config tests and runner-config translationgitnexus detect-changes --scope all: 6 files, 15 symbols, 2 affected config execution flows, MEDIUM; only the expected config/evidence/type/docs/tests surfaces changedgitnexus detect-changes --scope all: 9 files, 33 symbols, 7 affected processes, HIGH; this is the combined feat(config): expose supported runner options #151 runner-config plus feat(evidence): expose collision-safe failure artifact records #152 failure-evidence stack, so full gates and a fresh packed-consumer compile were required and passedfailureEvidenceName,captureStatePaths, and legacycaptureStatewas LOW; final compare detected 14 symbols, 0 processes, LOW. GitNexus-generatedAGENTS.md/CLAUDE.mdcount churn was restored before commit, leaving onlysrc/evidence.tsandtest/config.test.tschanged.captureStateandcaptureStatePathswas LOW. Final correctiondetect-changes --scope unstagedfound 3 files, 5 symbols, 0 processes, LOW; full PR compare to feat(config): expose supported runner options #151 found 7 files, 26 symbols, 2 config flows, MEDIUM. Generated index churn was restored before commit.Reviewer view
Not applicablewith the reason for a trivial PR:src/evidence.ts:captureStatePaths,captureState,failureEvidenceNamesrc/config.ts:FailureEvidence,composeAfterTest,buildWdioConfigtest/config.test.ts: exact known collision, unique identity, source-read rejection, exact-file callback, pass/no-callback, and failed-capture ordering casesnpm run check && npm test && npm run build/tmp/nativeproof-pr153-compat-packed(tarball SHA-256 above) exercises both public fallback and strict callback suppressionRisk
onFailureEvidenceis optional and reporter-neutral; no reporter package, Allure API, or NativeProof test DSL was added.afterTest; successful capture still occurs before both callbacks.captureState(prefix): Promise<string>keeps its established best-effort runtime behavior: source-read failure warns, degrades to"", and continues to screenshot/XML writes. InternalcaptureStatePathsremains strict; runner composition swallows its source failure, emits no structured record, and still invokes the consumer hook with the original result.Frameworks.Test/Frameworks.TestResulthook types.Notes
PR Proof Law
.agents/DEFINITION_OF_DONE.mdand.agents/skills/pr-inline-screenshot-proof/SKILL.md., or the proof section saysNot applicablewith the technical reason.Screenshot proof: Not applicable — this is a config/type/artifact-path association change with no rendered UI. The source-verified call-order view and executable baseline/fixed proofs above are the appropriate evidence.