Remediation proof packet: emit fix-proof.json for every fix attempt

[PhysShell]

Classification

• Change type: enhancement / remediation evidence contract
• Priority: P1
• Expected value: very high — turns Fix Arm output into one reviewable, machine-checkable proof artifact
• Estimated effort: medium
• Implementation risk: medium — false claims of fixed would corrupt audit truth
• Recommendation: YES, strong product improvement after finding identity v2 is stable
• Depends on: PhysShell/Own.NET#266
• Consumes derived lifecycle/evidence state from: PhysShell/Own.NET#268
• Consumes agent evidence from: PhysShell/007#42 and replay verification from PhysShell/007#43 when available

Problem

The Fix Arm already has the correct safety shape:

select -> dry-run -> apply -> re-audit -> reject regression -> tier gate

It also records a useful coverage ledger and reverts rejected/no-effect patches. However, the evidence that a remediation succeeded is spread across several artifacts:

• original finding(s);
• proposed diff;
• re-audit output;
• introduced/removed finding comparison;
• build/test logs;
• runtime evidence where available;
• optional agent run records;
• human review disposition.

A reviewer currently has to reconstruct the argument manually. Humans can do that, naturally, because apparently archaeology is now part of code review.

Goal

Emit one versioned, deterministic fix-proof.json for every attempted fix, including accepted, rejected, no-effect, unfixable and operational-error outcomes.

The proof packet must answer:

Which exact occurrence was targeted?
What evidence justified the target?
What patch was attempted?
Did the target disappear under comparable analysis inputs?
Did a semantic successor remain?
Were new findings introduced?
Which build/tests/runtime checks ran?
Was the agent execution replay-verified?
What remains unknown?

Proposed contract

Exact names are reviewable; the information must remain explicit.

{
 "schema": 1,
 "proof_id": "...",
 "status": "ACCEPTED | REVIEW | REJECTED | NO_EFFECT | UNFIXABLE | ERROR",
 "target": {
 "pattern_id": "...",
 "occurrence_id": "...",
 "lineage_id": "...",
 "rule": "OWN001",
 "before_evidence_digest": "..."
 },
 "patch": {
 "digest": "...",
 "changed_files": [],
 "changed_symbols": [],
 "diff_artifact": "..."
 },
 "verification": {
 "target_removed": true,
 "semantic_successor_present": false,
 "introduced_occurrences": [],
 "build": {},
 "tests": [],
 "runtime_witness": null,
 "reaudit_comparability": {}
 },
 "agent_run": {
 "run_id": null,
 "action_id": null,
 "replay_report_digest": null
 },
 "provenance": {
 "target_commit": "...",
 "own_version": "...",
 "ownaudit_version": "...",
 "config_digest": "...",
 "input_digest": "..."
 },
 "limitations": []
}

Required semantics

1. target_removed is matched by occurrence/lineage identity, not (rule, basename, nearest line) once #266 is available.
2. Disappearance under a changed analyzer/config/input is not presented as a verified fix.
3. A target disappearing while a semantic successor remains is not accepted as fixed.
4. Any introduced actionable occurrence remains explicit and rejects automatic acceptance.
5. Missing build/test/runtime checks remain unknown/not-run; they are never treated as pass.
6. An agent process exiting zero is not remediation proof.
7. Human review is an explicit disposition layered over machine evidence, not a rewritten machine verdict.
8. Rejected/no-effect attempts still produce proof packets so failed strategies do not vanish from evaluation data.

Integration with the current Fix Arm

run_fix() should return or construct a proof model in addition to the existing FixResult/ledger output.

Suggested output layout:

artifacts/fix-runs/<proof-id>/
 fix-proof.json
 diff.patch
 before-findings.json
 after-findings.json
 verification/
 build.json
 tests.json
 runtime.json
 logs/

The packet should reference large artifacts by relative path + digest rather than embedding unbounded logs.

Required controls

1. Target removed, no new findings, comparable config, tests pass → accepted/review according to tier.
2. Target removed but a new finding appears → rejected.
3. Target line moves but lineage remains → no false remove/add pair.
4. One repeated occurrence is fixed and another remains → only the targeted occurrence changes state.
5. Re-audit uses a different config digest → inconclusive/rejected, not fixed.
6. Target remains → NO_EFFECT with reverted patch.
7. Applier throws after modifying the tree → rollback evidence and ERROR proof retained.
8. Agent claims success but re-audit disagrees → machine verdict wins; claim remains provenance only.
9. Runtime witness is unavailable → explicit limitation, never synthetic pass.
10. Re-rendering the same normalized inputs produces byte-stable proof JSON.

Acceptance

• Versioned fix-proof.json schema is documented and implemented.
• Every fix attempt emits a proof packet, including rejected/no-effect/error paths.
• Identity uses #266 contracts when available and has an explicit legacy fallback.
• Tool/config/input comparability is checked and recorded.
• Large logs/artifacts are digest-addressed rather than embedded.
• Existing Fix Arm safety behavior and rollback semantics remain unchanged.
• The dashboard/report can link to the proof packet without re-deriving remediation truth.
• 007 action/replay evidence is optional but verifiable when attached.

Non-goals

• No new fix engine.
• No AI judge deciding whether a patch is correct.
• No automatic acceptance of T4 fixes.
• No replacement of the Finding Evidence Ledger.
• No requirement that every finding has a runtime witness.
• No storage of secrets or unbounded agent stdout in the proof JSON.

[PhysShell]

Design refinement: proof packets need claim/evidence adequacy, not baseline worship

fix-proof.json should not imply that the pre-fix run is an oracle. It is a reference observation. The proof packet should distinguish:

reference behavior
candidate behavior
required contract
accepted oracle/evidence class

A previous agent or implementation may already be flaky, incomplete, or wrong. Baseline self-comparison is useful for the noise floor; it does not confer correctness.

Evidence-strength vocabulary

Recommended shared ladder:

E0 prose assertion
E1 textual evidence
E2 parsed structural evidence
E3 graph/static-semantic evidence
E4 executable verification
E5 runtime/differential observation
E6 independent replay / attested evidence

Each claimed property in a proof packet should declare the minimum acceptable evidence class and the strongest actually observed class.

Example:

{
 "claim_id": "architecture:no-presentation-to-sql-path",
 "claim_kind": "no_dependency_path",
 "required_evidence_level": "E3",
 "observed_evidence_level": "E1",
 "evidence_refs": ["search:rg-output"],
 "verdict": "INSUFFICIENT"
}

rg is not bad evidence. It is simply insufficient for a transitive graph claim. Conversely, requiring a graph engine for a literal-symbol existence claim is wasted machinery.

Add separate proof axes

Do not collapse these into status or one confidence score:

{
 "axes": {
 "outcome": "PASS | FAIL | UNKNOWN",
 "obligation_coverage": "PASS | FAIL | BLOCKED",
 "evidence_adequacy": "PASS | INSUFFICIENT | CONFLICTING | UNKNOWN",
 "safety": "PASS | FAIL | BLOCKED",
 "robustness": "STABLE | FLAKY | INSUFFICIENT_SAMPLES | UNKNOWN",
 "efficiency": "IMPROVED | DEGRADED | LATERAL | UNKNOWN"
 }
}

The overall packet may then derive:

ACCEPTED
REVIEW
REJECTED
BLOCKED
NO_EFFECT
ERROR

but must retain the axis-level reason.

Claim/action consistency

Agent prose should remain provenance, not proof. Claims should be checked against typed events:

"tests passed"
 -> exact TestSummaryFinished(pass) evidence

"independent replay passed"
 -> replay report digest + verifier identity

"artifact verified"
 -> resolver record + digest equality

"finding fixed"
 -> comparable re-audit + no semantic successor

A zero exit and a confident report are not substitute events.

Reference/candidate sampling

Where agent or runtime behavior is nondeterministic, proof packets should support:

• reference↔reference noise-floor evidence;
• candidate↔candidate stability evidence;
• reference↔candidate semantic-delta evidence;
• sample counts and occurrence rates;
• stable finding identity including scenario, relation/action, field path, and normalized difference class.

A difference that already exists inside reference samples should be marked pre-existing/flaky, not attributed to the candidate. It must remain visible because pre-existing brokenness is still brokenness.

Argument and artifact provenance

For security- or correctness-sensitive values, record origin:

from task contract
from previous tool result
from deterministic derivation
from immutable registry
model-generated
unknown

Examples include commit SHA, case ID, artifact ID, registry digest, source path, command arguments, account/entity IDs, and verifier identity.

A syntactically valid value of unknown/model-generated provenance can be more dangerous than a malformed value that fails immediately.

Suggested schema additions

{
 "claims": [],
 "axes": {},
 "reference_runs": [],
 "candidate_runs": [],
 "noise_floor": {},
 "trajectory_evidence_ref": null,
 "claim_action_consistency": [],
 "value_provenance": []
}

Large trajectory artifacts remain digest-addressed outside the packet.

Acceptance additions

• Baseline/reference observations are not represented as oracle authority.
• Blocking claims declare minimum evidence levels.
• Correct-looking outcomes with insufficient evidence produce BLOCKED or REVIEW, never automatic acceptance.
• Outcome, evidence adequacy, safety, robustness, and efficiency are retained separately.
• Agent claims can be contradicted by canonical action/verifier evidence.
• Pre-existing baseline flakiness is visible but not attributed to the candidate.
• Sensitive arguments/artifacts retain provenance lineage.

Related design owner: PhysShell/007#74. Qodec-specific trajectory application: PhysShell/qodec#15.