feat: interactive STS dashboard + local-AI fixer tier (verify→revise)

[PhysShell]

Dashboard + AI fixer tier

Two threads built on the branch since #5.

1. Interactive STS dashboard (viz/)

viz/build_dashboard.py builds a self-contained, offline HTML dashboard from the validated sts_audit/ artifacts (72,569 findings + the module pain table): a where-it-hurts treemap, by-category/tool/top-rules bars, by-source coloured by whether it ships a code fix (the 85% story), and OWN-fixer shape coverage. Plotly is inlined (no CDN needed). Three live-switchable themes — Midnight / Terminal / Aurora — re-skin both the CSS and the Plotly palette on click. Verified headless (Chromium): 6 charts render in all themes, zero console errors. Generated HTML/PNG + vendored Plotly are git-ignored (reproducible from the README).

2. AI fixer tier (fix/fixarm/ai_fix.py)

A pluggable Applier that asks a local LLM to propose patches for the residual mechanical fixers can't touch (T3 detect-only, T4-refused suggest-only). The model is not trusted — riding the same wrapper, its proposal is verified by the no-new-findings re-audit, shown as a reviewable diff, gated to REVIEW, and rolled back on regression. The LLM proposes; the audit + human judge (so it doesn't break "never AI-judged").

• Local-only by design (STS is proprietary): OpenAI-compatible client, default Ollama (qwen2.5-coder); works with llama.cpp / LM Studio / vLLM. MockLlmClient drives the same path in CI.
• Verify→revise loop, no framework: with a reaudit, each proposal is re-audited per round; on failure the reason is fed back and the model revises up to --max-rounds. Every round still passes through the audit — the loop only helps a weak model converge, never relaxes the trust model. A framework (LangChain/LangGraph) buys nothing here; the thin client is zero-lock-in.
• cli: --applier ai --llm-url … --model … --max-rounds N (REVIEW gate forced).

Test plan

PYTHONPATH=fix python3 fix/tests/test_ai_fix.py # 7/7 (prompt/parse, verify+gate, no-change, regression, loop converge, give-up)
PYTHONPATH=fix python3 fix/tests/test_own_fix.py # 25/25
PYTHONPATH=fix python3 fix/tests/test_orchestrate.py # 7/7
python3 viz/build_dashboard.py # -> viz/sts-dashboard.html

🤖 Generated with Claude Code

---

Generated by Claude Code

Summary by CodeRabbit

• New Features
  • Added an AI-assisted fix mode (--applier ai) using local LLM proposals with review-only gating and a bounded verify→revise loop (--max-rounds, --llm-url, --model).
  • Added an offline-capable audit dashboard generator that produces a standalone interactive HTML report.
• Documentation
  • Documented the AI tier workflow and updated guidance for dashboard building/inputs.
• Bug Fixes
  • Updated ignore rules to prevent generated visualization artifacts from being committed.
• Tests
  • Added end-to-end AI fixing tests, including fenced replacement parsing, re-audit rollback, convergence, and give-up behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: dc36639d-89b2-43ae-9d72-32b67cce7292

📥 Commits

Reviewing files that changed from the base of the PR and between 110706c and 9a5ae0d.

📒 Files selected for processing (2)

• fix/fixarm/ai_fix.py
• fix/tests/test_ai_fix.py

🚧 Files skipped from review as they are similar to previous changes (1)

• fix/fixarm/ai_fix.py

---

📝 Walkthrough

Walkthrough

This PR adds a local LLM AI fixing path for audit findings, wires it into the fix CLI with review-only gating and tests, and adds a standalone Plotly dashboard generator for STS audit outputs with docs and ignore rules.

Changes

AI fix workflow

Layer / File(s)	Summary
Prompt, parser, and local clients fix/fixarm/ai_fix.py	Defines the AI fixer prompt, fenced replacement parser, and local/mock OpenAI-compatible clients.
Planning and revise loop fix/fixarm/ai_fix.py	Groups findings by file, proposes replacements, verifies candidates through re-audit, and restores or applies planned edits.
CLI wiring and documentation fix/fixarm/cli.py, docs/fix-arm.md, fix/README.md	Adds the AI applier CLI options, review-tier override, and documentation for the local-LLM workflow.
AI fixer tests fix/tests/test_ai_fix.py	Covers prompt construction, parsing, review gating, skip and rollback cases, revise-loop retries, and the standalone test runner.

STS dashboard build

Layer / File(s)	Summary
Data collection and metrics viz/build_dashboard.py	Loads STS audit artifacts, aggregates counters, parses health-report data, and computes summary metrics.
HTML template and chart wiring viz/build_dashboard.py	Defines the embedded dashboard HTML, Plotly chart rendering, theme controls, and data injection points.
Build entrypoint and generated artifacts viz/build_dashboard.py, viz/README.md, .gitignore	Loads Plotly locally or from the CDN, writes the dashboard output, and documents or ignores generated visualization files.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• PhysShell/OwnAudit#1: Extends the same fix/applier and re-audit flow by adding the AI tier integrated here.
• PhysShell/OwnAudit#2: Updates the same CLI applier plumbing and review-gating path used for fix execution.

Poem

I thumped the keys and built a bright new trail,
With fenced-up fixes and a dashboard sail.
The audit charts went twinkly in the night,
While local LLM hops kept the patches tight.
🐇

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 11.43% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes both main additions: the interactive STS dashboard and the local AI fixer tier with verify→revise behavior.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/sts-runtime-analysis-2mo4z9

---

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:c12a3174f5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Restore AI candidates when re-audit raises

When the verify→revise loop is used and the re-audit fails (for example ScriptReaudit exits nonzero or its output is missing), this call happens during dry_run after the candidate has already been written to the real worktree. Because the restore at the end of planning is not in a finally block and run_fix only rolls back after apply, the command can abort with unverified LLM edits left in the checkout, breaking the safety contract that planning is side-effect-free. Wrap the candidate write/re-audit in cleanup that restores the original/current contents before propagating the error.

Useful? React with 👍 / 👎.

[PhysShell]

@codex Fixed in 28a28a7. Wrapped the per-file planning body in try/finally so the original content is always restored before any error propagates — including when reaudit raises mid-loop (ScriptReaudit nonzero / missing output). Planning stays side-effect-free, so dry_run can never leave an unverified LLM edit in the worktree. Test: test_ai_plan_restores_tree_when_reaudit_raises (reaudit throws → dry_run re-raises, tree byte-for-byte unchanged).

---

Generated by Claude Code

[chatgpt-codex-connector]

To use Codex here, create an environment for this repo.

[coderabbitai]

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@fix/fixarm/ai_fix.py`:
- Around line 143-157: The retry feedback in ai_fix.py is being carried across
findings through self._feedback, so one finding’s rejection message can leak
into the next finding’s first prompt. Make the feedback state local to each
finding by resetting or scoping it inside the per-finding repair flow in the
revise/propose path (using the relevant methods around _propose and the retry
loop), so each finding starts with no prior rejection context unless it was
generated for that same finding.
- Around line 148-155: The _revise() flow leaves the candidate written to disk
if self.reaudit(workdir) raises, which breaks the side-effect-free behavior of
_plan()/dry_run(). Update the _revise loop in ai_fix.py so the temporary
candidate write is always restored in a finally block around the re-audit call,
ensuring the original contents are put back whether re-audit succeeds, returns,
or throws; use the existing _revise, self.reaudit, diff_findings, and run_fix
flow as the anchor points.
- Around line 68-80: Restrict LocalLlmClient.base_url in the __init__ and
complete flow so only loopback/local endpoints are accepted; currently --llm-url
can point to a remote host and leak source code. Add validation in
LocalLlmClient to reject non-HTTP(S) schemes and any hostname other than
localhost, 127.0.0.1, or ::1 before constructing the urllib.request.Request in
complete.
In `@fix/fixarm/cli.py`:
- Around line 72-81: Thread args.line_tol through the AI verification path so
AiFixApplier uses the same tolerance as run_fix. Update the AI revise/verify
loop to pass the line tolerance into the introduced and still-present checks
instead of relying on exact-line matching and the hard-coded <= 2 threshold, so
proposals that only shift diagnostics are judged consistently.
In `@viz/build_dashboard.py`:
- Around line 78-79: The dashboard summary in build_dashboard is still using
hardcoded snapshot values for clusters, totals, fixable percentage, and related
UI text instead of deriving them from the loaded sts_audit artifacts. Update the
summary/card/footer/caption logic in build_dashboard and the helper sections
that feed those fields so they compute from the current findings/by_source data
rather than frozen literals, keeping the UI consistent with the generated
charts.
- Around line 258-259: The payload inserted into the HTML template from
build_dashboard.py is not safe for a script context because json.dumps(data) can
leave </script> intact and break out of the block. Update the HTML generation
path around HTML.replace and the %DATA% substitution to encode/escape the JSON
for embedding inside the <script> tag before it is inserted, so values in
module, rule, or category cannot terminate the script early.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7f266450-2f81-4f77-a4c2-f0a4f8eea028

📥 Commits

Reviewing files that changed from the base of the PR and between 79ff930 and c12a317.

📒 Files selected for processing (8)

• .gitignore
• docs/fix-arm.md
• fix/README.md
• fix/fixarm/ai_fix.py
• fix/fixarm/cli.py
• fix/tests/test_ai_fix.py
• viz/README.md
• viz/build_dashboard.py

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

fix/fixarm/ai_fix.py (1)

186-194: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Process per-file findings from bottom to top to keep line windows stable.

Line 194 updates cur after each accepted rewrite, but Lines 187-188 keep computing later windows from original audit line numbers. If an earlier rewrite inserts/removes lines, the next finding can be prompted/spliced against the wrong region. Iterate in descending line order so edits lower in the file cannot shift pending windows above them.

Proposed fix

- for f in fs:+ for f in sorted(fs, key=lambda f: f.line, reverse=True):

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@fix/fixarm/ai_fix.py` around lines 186 - 194, Process the findings in
descending line order inside the loop in ai_fix.py so each later rewrite does
not shift the line windows for earlier pending items; update the iteration over
fs in the revision flow around _revise, cur, and occupied to process
bottom-to-top and keep each prompt/splice aligned with the intended region.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@viz/build_dashboard.py`:
- Line 273: The footer text is using data['clusters']['total'] from the parsed
findings total, so the “raw findings clustered to N clusters” message can show
the wrong count. Update the footer generation in build_dashboard.py where the
dashboard template replacement happens to use the parsed candidate/high cluster
count instead of the findings count, and make sure the symbol handling the
cluster summary data feeds the footer with the correct cluster total.
---
Outside diff comments:
In `@fix/fixarm/ai_fix.py`:
- Around line 186-194: Process the findings in descending line order inside the
loop in ai_fix.py so each later rewrite does not shift the line windows for
earlier pending items; update the iteration over fs in the revision flow around
_revise, cur, and occupied to process bottom-to-top and keep each prompt/splice
aligned with the intended region.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b56b37fb-5af1-428d-8cc9-ff71ecc81247

📥 Commits

Reviewing files that changed from the base of the PR and between 28a28a7 and 110706c.

📒 Files selected for processing (3)

• fix/fixarm/ai_fix.py
• fix/fixarm/cli.py
• viz/build_dashboard.py

🚧 Files skipped from review as they are similar to previous changes (1)

• fix/fixarm/cli.py

[PhysShell]

@coderabbitai re: the outside-diff finding on fix/fixarm/ai_fix.py:186-194 (process per-file findings bottom-to-top) — fixed in 9a5ae0d. Good catch: each window is computed from the finding's original audit f.line, so an accepted rewrite that changed a file's line count was shifting the windows of findings still pending above it. Now iterating sorted(fs, key=lambda f: f.line, reverse=True) — edits lower in the file leave upper line numbers intact, so every window stays aligned. Locked with a new test test_ai_plans_findings_bottom_to_top (asserts the lower-in-file finding is proposed first). 9/9 ai_fix tests green.

Note I did not apply the other finding from this run (footer cluster count, line 273) — replied inline explaining why it's a false positive: the headline's "15803 findings" is already the clustered count (3602 + 12201), so clusters.total is the correct footer value.

---

Generated by Claude Code

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

@PhysShell Thanks — that addresses the fix/fixarm/ai_fix.py window-shift issue cleanly. Processing each file’s findings bottom-to-top is the right fix when splice windows are anchored to the original audit line numbers, and the new test_ai_plans_findings_bottom_to_top coverage closes the loop nicely.

Also agreed on the dashboard note: if the health-report.md headline total is already the clustered total, then using clusters.total in the footer is correct, so that earlier concern doesn’t apply.

(ᵔᴥᵔ)ﾉ

---

✏️ Learnings added

Learnt from: PhysShell
URL: https://github.com/PhysShell/OwnAudit/pull/6
Timestamp: 2026-06-25T23:10:33.637Z
Learning: In `fix/fixarm/ai_fix.py`, per-file AI fix planning must process findings in descending `f.line` order because each replacement window is computed from the original audit line number; applying a lower-file rewrite first avoids shifting the line positions of findings that are still pending above it. This behavior is covered by the test `fix/tests/test_ai_fix.py::test_ai_plans_findings_bottom_to_top`.

---

Learnt from: PhysShell
URL: https://github.com/PhysShell/OwnAudit/pull/6
Timestamp: 2026-06-25T23:10:33.637Z
Learning: In `viz/build_dashboard.py`, the findings headline parsed from `sts_audit/health-report.md` already represents the clustered total findings count, so the dashboard footer should use `clusters.total` rather than recomputing or substituting a different total. The user clarified the example total `15803` equals `3602 + 12201`.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.