feat(metadata): per-group review metadata on FlowGroup - #14
Merged
jamesaphoenix merged 1 commit intoSep 2, 2026
Merged
Conversation
Group narrative previously existed only behind a manual Pass 1 call, keyed in a side-car object the consumer had to join against the group. Every group now carries its own review metadata: group_type, risk, impact, complexity, review_focus, description, summary and invariant. A deterministic heuristic floor fills risk, group_type and impact for free on every analysis. An optional batched LLM pass runs over the final groups — after refinement ops are applied, since refinement cannot key metadata to group ids it has not minted yet — and overrides what the model has an opinion on. risk_score stays deterministic and remains the only input to review ranking. Desktop fires both the metadata pass and Pass 1 from the analyze path, so the button count drops from four to two: Refine and Analyze This Flow. CLI stays explicit behind --describe. Fixes found on the way, all pre-existing: - Nothing re-scored after refinement, so merged groups sat at risk_score 0.0 and sorted as the least risky change in the diff. - The streaming refine path never wrote its result back to last_analysis, so annotate_overview summarized pre-refinement groups. - RefinementConfig::max_iterations was parsed, validated, merged, exposed as a desktop settings control, and read by nothing. Removed, along with the CLAUDE.md claim about an evaluator-optimizer loop that never existed. Removes Pass1GroupAnnotation; Pass 1 keeps only its PR-level overview. Desktop and the VS Code extension read the group's own fields instead. Spec: specs/group-metadata.md Refs: jamesaphoenix#13
Uh oh!
There was an error while loading. Please reload this page.
jamesaphoenix
commented
Sep 2, 2026
Owner
Thanks @jakob1379, going to do a new release now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes#13.
Group narrative previously existed only behind a manual Pass 1 call, keyed in a side-car object the consumer had to join against the group. Every
FlowGroupnow carries its own review metadata —group_type,risk,impact,complexity,review_focus,description,summary,invariant— answering how should I review this group rather than restating what changed.How it works
A deterministic heuristic floor fills
risk,group_typeandimpactfor free on every analysis.description,invariant,review_focusandcomplexityare deliberately left empty without an LLM — a wrong invariant sends a reviewer hunting for a property that was never at stake.An optional batched LLM pass runs over the final groups, after refinement ops are applied, and overrides what the model has an opinion on. It cannot ride inside
RefinementResponse:apply_splitmints ids at apply time andapply_mergereuses the first source id, so a model answering the refinement prompt cannot key metadata to groups that do not exist yet. Batches carry a read-only index of every group soCrossCuttingstays judgeable, and never read each other's results, so output does not depend on completion order.risk_scorestays deterministic and remains the only input to review ranking.Riskis a label derived from it; the model may override the label, never the score.Desktop fires both the metadata pass and Pass 1 from the analyze path, taking the button count from four to two — Refine and Analyze This Flow. The CLI stays explicit behind
--describe, sodiffcore analyzein CI never starts billing silently.Pre-existing bugs fixed on the way
risk_score: 0.0and sorted as the least risky change in the diff.last_analysis, soannotate_overviewsummarized pre-refinement groups.RefinementConfig::max_iterationswas parsed, validated, merged, and exposed as a desktop settings control, and read by nothing. Removed, along with the CLAUDE.md claim about an evaluator-optimizer loop that never existed.Breaking
Pass1GroupAnnotationis removed; Pass 1 keeps only its PR-level overview. Desktop and the VS Code extension read the group's own fields instead. NewFlowGroupfields are all#[serde(default)], so previously written analysis JSON still deserializes.Verification
cargo test --workspace26/26 binaries green · Playwright 225 passed / 2 skipped · VS Code extension 120 passed ·nix build .#diff-coreand.#diffcore-webclean.Not verified:
--describehas never made a live API call. The pass is unit-tested and the UI is E2E-tested against mocks, but no real model response has been through it — worth a look before this is trusted.Design decisions and their rationale are in
specs/group-metadata.md.