Skip to content

REGRESSION - IMPROVEMENT - Reuse one worktree per commit instead of per (case, ref) - #369

Merged
matt-pharr merged 5 commits into
developfrom
performance/regression-harness-worktree-reuse
Aug 14, 2026
Merged

REGRESSION - IMPROVEMENT - Reuse one worktree per commit instead of per (case, ref)#369
matt-pharr merged 5 commits into
developfrom
performance/regression-harness-worktree-reuse

Conversation

@jhalpern30

Copy link
Copy Markdown
Collaborator

Summary

  • The regression harness (regression-harness/) created and tore down a brand-new git worktree for every (case, ref) pair, so an N-case run against one commit paid for Pkg.instantiate() + a full precompile of GeneralizedPerturbedEquilibrium N separate times.
  • Cases are now run grouped by ref (run_cases_at_ref in runner.jl), sharing a single worktree across every case that needs the same commit. Worktree creation is skipped entirely when every case at that commit is already cached. run_commit/run_computed_at_commit/run_at_commit gained an optional worktree_path kwarg so they can reuse a shared worktree without changing behavior for any other caller.
  • Reporting is unchanged — still grouped by case, comparing across all refs.

Test plan

  • Benchmarked 3 lightweight kind="computed" cases (ggj_reference, ggj_ray_q500i, efit_fixedbdy_separatrix) against develop with a warm package depot: 2:02 → 1:28 wall time, 3 worktrees → 1.
  • Ran the full 11-case suite (excluding the unrelated fixed-boundary equilibrium case) against develop and local on refactor/vacuum-response-struct: completed in 40m52s end-to-end. The same run with the old code did not finish Pkg.instantiate() for even 9 of 11 develop-side cases in 52 minutes before it had to be killed.
  • Confirmed the harness itself still reports correctly: all 11 cases showed 0 numerical diff between develop and the working tree in that run.
  • No new regression-harness case needed — this is a tooling change to the harness driver, not physics code.

⚠️ Per repo policy: this PR must not be merged without a third-party human review approval — no exceptions.

🤖 Generated with Claude Code

…er (case, ref)
Every (case, ref) pair created and tore down its own git worktree, so N cases
against the same commit paid for Pkg.instantiate()/precompile N times over.
Cases now run grouped by ref, sharing one worktree per unique commit; worktree
creation is skipped entirely when every case at that commit is already cached.
Benchmarked on 3 lightweight cases against develop: 2:02 -> 1:28 wall time with
a warm package depot. On a cold depot or heavier cases the win is much larger —
a prior 11-case run against develop didn't finish Pkg.instantiate() for even
half the cases in 52 minutes; the same run with this change completes all 11
cases (both develop and local) in under 41 minutes.
@jhalpern30
jhalpern30 requested a review from d-burgAugust 13, 2026 15:37
@jhalpern30jhalpern30 added the feature New capability label Aug 13, 2026
@jhalpern30

Copy link
Copy Markdown
CollaboratorAuthor

Figured Mr. Regression himself @d-burg would be best to review this, but wanted to flag this for @logan-nc@matt-pharr and @priyanshlunia - I found this substantially speeds up multi-case runs which were taking forever by running all cases on each branch then switching to the other branch (instead of going case by case, switching branches each time). Should be helpful during the remainder of the cleanup hackathon

@logan-nclogan-nc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manual read checked out.
My AI caught a few minor robustness improvements that will land soon (checking regression run now), at which point we can merge. I'll add a follow-up PR that adds threads for more performance wins.

logan-ncand others added 2 commits August 14, 2026 11:56
…guard stale outputs
Worktree-creation failure at one commit is now stored as a failed run for
each uncached case and the remaining refs proceed (a bad commit no longer
aborts a --ref-range scan). Remove any pre-existing gpec.h5 before running
so a stale output cannot mask a failed run in a shared worktree or in the
repo example dir. Document the worktree_path ownership contract and the
single-invocation worktree sharing; note in CLAUDE.md that the pre-commit
JuliaFormatter version is not pinned.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…sion-harness-worktree-reuse
# Conflicts:
#	regression-harness/regress.jl
#	regression-harness/src/runner.jl
@logan-nc
logan-nc enabled auto-merge August 14, 2026 16:51
@matt-pharr
matt-pharr disabled auto-merge August 14, 2026 18:32
@matt-pharr
matt-pharr merged commit 0b196ed into developAug 14, 2026
4 checks passed
@matt-pharr
matt-pharr deleted the performance/regression-harness-worktree-reuse branch August 14, 2026 18:32
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

featureNew capabilitytestTests only

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jhalpern30@logan-nc@matt-pharr