Skip to content

fix(perf)!: unify local and release performance workflows - #210

Merged
acgetchell merged 1 commit into
mainfrom
fix/performance-workflow
Aug 12, 2026
Merged

fix(perf)!: unify local and release performance workflows#210
acgetchell merged 1 commit into
mainfrom
fix/performance-workflow

Conversation

@acgetchell

@acgetchellacgetchell commented Aug 12, 2026

Copy link
Copy Markdown
Owner
  • Retain local Markdown, CSV, and provenance artifacts while allowing same-version comparisons of tracked changes.
  • Exclude untracked files explicitly and isolate the narrowed non-exact comparison bundle.
  • Promote distinct-release artifacts through performance-doc and keep performance-release atomic and rollback-capable.
  • Render one consolidated table per benchmark suite and reject invalid release publication states.

BREAKING CHANGE: rename performance-rerender to performance-doc and performance-local-vs-linalg to performance-local-non-exact, replace --rerender with --promote-artifacts, and remove performance-archive-published.

Summary by CodeRabbit

  • New Features

    • Added a dedicated performance-doc workflow for generating and promoting documentation-ready reports.
    • Added retained CSV, JSON, and provenance artifacts for repeatable performance comparisons.
    • Added support for local same-version comparisons and non-exact scratch reports.
    • Consolidated benchmark results into clearer suite-based tables with a shared Case column.
  • Documentation

    • Expanded benchmarking, performance, release, and contribution guidance.
    • Clarified artifact validation, promotion behavior, tag checks, and tracked-file handling.
  • Bug Fixes

    • Prevented invalid same-version release promotions and improved command validation.

- Retain local Markdown, CSV, and provenance artifacts while allowing same-version comparisons of tracked changes.
- Exclude untracked files explicitly and isolate the narrowed non-exact comparison bundle.
- Promote distinct-release artifacts through performance-doc and keep performance-release atomic and rollback-capable.
- Render one consolidated table per benchmark suite and reject invalid release publication states.
BREAKING CHANGE: rename performance-rerender to performance-doc and performance-local-vs-linalg to performance-local-non-exact, replace --rerender with --promote-artifacts, and remove performance-archive-published.
@acgetchellacgetchell self-assigned this Aug 12, 2026
@acgetchell
acgetchell merged commit 58cccfc into mainAug 12, 2026
12 of 13 checks passed
@acgetchell
acgetchell deleted the fix/performance-workflow branch August 12, 2026 20:43
@coderabbitai

coderabbitaiBot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0d65ddfe-a705-4a36-af84-552dda1dd9f2

📥 Commits

Reviewing files that changed from the base of the PR and between a94e698 and c601c41.

📒 Files selected for processing (14)
  • CONTRIBUTING.md
  • docs/BENCHMARKING.md
  • docs/PERFORMANCE.md
  • docs/RELEASING.md
  • justfile
  • scripts/README.md
  • scripts/archive_performance.py
  • scripts/bench_compare.py
  • scripts/check_docs_version_sync.py
  • scripts/performance_artifacts.py
  • scripts/tests/test_archive_performance.py
  • scripts/tests/test_bench_compare.py
  • scripts/tests/test_check_docs_version_sync.py
  • scripts/tests/test_performance_artifacts.py

📝 Walkthrough

Walkthrough

The PR updates performance workflows to retain validated CSV and provenance artifacts, support separate local and documentation-only commands, validate release identifiers, promote reports without rerunning benchmarks, and render consolidated suite-level comparison tables.

Changes

Performance artifact workflow

Layer / File(s)Summary
Workflow commands and documentation
CONTRIBUTING.md, docs/BENCHMARKING.md, docs/PERFORMANCE.md, docs/RELEASING.md, justfile, scripts/README.md, scripts/check_docs_version_sync.py
The workflows use performance-local-non-exact and performance-doc. Documentation describes retained CSV and provenance artifacts, tracked-file handling, validation, and promotion.
Artifact generation and promotion
scripts/archive_performance.py, scripts/performance_artifacts.py, scripts/tests/test_archive_performance.py, scripts/tests/test_performance_artifacts.py
Generation writes validated artifacts and Markdown. Promotion reloads artifacts without rerunning benchmarks. Same-version local comparisons are supported, while same-version promotion is rejected.
Suite-level comparison rendering
scripts/bench_compare.py, scripts/tests/test_bench_compare.py
Snapshot and comparison output now uses one table per suite with a shared Case column. Tests validate the consolidated format and retained-artifact workflow text.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
participant CLI
participant archive_performance
participant performance_artifacts
participant bench_compare
participant ReleaseArchive
CLI->>archive_performance: Generate comparison
archive_performance->>performance_artifacts: Export CSV and provenance JSON
archive_performance->>performance_artifacts: Reload and validate artifacts
performance_artifacts->>bench_compare: Provide comparison inputs
bench_compare-->>archive_performance: Render Markdown report
CLI->>archive_performance: Promote retained artifacts
archive_performance->>ReleaseArchive: Archive promoted report
Loading

Possibly related issues

Possibly related PRs

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/performance-workflow

Comment @coderabbitai help to get the list of available commands.

@codecov

codecovBot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.88%. Comparing base (a94e698) to head (c601c41).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@ Coverage Diff @@## main #210 +/- ##
=======================================
Coverage 97.88% 97.88% =======================================
Files 8 8 Lines 5007 5007 =======================================
Hits 4901 4901 Misses 106 106 
FlagCoverage Δ
unittests97.88% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@acgetchell