Skip to content

chore: benchmarks - #567

Merged
FabianHofmann merged 5 commits into
PyPSA:masterfrom
fluxopt:feat/benchmarks
Mar 30, 2026
Merged

chore: benchmarks#567
FabianHofmann merged 5 commits into
PyPSA:masterfrom
fluxopt:feat/benchmarks

Conversation

@FBumann

@FBumannFBumann commented Feb 2, 2026

Copy link
Copy Markdown
Collaborator

Internal performance benchmarks

Adds a benchmarks/ directory for tracking linopy's own build time, LP write speed, matrix generation, and peak memory across problem sizes.

Tools

Models

ModelDescriptionSizes
basicDense NN model, 2N^2 vars/cons10 — 1600
knapsackN binary variables, 1 constraint100 — 1M
expression_arithmeticBroadcasting, scaling, summation across dims10 — 1000
sparse_networkRing network with mismatched bus/line coords10 — 1000
pypsa_scigridReal power system (requires pypsa)10 — 200 snapshots

Timing phases

PhaseFileWhat it measures
Buildtest_build.pyModel construction (add_variables, add_constraints, add_objective)
LP writetest_lp_write.pyWriting the model to an LP file
Matricestest_matrices.pyGenerating sparse matrices (A, b, c, bounds) from the model

Memory benchmarks

memory.py runs each test in a separate process with pytest-memray for accurate per-test peak memory. Results are saved as JSON for cross-branch comparison.

python benchmarks/memory.py save master
python benchmarks/memory.py save my-feature
python benchmarks/memory.py compare master my-feature

Quick start

pip install -e ".[benchmark]"# Timing
pytest benchmarks/ --quick
pytest benchmarks/test_build.py --benchmark-save=master
# Memory
python benchmarks/memory.py save master --quick

See benchmarks/README.md for full details.

Checklist

  • Code changes are sufficiently documented
  • A note for the release notes doc/release_notes.rst is included
  • I consent to the release of this PR's code under the MIT license

@FBumann

Copy link
Copy Markdown
CollaboratorAuthor

Maybe we should merge sth like this before merging #591

@FBumann
FBumann requested a review from lkstrpMarch 12, 2026 10:12
@FBumann
FBumannforce-pushed the feat/benchmarks branch 2 times, most recently from 54ca82f to 769616eCompareMarch 12, 2026 12:14
Adds benchmarks/ directory with pytest-benchmark for timing and
pytest-memray for peak memory measurement across problem sizes.
Models: basic (dense N*N), knapsack (N binary vars), expression
arithmetic (broadcasting/scaling), sparse network (ring topology),
and pypsa_scigrid (real power system).
Timing phases: build (test_build.py), LP write (test_lp_write.py),
matrix generation (test_matrices.py). Memory benchmarks (memory.py)
measure the build phase only — memray tracks all allocations within
a test including setup, so other phases would conflate build and
phase-specific memory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
FBumannand others added 3 commits March 13, 2026 09:16
Benchmarks are not run in CI and should not affect coverage metrics.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevents false failures from minor coverage fluctuations when adding
non-library files like benchmarks or config changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The codecov/project failure is a pre-existing repo-wide issue (multiple
open PRs fail the same check), not caused by this PR.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@FBumann

Copy link
Copy Markdown
CollaboratorAuthor

Should be merged before #616#617#618#619

@FabianHofmann

FabianHofmann commented Mar 30, 2026

Copy link
Copy Markdown
Collaborator

I really like this and would pull this in ASAP as I am using something very similar in #630 , do you see there is anything missing?

@FabianHofmannFabianHofmann added this to the v0.7.0 milestone Mar 30, 2026
@FBumann

Copy link
Copy Markdown
CollaboratorAuthor

@FabianHofmann No, nothing missing. Its also not critical as its not part of the package and needs to evolve over time anyway.

@FabianHofmann
FabianHofmann merged commit c6b07e5 into PyPSA:masterMar 30, 2026
21 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@FBumann@FabianHofmann