Uh oh!
There was an error while loading. Please reload this page.
Refresh the benchmark charts from the v1.0.0-rc.1 run - #2
Merged
Conversation
The published charts were generated on 2026-05-05 and have been through three releases since. These come from CI run 33888625464: a median of five runs on Linux, EnergyPlus 9.3, the same 1,700-object model and the same workflow that produced the previous set. The headline ratio on "Get single object by name" reads lower than the charts it replaces, 1768x against roughly 3000x, but nothing regressed. idfkit's own time on that operation is a sub-microsecond measurement and jitters between 1502ns and 2765ns across the last five CI runs while the slowest competitor holds steady at 4.6 to 4.9ms, so the ratio has swung between 1775x and 3275x on noise alone. v0.15.0 already measured 1775x. The ESO charts are untouched: bench_eso.py needs a 104 MB input the benchmark workflow does not carry, so those are still refreshed by hand.
Contributor
Docs preview for this PR is available at: |
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
All 14 benchmark charts under
docs/assets/now come from CI run 33888625464: a median of five runs on Linux, EnergyPlus 9.3, the same 1,700-object model and the same workflow that produced the set they replace. The previous charts were generated on 2026-05-05 and have been through three releases since.docs/benchmarks.mdneeds no edit: it describes a 1,700-object IDF file, and the new run reportstotal_objects: 1700.The headline number reads lower, and nothing regressed
The hero chart now says 1768x where the old one said roughly 3000x. Recovering
results.jsonfrom past release assets shows why. All five are CI median-of-5 on the same runner image:The competitors are stable within 4.6-4.9ms. idfkit's own time on
Get single object by nameis a sub-microsecond measurement that jitters between 1502ns and 2765ns, so the published ratio has swung between 1775x and 3275x on noise alone. The 3000x figure came from v0.14.0's low end of that range; v0.15.0 already measured 1775x, and this run lands beside it.The ESO charts are untouched
benchmark_eso_*.svgcome frombench_eso.py, which needs a 104 MB annual.esothe benchmark workflow does not carry. Those four are still refreshed by hand and are unchanged here.Related
docs/tree that moved to this repository.README.mdbut only uploads it as an artifact, so no run has ever committed the result. That claim is stale independently of this PR and is worth its own change in idfkit.Checks
make docs-testfails locally on this machine for want oflibcairo, identically on pristinemain(106 warnings either way), so this change is neutral to it.make checkfails on pristinemaintoo, on an S310 finding inscripts/sync_js_artifacts.pyfrom a pre-commit-pinned ruff newer than the project's.