Added benchmarks for plots

[LucaMarconato]

I added some basic sets of benchmarks for measuring the time to produce plots. The correctness of the plots is currently not checked, we can modify it later.

The only thing missing from the PR is the integration with GitHub Actions, so that we will catch on time eventual future performance drops. To finalize that I need some permissions from scverse.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.34%. Comparing base (ac4f287) to head (ae80eec).
Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #295 +/- ##
=======================================
Coverage 80.34% 80.34% =======================================
Files 11 11 Lines 1745 1745 =======================================
Hits 1402 1402 Misses 343 343 

[melonora]

you mean for the token?

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕12 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_plot_can_render_large_circles[datashader] (318.2 ms)
• test_plot_can_render_large_circles[matplotlib] (402.9 ms)
• test_plot_can_render_large_image[image] (273.7 ms)
• test_plot_can_render_large_image[multiscale_image] (298.1 ms)
• test_plot_can_render_large_labels[labels] (431.3 ms)
• test_plot_can_render_large_labels[multiscale_labels] (448.4 ms)
• test_plot_can_render_large_multipolygons[datashader] (499 ms)
• test_plot_can_render_large_multipolygons[matplotlib] (622.9 ms)
• test_plot_can_render_large_points[datashader] (514.3 ms)
• test_plot_can_render_large_points[matplotlib] (250.1 ms)
• test_plot_can_render_large_polygons[datashader] (382.1 ms)
• test_plot_can_render_large_polygons[matplotlib] (438.8 ms)

[LucaMarconato]

Good news I setup everything. Bad news, CodSpeed doesn't seem to be suitable for the benchmarking use cases of spatialdata-plot because it excludes system calls by design, and we need them when looking at plotting performance.

See for example this screenshot; the execution time for test_plot_can_render_large_image[image] is 54.2s, but CodSpeed reports 273.7 milliseconds because most of the time is spent on system calls:

I wrote to the devs to see if there is an option to get around this, but most likely I'll end up closing this PR and switching to asv; which btw @Czaki is setting up on napari-spatialdatascverse/napari-spatialdata#289 right today, good timing!

EDIT: gonna close the PR already, wait for an answer and unless we can go back to CodSpeed, we can proceed with airspeed velocity (asv).

[LucaMarconato]

I archived the branch behind this PR here https://github.com/scverse/spatialdata-plot/releases/tag/archive%2Fcodspeed_benchmarks, I'll delete this branch.

[LucaMarconato]

I added some basic sets of benchmarks for measuring the time to produce plots. The correctness of the plots is currently not checked, we can modify it later.

The only thing missing from the PR is the integration with GitHub Actions, so that we will catch on time eventual future performance drops. To finalize that I need some permissions from scverse.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.34%. Comparing base (ac4f287) to head (ae80eec).
Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #295 +/- ##
=======================================
Coverage 80.34% 80.34% =======================================
Files 11 11 Lines 1745 1745 =======================================
Hits 1402 1402 Misses 343 343 

[melonora]

you mean for the token?

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕12 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_plot_can_render_large_circles[datashader] (318.2 ms)
• test_plot_can_render_large_circles[matplotlib] (402.9 ms)
• test_plot_can_render_large_image[image] (273.7 ms)
• test_plot_can_render_large_image[multiscale_image] (298.1 ms)
• test_plot_can_render_large_labels[labels] (431.3 ms)
• test_plot_can_render_large_labels[multiscale_labels] (448.4 ms)
• test_plot_can_render_large_multipolygons[datashader] (499 ms)
• test_plot_can_render_large_multipolygons[matplotlib] (622.9 ms)
• test_plot_can_render_large_points[datashader] (514.3 ms)
• test_plot_can_render_large_points[matplotlib] (250.1 ms)
• test_plot_can_render_large_polygons[datashader] (382.1 ms)
• test_plot_can_render_large_polygons[matplotlib] (438.8 ms)

[LucaMarconato]

Good news I setup everything. Bad news, CodSpeed doesn't seem to be suitable for the benchmarking use cases of spatialdata-plot because it excludes system calls by design, and we need them when looking at plotting performance.

See for example this screenshot; the execution time for test_plot_can_render_large_image[image] is 54.2s, but CodSpeed reports 273.7 milliseconds because most of the time is spent on system calls:

I wrote to the devs to see if there is an option to get around this, but most likely I'll end up closing this PR and switching to asv; which btw @Czaki is setting up on napari-spatialdatascverse/napari-spatialdata#289 right today, good timing!

EDIT: gonna close the PR already, wait for an answer and unless we can go back to CodSpeed, we can proceed with airspeed velocity (asv).

[LucaMarconato]

I archived the branch behind this PR here https://github.com/scverse/spatialdata-plot/releases/tag/archive%2Fcodspeed_benchmarks, I'll delete this branch.

[LucaMarconato]

I added some basic sets of benchmarks for measuring the time to produce plots. The correctness of the plots is currently not checked, we can modify it later.

The only thing missing from the PR is the integration with GitHub Actions, so that we will catch on time eventual future performance drops. To finalize that I need some permissions from scverse.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.34%. Comparing base (ac4f287) to head (ae80eec).
Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #295 +/- ##
=======================================
Coverage 80.34% 80.34% =======================================
Files 11 11 Lines 1745 1745 =======================================
Hits 1402 1402 Misses 343 343 

[melonora]

you mean for the token?

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕12 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_plot_can_render_large_circles[datashader] (318.2 ms)
• test_plot_can_render_large_circles[matplotlib] (402.9 ms)
• test_plot_can_render_large_image[image] (273.7 ms)
• test_plot_can_render_large_image[multiscale_image] (298.1 ms)
• test_plot_can_render_large_labels[labels] (431.3 ms)
• test_plot_can_render_large_labels[multiscale_labels] (448.4 ms)
• test_plot_can_render_large_multipolygons[datashader] (499 ms)
• test_plot_can_render_large_multipolygons[matplotlib] (622.9 ms)
• test_plot_can_render_large_points[datashader] (514.3 ms)
• test_plot_can_render_large_points[matplotlib] (250.1 ms)
• test_plot_can_render_large_polygons[datashader] (382.1 ms)
• test_plot_can_render_large_polygons[matplotlib] (438.8 ms)

[LucaMarconato]

Good news I setup everything. Bad news, CodSpeed doesn't seem to be suitable for the benchmarking use cases of spatialdata-plot because it excludes system calls by design, and we need them when looking at plotting performance.

See for example this screenshot; the execution time for test_plot_can_render_large_image[image] is 54.2s, but CodSpeed reports 273.7 milliseconds because most of the time is spent on system calls:

I wrote to the devs to see if there is an option to get around this, but most likely I'll end up closing this PR and switching to asv; which btw @Czaki is setting up on napari-spatialdatascverse/napari-spatialdata#289 right today, good timing!

EDIT: gonna close the PR already, wait for an answer and unless we can go back to CodSpeed, we can proceed with airspeed velocity (asv).

[LucaMarconato]

I archived the branch behind this PR here https://github.com/scverse/spatialdata-plot/releases/tag/archive%2Fcodspeed_benchmarks, I'll delete this branch.

[LucaMarconato]

I added some basic sets of benchmarks for measuring the time to produce plots. The correctness of the plots is currently not checked, we can modify it later.

The only thing missing from the PR is the integration with GitHub Actions, so that we will catch on time eventual future performance drops. To finalize that I need some permissions from scverse.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.34%. Comparing base (ac4f287) to head (ae80eec).
Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #295 +/- ##
=======================================
Coverage 80.34% 80.34% =======================================
Files 11 11 Lines 1745 1745 =======================================
Hits 1402 1402 Misses 343 343 

[melonora]

you mean for the token?

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕12 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_plot_can_render_large_circles[datashader] (318.2 ms)
• test_plot_can_render_large_circles[matplotlib] (402.9 ms)
• test_plot_can_render_large_image[image] (273.7 ms)
• test_plot_can_render_large_image[multiscale_image] (298.1 ms)
• test_plot_can_render_large_labels[labels] (431.3 ms)
• test_plot_can_render_large_labels[multiscale_labels] (448.4 ms)
• test_plot_can_render_large_multipolygons[datashader] (499 ms)
• test_plot_can_render_large_multipolygons[matplotlib] (622.9 ms)
• test_plot_can_render_large_points[datashader] (514.3 ms)
• test_plot_can_render_large_points[matplotlib] (250.1 ms)
• test_plot_can_render_large_polygons[datashader] (382.1 ms)
• test_plot_can_render_large_polygons[matplotlib] (438.8 ms)

[LucaMarconato]

Good news I setup everything. Bad news, CodSpeed doesn't seem to be suitable for the benchmarking use cases of spatialdata-plot because it excludes system calls by design, and we need them when looking at plotting performance.

See for example this screenshot; the execution time for test_plot_can_render_large_image[image] is 54.2s, but CodSpeed reports 273.7 milliseconds because most of the time is spent on system calls:

I wrote to the devs to see if there is an option to get around this, but most likely I'll end up closing this PR and switching to asv; which btw @Czaki is setting up on napari-spatialdatascverse/napari-spatialdata#289 right today, good timing!

EDIT: gonna close the PR already, wait for an answer and unless we can go back to CodSpeed, we can proceed with airspeed velocity (asv).

[LucaMarconato]

I archived the branch behind this PR here https://github.com/scverse/spatialdata-plot/releases/tag/archive%2Fcodspeed_benchmarks, I'll delete this branch.

[LucaMarconato]

I added some basic sets of benchmarks for measuring the time to produce plots. The correctness of the plots is currently not checked, we can modify it later.

The only thing missing from the PR is the integration with GitHub Actions, so that we will catch on time eventual future performance drops. To finalize that I need some permissions from scverse.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.34%. Comparing base (ac4f287) to head (ae80eec).
Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #295 +/- ##
=======================================
Coverage 80.34% 80.34% =======================================
Files 11 11 Lines 1745 1745 =======================================
Hits 1402 1402 Misses 343 343 

[melonora]

you mean for the token?

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕12 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_plot_can_render_large_circles[datashader] (318.2 ms)
• test_plot_can_render_large_circles[matplotlib] (402.9 ms)
• test_plot_can_render_large_image[image] (273.7 ms)
• test_plot_can_render_large_image[multiscale_image] (298.1 ms)
• test_plot_can_render_large_labels[labels] (431.3 ms)
• test_plot_can_render_large_labels[multiscale_labels] (448.4 ms)
• test_plot_can_render_large_multipolygons[datashader] (499 ms)
• test_plot_can_render_large_multipolygons[matplotlib] (622.9 ms)
• test_plot_can_render_large_points[datashader] (514.3 ms)
• test_plot_can_render_large_points[matplotlib] (250.1 ms)
• test_plot_can_render_large_polygons[datashader] (382.1 ms)
• test_plot_can_render_large_polygons[matplotlib] (438.8 ms)

[LucaMarconato]

Good news I setup everything. Bad news, CodSpeed doesn't seem to be suitable for the benchmarking use cases of spatialdata-plot because it excludes system calls by design, and we need them when looking at plotting performance.

See for example this screenshot; the execution time for test_plot_can_render_large_image[image] is 54.2s, but CodSpeed reports 273.7 milliseconds because most of the time is spent on system calls:

I wrote to the devs to see if there is an option to get around this, but most likely I'll end up closing this PR and switching to asv; which btw @Czaki is setting up on napari-spatialdatascverse/napari-spatialdata#289 right today, good timing!

EDIT: gonna close the PR already, wait for an answer and unless we can go back to CodSpeed, we can proceed with airspeed velocity (asv).

[LucaMarconato]

I archived the branch behind this PR here https://github.com/scverse/spatialdata-plot/releases/tag/archive%2Fcodspeed_benchmarks, I'll delete this branch.

[LucaMarconato]

I added some basic sets of benchmarks for measuring the time to produce plots. The correctness of the plots is currently not checked, we can modify it later.

The only thing missing from the PR is the integration with GitHub Actions, so that we will catch on time eventual future performance drops. To finalize that I need some permissions from scverse.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.34%. Comparing base (ac4f287) to head (ae80eec).
Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #295 +/- ##
=======================================
Coverage 80.34% 80.34% =======================================
Files 11 11 Lines 1745 1745 =======================================
Hits 1402 1402 Misses 343 343 

[melonora]

you mean for the token?

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕12 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_plot_can_render_large_circles[datashader] (318.2 ms)
• test_plot_can_render_large_circles[matplotlib] (402.9 ms)
• test_plot_can_render_large_image[image] (273.7 ms)
• test_plot_can_render_large_image[multiscale_image] (298.1 ms)
• test_plot_can_render_large_labels[labels] (431.3 ms)
• test_plot_can_render_large_labels[multiscale_labels] (448.4 ms)
• test_plot_can_render_large_multipolygons[datashader] (499 ms)
• test_plot_can_render_large_multipolygons[matplotlib] (622.9 ms)
• test_plot_can_render_large_points[datashader] (514.3 ms)
• test_plot_can_render_large_points[matplotlib] (250.1 ms)
• test_plot_can_render_large_polygons[datashader] (382.1 ms)
• test_plot_can_render_large_polygons[matplotlib] (438.8 ms)

[LucaMarconato]

Good news I setup everything. Bad news, CodSpeed doesn't seem to be suitable for the benchmarking use cases of spatialdata-plot because it excludes system calls by design, and we need them when looking at plotting performance.

See for example this screenshot; the execution time for test_plot_can_render_large_image[image] is 54.2s, but CodSpeed reports 273.7 milliseconds because most of the time is spent on system calls:

I wrote to the devs to see if there is an option to get around this, but most likely I'll end up closing this PR and switching to asv; which btw @Czaki is setting up on napari-spatialdatascverse/napari-spatialdata#289 right today, good timing!

EDIT: gonna close the PR already, wait for an answer and unless we can go back to CodSpeed, we can proceed with airspeed velocity (asv).

[LucaMarconato]

I archived the branch behind this PR here https://github.com/scverse/spatialdata-plot/releases/tag/archive%2Fcodspeed_benchmarks, I'll delete this branch.

[LucaMarconato]

I added some basic sets of benchmarks for measuring the time to produce plots. The correctness of the plots is currently not checked, we can modify it later.

The only thing missing from the PR is the integration with GitHub Actions, so that we will catch on time eventual future performance drops. To finalize that I need some permissions from scverse.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.34%. Comparing base (ac4f287) to head (ae80eec).
Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #295 +/- ##
=======================================
Coverage 80.34% 80.34% =======================================
Files 11 11 Lines 1745 1745 =======================================
Hits 1402 1402 Misses 343 343 

[melonora]

you mean for the token?

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕12 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_plot_can_render_large_circles[datashader] (318.2 ms)
• test_plot_can_render_large_circles[matplotlib] (402.9 ms)
• test_plot_can_render_large_image[image] (273.7 ms)
• test_plot_can_render_large_image[multiscale_image] (298.1 ms)
• test_plot_can_render_large_labels[labels] (431.3 ms)
• test_plot_can_render_large_labels[multiscale_labels] (448.4 ms)
• test_plot_can_render_large_multipolygons[datashader] (499 ms)
• test_plot_can_render_large_multipolygons[matplotlib] (622.9 ms)
• test_plot_can_render_large_points[datashader] (514.3 ms)
• test_plot_can_render_large_points[matplotlib] (250.1 ms)
• test_plot_can_render_large_polygons[datashader] (382.1 ms)
• test_plot_can_render_large_polygons[matplotlib] (438.8 ms)

[LucaMarconato]

Good news I setup everything. Bad news, CodSpeed doesn't seem to be suitable for the benchmarking use cases of spatialdata-plot because it excludes system calls by design, and we need them when looking at plotting performance.

See for example this screenshot; the execution time for test_plot_can_render_large_image[image] is 54.2s, but CodSpeed reports 273.7 milliseconds because most of the time is spent on system calls:

I wrote to the devs to see if there is an option to get around this, but most likely I'll end up closing this PR and switching to asv; which btw @Czaki is setting up on napari-spatialdatascverse/napari-spatialdata#289 right today, good timing!

EDIT: gonna close the PR already, wait for an answer and unless we can go back to CodSpeed, we can proceed with airspeed velocity (asv).

[LucaMarconato]

I archived the branch behind this PR here https://github.com/scverse/spatialdata-plot/releases/tag/archive%2Fcodspeed_benchmarks, I'll delete this branch.

[LucaMarconato]

I added some basic sets of benchmarks for measuring the time to produce plots. The correctness of the plots is currently not checked, we can modify it later.

The only thing missing from the PR is the integration with GitHub Actions, so that we will catch on time eventual future performance drops. To finalize that I need some permissions from scverse.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.34%. Comparing base (ac4f287) to head (ae80eec).
Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #295 +/- ##
=======================================
Coverage 80.34% 80.34% =======================================
Files 11 11 Lines 1745 1745 =======================================
Hits 1402 1402 Misses 343 343 

[melonora]

you mean for the token?

[codspeed-hq]

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕12 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_plot_can_render_large_circles[datashader] (318.2 ms)
• test_plot_can_render_large_circles[matplotlib] (402.9 ms)
• test_plot_can_render_large_image[image] (273.7 ms)
• test_plot_can_render_large_image[multiscale_image] (298.1 ms)
• test_plot_can_render_large_labels[labels] (431.3 ms)
• test_plot_can_render_large_labels[multiscale_labels] (448.4 ms)
• test_plot_can_render_large_multipolygons[datashader] (499 ms)
• test_plot_can_render_large_multipolygons[matplotlib] (622.9 ms)
• test_plot_can_render_large_points[datashader] (514.3 ms)
• test_plot_can_render_large_points[matplotlib] (250.1 ms)
• test_plot_can_render_large_polygons[datashader] (382.1 ms)
• test_plot_can_render_large_polygons[matplotlib] (438.8 ms)

[LucaMarconato]

Good news I setup everything. Bad news, CodSpeed doesn't seem to be suitable for the benchmarking use cases of spatialdata-plot because it excludes system calls by design, and we need them when looking at plotting performance.

See for example this screenshot; the execution time for test_plot_can_render_large_image[image] is 54.2s, but CodSpeed reports 273.7 milliseconds because most of the time is spent on system calls:

I wrote to the devs to see if there is an option to get around this, but most likely I'll end up closing this PR and switching to asv; which btw @Czaki is setting up on napari-spatialdatascverse/napari-spatialdata#289 right today, good timing!

EDIT: gonna close the PR already, wait for an answer and unless we can go back to CodSpeed, we can proceed with airspeed velocity (asv).

[LucaMarconato]

I archived the branch behind this PR here https://github.com/scverse/spatialdata-plot/releases/tag/archive%2Fcodspeed_benchmarks, I'll delete this branch.