Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent - #703

Merged
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper
Jun 14, 2026
Merged

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent#703
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper

Conversation

@timtreis

@timtreistimtreis commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

Fast rendering for shapes/labels, in spatialdata-plot's API:

  1. as_points=True — draw each cell as a dot at its centroid instead of its full geometry/mask (the squidpy spatial_scatter idea). matplotlib by default; auto-switches to datashader above ~500k dots, or with method="datashader".
  2. Fast axis-aligned extentpl.show() skips transforming every geometry to size the axes when the element's transform is axis-aligned (falls back to get_extent otherwise).
sdata.pl.render_shapes("cells", color="cell_type", as_points=True).pl.show()
sdata.pl.render_labels("cells", color="leiden", as_points=True, method="datashader").pl.show()

Notes

  • Default (as_points=False) output is unchanged vs main.
  • render_points' datashader pipeline is extracted into a shared _datashader_points (byte-identical) and reused by the centroid path.
  • Datashader can't represent the random-per-cell colours of uncolored labels, so that case stays matplotlib (with a warning).
  • Centroids are transformed to coordinate-system coords, fixing dot placement under non-identity transforms.

Tests

Position parity vs matplotlib (incl. non-identity transform), backend selection, no-color fallback, and test_plot_* visual baselines for matplotlib + datashader as_points (continuous, categorical, no-color).

… helper
Infrastructure for an upcoming "render cells as centroid points" fast mode
(no user-facing render option yet).
Phase 0 — shared scatter primitive:
- Extract `_scatter_points(ax, x, y, color_vector, ...)` from `_render_points`'s
matplotlib branch; `_render_points` now calls it. Byte-identical output
(verified vs main on categorical and continuous point renders). This is the
reuse seam the fast mode will draw through.
Phase 1 — centroid + caching core (headless, fully unit-tested):
- `_compute_element_centroids` / `_compute_label_centroids`: per-instance
centroids in a coordinate system. Shapes use spatialdata's vectorized
`get_centroids`; labels use skimage `regionprops` (the per-label reduction is
orders of magnitude faster than `get_centroids` on rasters), mapped onto the
raster's intrinsic coordinate arrays so it reproduces `get_centroids` exactly
(incl. the pixel-center 0.5 offset) then transformed to the target CS.
- `_get_or_compute_centroids`: reuses/persists centroids via the squidpy
convention. A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted
as the cells' locations; otherwise centroids are computed and written back into
the annotating table's `obsm["spatial"]` with a coordinate-system provenance
marker in `uns`, so later renders are instant. Reads run before writes, so a
valid existing cache is reused rather than clobbered; an incompatible existing
`obsm["spatial"]` is never overwritten; the cache is invalidated when the
requested coordinate system differs.
- Tests: shapes/labels centroids match `get_centroids`; cache round-trip +
provenance; CS invalidation; pre-existing obsm trusted; no-table compute path;
`cache=False` writes nothing.
… provenance
Cleanup from /simplify (no behavioral change):
- Extract `_region_mask_and_keys(table, element)` used by both read and write,
removing the duplicated `get_table_keys` + O(n_obs) `region_key`-string-cast
mask that was computed twice per cold call.
- Read path: validate shape on the raw obsm array and cast only the masked
subset to float, instead of casting the whole `obsm["spatial"]` on every
cache hit (the hot path).
- Write path: coerce a non-dict `uns["spatialdata_plot"]` instead of early
returning after `obsm` was already mutated, so obsm and the provenance marker
are always written together (no half-write).
- Drop the dead `"key"` provenance field (constant, never read back).
- Rename the misleading `table` local (held a table *name*) in
`_get_or_compute_centroids`.
Refactor the centroid cache to store element-*intrinsic* coordinates and
transform to the render coordinate system on demand, instead of caching
coords already mapped into one coordinate system. Decisions from design pass:
- Intrinsic storage: one `obsm["spatial"]` cache serves every coordinate
system (proven equivalent to per-CS computation). `_compute_element_centroids`
returns intrinsic coords (shapes via shapely `.centroid`, labels via
`regionprops`); `_centroids_to_coordinate_system` maps them to the requested
CS via the element's transform; `_get_or_compute_centroids` reads/computes
intrinsic then transforms on return.
- Provenance records `{n, scale_level}` (no coordinate system). Cache is
invalidated when the region's instance count changes (cells added/removed),
not on CS change.
- A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted as the
cells' intrinsic locations and transformed to the render CS.
- Exhaustive model dispatch: shapes and 2D labels supported; other element
types raise NotImplementedError.
- Labels are reduced at full resolution (scale0).
Drops the now-unused `get_centroids` import; adds `ShapesModel`. Tests updated:
parametrized shapes/labels match `get_centroids`; new coordinate-system-
independence test (one cache, two CS); staleness-by-instance-count; trusted
pre-existing obsm; unsupported-type rejection.
…e import, shared obsm gate
Cleanup from /simplify:
- `_centroids_to_coordinate_system` ran `PointsModel.parse` + a dask
`transform(...).compute()` round trip on every call, including cache hits
(~19 ms fixed floor, ~100 ms at 1M cells) — defeating the cache. Replace with
`to_affine_matrix` + a plain numpy matmul: numerically identical (verified
against the dask path for multiple coordinate systems), ~80-140x faster, and
it removes the private-API import `spatialdata._core.operations.transform`
(a repo non-negotiable) plus the now-unused `PointsModel`.
- Widen `_transformable_raster` -> `_transform_carrier` to accept any element
(rasters -> scale0, others as-is), dropping the `isinstance` branch in
`_centroids_to_coordinate_system`.
- Extract `_valid_spatial_obsm(arr, n_obs)` shared by the read and write paths,
reconciling their previously divergent obsm-shape checks (read accepted >=2
columns, write required exactly 2) so they cannot drift.
`render_shapes(..., as_points=True)` and `render_labels(..., as_points=True)`
draw one dot per cell at its centroid instead of the full geometry / rasterized
mask — a large speedup when only cell location matters. New `size=` controls the
marker size.
- Shared `_render_centroids_as_points` draws the scatter (via `_scatter_points`)
and the legend/colorbar. The per-cell color vector is the *same* one the
geometry/raster path computes (`_set_color_source_vec`), so colors match the
full rendering exactly; only the apply step (scatter vs patches/imshow) differs.
- Shapes: centroids from shapely `.centroid` of the (filtered) geometry,
positionally aligned to the color vector, drawn in intrinsic coords via the
element transform. Labels: centroids from `_get_or_compute_centroids`
(regionprops, fast) reindexed to `instance_id`. Positions verified identical to
`sd.get_centroids`.
- `as_points` short-circuits before the geometry/raster path; outline_*, shape
(shapes) and contour_px, outline_* (labels) are ignored with an info log.
- Default (`as_points=False`) output is byte-identical to main.
Tests: non-visual checks that centroids land exactly on `get_centroids` for both
element types and that outline/shape are ignored without error.
Note: as_points currently always uses the matplotlib scatter backend; routing
through datashader for very large cell counts (and persisting the obsm cache to
the user's object rather than show()'s working copy) are follow-ups.
@timtreistimtreis changed the title Centroid extraction + squidpy obsm["spatial"] caching; shared scatter helperFast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingJun 8, 2026
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.08046% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.89%. Comparing base (b370b1f) to head (882d208).

Files with missing linesPatch %Lines
src/spatialdata_plot/pl/utils.py81.81%6 Missing and 6 partials ⚠️
src/spatialdata_plot/pl/render.py92.70%4 Missing and 3 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #703 +/- ##
==========================================
+ Coverage 76.28% 76.89% +0.60% 
==========================================
Files 14 14 Lines 4327 4458 +131 Branches 1006 1035 +29 ==========================================
+ Hits 3301 3428 +127 + Misses 667 664 -3 - Partials 359 366 +7 
Files with missing linesCoverage Δ
src/spatialdata_plot/pl/_datashader.py90.50% <100.00%> (ø)
src/spatialdata_plot/pl/basic.py79.61% <100.00%> (+0.19%)⬆️
src/spatialdata_plot/pl/render_params.py89.02% <100.00%> (+0.22%)⬆️
src/spatialdata_plot/pl/render.py88.16% <92.70%> (+1.11%)⬆️
src/spatialdata_plot/pl/utils.py69.40% <81.81%> (+0.51%)⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`render_labels(element, as_points=True)` with no color crashed:
`instance_id` (the raster's unique values) includes the background label `0`,
which has no centroid, and the literal/no-color color vector is sized to the
raster (not per-instance), so `ax.scatter` got mismatched `c` vs `x`/`y`.
Drop the background label from the rendered instances and align the per-cell
color: for data-driven color the vector is already per-instance and is subset to
match; for the literal/no-color path it is replaced with one na/literal color
per centroid. Data-driven (categorical/continuous) renders are unchanged and
still land exactly on `get_centroids`.
Adds a regression test for the no-color labels case.
…ator
Replace the `regionprops` reduction in `_compute_label_centroids` with an
additive bincount aggregator that streams the labels raster block by block —
one dask chunk (or bounded numpy row-block) in memory at a time — accumulating
per-label `count`/`sum_x`/`sum_y`. This is what makes the feature usable at
Xenium scale:
- Out-of-core: peak memory is one chunk + O(n_labels) accumulators, NOT the
whole raster (measured: 9 MB peak streaming a 268 MB mask). `regionprops`
needs the full array materialized and OOMs on large morphology masks.
- Scales in cell count: 500k+ labels are just array indexing (562k labels in
~1.4 s with 13.5 MB of accumulators); `regionprops`' per-label table does not.
- Faster than `regionprops` (~1.3-1.6x) on in-memory rasters.
- Exact across chunk boundaries (additive reduction) — verified numpy ==
dask-chunked, and identical to `sd.get_centroids`.
- `count` is the cell area, a free by-product (ready for footprint-based dot
sizing).
Drops the `regionprops_table` import; adds `slices_from_chunks`. Adds a unit
test locking the chunk-exact, out-of-core, area-correct behavior.
Note: the chunk loop is currently sequential; parallelizing the per-chunk
partials (dask map_blocks + tree-reduce) is a future speedup.
#705 (measure_obs) merged to main, superseding #703's pre-extraction centroid
block. This merge takes main's utils.py + test_utils.py wholesale (zero #703
delta there) and rewires the labels as_points path onto main's primitive:
- render.py labels branch: drop the deleted `_get_or_compute_centroids` + the
whole cache layer; compute full-resolution (scale0) centroids via main's
`_compute_element_measurements`, and draw them with a transform built from the
*same* scale0 element (`_prepare_transformation(_get_top_data_array(...))`) —
so positions are independent of any rasterization applied to the rendered
`label`. Coerce point_ids to the label dtype so str/object instance ids
(e.g. Xenium readers) align instead of silently reindexing to NaN.
- Net: utils.py == main (the 242-line #703 centroid block + cache layer is
gone); the surviving diff vs main is just the as_points feature.
- Tests: added a non-identity-transform regression test asserting the dots land
at the cells' coordinate-system positions in display space (the guard for the
transform pairing); existing as_points position tests still pass.
Shapes branch unchanged (already intrinsic centroid + trans_data, positionally
aligned to its post-filter color vector).
…ked fields
The fast-mode draw wrapper read cmap/size/alpha/zorder/colorbar/colorbar_params
straight off render_params at both call sites. Pass the render_params dataclass
instead and read them internally; only the genuinely per-branch values
(x/y/color vectors, norm, na_color, transform, adata, palette, col_for_color)
stay explicit. Signature 20->15 params; both as_points call sites lose the
restated render_params plumbing.
Collapse the per-centroid color alignment (drop the unconditional asarray
pre-init and the nested None-guard into a single if/else + ternary), and drop a
redundant np.asarray around the already-ndarray point_ids in the reindex.
Behavior unchanged.
Benchmark showed labels as_points was 16-34x SLOWER than the normal render
because it recomputed full-resolution scale0 centroids while the normal path
downsamples + imshows. Compute centroids on the already-rendered (downsampled)
raster and draw with its trans_data instead: 671k cells goes 19.7s -> 0.88s
(now ~parity with imshow). Centroid error is sub-pixel at display resolution;
position tests updated to display-space within a few-px tolerance.
…gned
show()'s get_extent(exact=True) transforms EVERY shapes/points geometry into
the coordinate system just to take a bounding box - the dominant cost for large
shape collections (~85% of a 5.5M-shape render), unrelated to what is drawn.
Add a self-contained get_extent_fast (drop-in for spatialdata.get_extent) that,
for axis-aligned transforms (scale/flip/90deg/swap + translation - all real
Visium/Xenium data), transforms only the 4 bounding-box corners and reads the
intrinsic bounds vectorised (no per-geometry .apply(is_empty)). Proven identical
to the exact extent for such transforms (spatialdata's own get_extent docstring
notes this); falls back to spatialdata's get_extent for rotation/shear, for
anisotropically-scaled circles (radius->ellipse divergence), and for
images/labels (whose get_extent is already a cheap corner transform).
Measured on real Visium HD render_shapes(as_points=True): 351k 8.5s->1.8s
(4.7x), 5.48M 129s->24s (5.5x); the full (non-as_points) render benefits too.
The whole block is isolated so it can be lifted into spatialdata's get_extent
verbatim (see issue #706). Tests assert it matches get_extent across scale/flip/
rotation/shear for circles and polygons.
@timtreis
timtreisforce-pushed the feat/centroid-scatter-helper branch from 35c8f1f to 57bcf8dCompareJune 9, 2026 23:20
…ompute, underscore)
Follow-up cleanups from review of the get_extent fast path:
- fold _intrinsic_xy_bounds into _element_extent_fast (drop duplicate get_model / geom_type)
- geom.bounds -> geom.total_bounds (C-level union, avoids an Nx4 alloc on large collections)
- batch the four points min/max into one dask.compute
- rename get_extent_fast -> _get_extent_fast (internal helper; matches sibling underscored helpers)
No behavior change; output identical for axis-aligned and rotated/sheared elements.
The datashader shapes canvas sized itself via spatialdata's exact get_extent, transforming every
geometry (O(N)) just for a bounding box -- the same cost _get_extent_fast already removed from
show()'s axis limits, in a second code path. Reuse _element_extent_fast (corner transform for
axis-aligned elements; None -> exact get_extent fallback for rotation/shear), so the result is
pixel-identical and the per-geometry pass is skipped for the common case.
Measured on Curio (69,713 colored shapes): render_shapes 3146 ms -> 1675 ms (1.88x), on top of
the get_extent_fast win already in show(). Mirrors _datashader_canvas_from_dataframe, which
already avoids get_extent for the points path.
…de review)
From the perf-stack review of labels as_points:
- No color column collapsed every dot to a single na_color; now each cell gets a distinct random
colour, matching the mask path's _map_color_seg Case C. Adds a color assertion to the regression test.
- The rasterize drop-filter only ran when a color column was set, so as_points could emit dots at NaN
positions (or drop cells) when rasterization removed labels; extend it to as_points so point ids stay
within the rendered raster.
- ax.scatter autoscales the Normalize in place; copy the shared cmap_params.norm (the shapes path already does).
- render_shapes/render_labels(as_points=True) did not validate size; add the points-path numeric/positive
check so size=-5 / 'big' raise an actionable error instead of a raw matplotlib failure.
Extract the duplicated as_points size-validation block from render_shapes
and render_labels into a shared _validate_as_points_size helper. Defer the
np.asarray(color_vector) conversion in the centroid color path to the only
branch that uses it.
…to feat/centroid-scatter-helper
# Conflicts:
#	tests/pl/test_utils.py
_get_extent_fast already reads get_transformation(element, get_all=True) for
the coordinate-system membership check; pass it through so the fast path does
not re-fetch it per element. The optional kwarg leaves the datashader-canvas
call site unchanged.
_element_extent_fast returned a NaN extent for an element whose geometries are
all empty, which silently poisoned the union in _get_extent_fast (blank axes)
instead of raising spatialdata's clear 'empty collection' error. Guard against
non-finite bounds and fall back. Also trims the verbose extent-block comment and
the scatter/centroid docstrings (net -12 LOC, no behavior change).
@timtreistimtreis changed the title Fast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingFast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extentJun 10, 2026
Four PlotTester baselines: render_shapes/labels(as_points=True) at a base size
and a larger size (size = scatter marker area). Baselines to be generated from CI.
…l tests
Rendered on hatch-test.py3.11-stable; verified dots land at shape/label
centroids, colored by instance_id (labels), larger at size=600.
The empty-shape check ran a per-geometry Python lambda (`.apply(lambda g: not
g.is_empty)`) over every geometry on every shapes render; GeoSeries.is_empty is
GEOS-vectorized. `.is_empty.all()` is identical and ~134x faster on this guard
(665ms -> 5ms at 351k shapes, ~10s -> 80ms at 5.5M).
Shared datashader draw primitive (canvas->aggregate->norm->color-key->shade->
image->colorbar), taking explicit primitives instead of render_params so it can
also serve the as_points centroid path (shapes/labels params use fill_alpha and
lack the density fields). Returns the possibly-recomputed color vectors so the
caller's legend/colorbar stays identical. Verified pixel-identical on
render_points datashader (categorical/continuous/plain/density).
- Route as_points centroids through the shared _datashader_points when method=
'datashader' or above ~500k dots (AS_POINTS_DS_AUTO); matplotlib otherwise.
- No-color labels (one random colour per cell) cannot be aggregated by datashader;
force matplotlib there (warn on explicit method='datashader').
- Fix: shapes as_points drew dots at intrinsic centroid coords while the axes are
in coordinate-system coords, so non-identity transforms misplaced them. Transform
centroids to CS via the element->CS affine for both backends (labels too).
- render_labels gains 'method'; LabelsRenderParams gains 'method'. as_points uses a
fixed datashader reduction ('max', closest to matplotlib) — no user knob.
- shapes non-identity transform regression (the coordinate bug fixed in 5854ae6)
- backend selection: method='datashader' -> datashader image; default -> matplotlib
- no-color labels force matplotlib even with method='datashader' (warns)
- _resolve_as_points_method unit test (threshold/explicit/no-color/empty)
- two test_plot_* visual tests for datashader as_points (baselines from CI)
timtreis added 22 commits June 11, 2026 06:40
Rendered on hatch-test.py3.11-stable; verified datashaded centroids coloured by
category with a categorical legend. Only this test failed on stable CI (missing
baseline); existing datashader baselines unchanged.
_datashader_points' two near-identical reindex/align branches (categorical
source vs colour vector) differed only in which vector they materialize; pick
the source once, then share the reindex/construct. render_points datashader
output verified byte-identical (categorical/continuous/plain/density). -9 LOC.
The datashader as_points canvas was sized to the exact centroid bounding box,
so the outermost centroids sat on the canvas edge and their marker spread was
clipped (half-circles), looking nothing like the matplotlib backend. Grow the
canvas by the spread radius on each side (factor unchanged, so placement still
aligns). render_points keeps pad_for_markers=False -> byte-identical.
…padding fix
The padding fix changed the datashader as_points renders within the comparison
tolerance, so the baselines didn't auto-update. Regenerate from CI so the
committed images reflect the un-clipped output.
CI-rendered (py3.11-stable) after the canvas-padding fix; verified the edge
centroids now render as full circles matching the matplotlib backend.
Restructure the as_points visual tests into matplotlib+datashader pairs that
render identical params (shared helper) for shapes (no color), labels
(instance_id), and labels (categorical). Drop the old inconsistent/stale
baselines; all 6 will be regenerated from CI so the two backends are directly
comparable and look maximally similar.
Positions and sizes align after the canvas-padding fix; dots land at the same
centroids at the same size. Remaining difference is color shading on the
categorical path (datashader modulates alpha by per-pixel count).
Datashader faded single-cell dots (count-driven alpha floor + a second user-alpha
multiply ~= alpha^2), making categorical as_points read much paler than the
matplotlib backend. Add uniform_alpha for the as_points marker mode: a full alpha
floor so each dot is one flat colour at fill_alpha, like a matplotlib marker.
Renamed the as_points flag pad_for_markers -> as_markers (pads canvas + uniform
alpha). render_points unchanged (byte-identical).
…h matplotlib)
CI-rendered (py3.11-stable); the datashader as_points dots now match the
matplotlib markers in colour saturation, position, and size.
…matplotlib
Size (derived, no heuristic): datashader previously rasterized over the centroid
bounding box, so dot display size depended on the canvas/axes extent ratio (shapes
~0.9x, labels ~1.7x matplotlib). Now the canvas spans the same extent as the axes
(like render_points), and the spread radius is set to matplotlib's marker radius
sqrt(s)*dpi/144 (its 'o' marker has diameter sqrt(s)*dpi/72). Result: ds/mpl size
ratio 1.02 +/- 0.03 across element type, size, figure, and dpi.
Continuous colorbar: the spread combined overlapping dots with 'add' (ds_reduction
None -> 'sum'), summing ids and inflating reduction_bounds; marker mode now spreads
with 'max' so overlaps overlay and the colorbar keeps the true range.
render_points unchanged (byte-identical). Replaces the canvas-padding workaround
(the axes-extent canvas already gives edge dots their margin).
… colorbar)
CI-rendered (py3.11-stable): datashader dots now match the matplotlib markers in
size, and the continuous colorbar uses the true value range.
The datashader result is a data-coordinate image that scales with the axes,
while matplotlib markers are fixed in display points. The canvas was sized from
fig.get_size_inches()*dpi (the whole figure), but the axes are smaller (margins,
colorbar), so the image - and every dot - was scaled down: labels (with colorbar)
0.81x matplotlib, shapes 0.88x. Size the as_markers canvas to the axes display box
(ax.get_window_extent()) so 1 canvas px == 1 axes-display px and the sqrt(s)*dpi/144
spread radius matches the marker. Now mean 0.99 +/-5% across sizes/figs/dpi/elements;
render_points untouched (byte-identical). Visual tests render at a non-overlapping
size so the engines' overlap handling (stack vs aggregate) doesn't enter the pairs.
…der canvas
CI-rendered (py3.11-stable). matplotlib and datashader pairs now match in dot
size (the canvas-vs-figure scaling bug is fixed) at a non-overlapping size.
Cleanup pass over the as_points feature (no behavior change):
- trim restated comments/docstrings in _datashader_points, keeping the
load-bearing marker-radius (sqrt(s)*dpi/144) and canvas-vs-axes rationale
- collapse the 3-way min_alpha if/elif/else to a ternary
- reuse the already-computed label extent instead of recomputing get_extent
- np.full instead of list*N for the literal na_color vector
- tighten the as_points test-helper comment
render_points output verified byte-identical; as_points renders unchanged.
Datashader pays off for as_points from ~50k cells (~1.66x faster than the
matplotlib scatter backend, rising to ~1.8x at 1M), so switch over there
instead of 500k. Users can still force either backend via method=.
The shapes as_points block computed its datashader canvas extent via
get_extent (exact=True), which transforms every geometry — on 351k Visium HD
shapes that was ~7s/render and dominated the whole as_points cost, so the fast
mode gave no speedup over full-geometry rendering. Use _element_extent_fast
(corner-transform; identical result for axis-aligned transforms, falls back to
get_extent for rotation/shear), which the PR already added for exactly this.
Real overlay (image + shapes color=gene), as_points:
91k: 7.2s -> 2.0s (3.3x)
351k: 24.3s -> 3.1s (7.3x, vs 22.8s full-geometry)
Output is unchanged: the fast extent is byte-identical for axis-aligned data.
- extract _fast_extent(element, cs) helper; the '_element_extent_fast(...) or
get_extent(...)' idiom was duplicated verbatim at the shapes as_points site
and the datashader-canvas helper
- fix docstrings that still said '~500k' after the threshold dropped to 50k
- trim _render_centroids_as_points docstring and a non-actionable comment
No behavior change: _fast_extent is identical to the inline form (verified
== get_extent on blobs), render_points byte-identical.
@timtreis
timtreis merged commit 8730ff4 into mainJun 14, 2026
7 of 8 checks passed
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #702 (per-panel title fix) and #703 (as_points + fast extent)
into the show() decomposition.
Conflict resolution in src/spatialdata_plot/pl/basic.py:
- #702 up-front title-count validation: kept (after num_panels). The
adjacent axes/panel-count check was dropped here because the
decomposition already relocated it into _plan_panels.
- #702 simplified title selection (dropped the per-panel try/except):
applied to the extracted _finalize_panel helper.
- #703 had no show()-level render-dispatch changes (as_points is
param-driven in render.py, handled inside _render_panel already); its
only show()-level change, get_extent -> _get_extent_fast, auto-merged
into the extent block and consumes _render_panel's `wants` dict.
Also sweeps up two pre-existing #703 lint/type nits in utils.py surfaced
by the merge: _fast_extent docstring (ruff D205) and _get_extent_fast
Any-return (mypy).
Verified: ruff + mypy clean; 109 non-visual show/shapes/labels tests pass.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 21 color helpers + 6 color-only format/uniqueness helpers verbatim from
utils.py into a new sibling module pl/_color.py. Imports flow one way:
_color -> utils (downward, for _get_list/to_hex/_build_alignment_dtype_hint/
_MPL_SINGLE_LETTER_COLORS). render.py, basic.py and _datashader.py are repointed
to import color symbols from _color.
The two validators still in utils (_type_check_params, _validate_graph_render_params)
use color symbols; they carry temporary function-local imports of _color (cycle-safe)
until they move to _validate.py in the next commit, where these become top-level imports.
No behavior change (verbatim move; all-private except set_zero_in_cmap_to_transparent,
which is not re-exported). Verified: no import cycle; 410 non-visual tests pass;
ruff + ruff-format clean. Pre-existing #703/#705 mypy/ruff debt in utils.py is
unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 17 validation/type-check functions verbatim from utils.py into a new sibling
module pl/_validate.py. Imports flow one way: _validate -> _color (_is_color_like,
_prepare_cmap_norm, _get_colors_for_categorical_obs, now top-level) and
_validate -> utils (downward). The temporary function-local _color imports added in
the previous commit are hoisted to top-level here and removed from utils.
render.py and basic.py repointed to _validate.
Completes the utils.py split (#696): utils 4918 -> 1311 lines, with
_geometry / _datashader / _color / _validate as single-concern siblings.
No import cycle anywhere. No behavior change (verbatim moves; all-private).
Verified: 410 non-visual tests pass; ruff + ruff-format clean. Pre-existing
#703/#705 mypy debt in utils.py is unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #714 (show() decomposition) into the utils.py split. Only utils.py
conflicted:
- import block: dropped the now-unused `_locate_value` import (it moved to
_color.py with the color code that uses it); kept main's `_locate_value`
out of utils.
- `_fast_extent` docstring: took main's #714 version (D205 fix).
basic.py auto-merged: #714's decomposed show()/helpers now import color and
validation symbols from _color/_validate (the split's repoints), not utils.
Bonus: merging #714 brings its fixes for the pre-existing #703/#705 debt
(_resolve_measure_table str-return, _get_extent_fast Any-return, _fast_extent
D205), so the branch is now fully ruff + mypy clean (no --no-verify).
Verified: no import cycle; ruff + ruff-format + mypy all pass; 410 non-visual
tests pass.
@timtreis
timtreis deleted the feat/centroid-scatter-helper branch July 10, 2026 11:11
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@timtreis@codecov-commenter
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent - #703

Merged
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper
Jun 14, 2026
Merged

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent#703
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper

Conversation

@timtreis

@timtreistimtreis commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

Fast rendering for shapes/labels, in spatialdata-plot's API:

  1. as_points=True — draw each cell as a dot at its centroid instead of its full geometry/mask (the squidpy spatial_scatter idea). matplotlib by default; auto-switches to datashader above ~500k dots, or with method="datashader".
  2. Fast axis-aligned extentpl.show() skips transforming every geometry to size the axes when the element's transform is axis-aligned (falls back to get_extent otherwise).
sdata.pl.render_shapes("cells", color="cell_type", as_points=True).pl.show()
sdata.pl.render_labels("cells", color="leiden", as_points=True, method="datashader").pl.show()

Notes

  • Default (as_points=False) output is unchanged vs main.
  • render_points' datashader pipeline is extracted into a shared _datashader_points (byte-identical) and reused by the centroid path.
  • Datashader can't represent the random-per-cell colours of uncolored labels, so that case stays matplotlib (with a warning).
  • Centroids are transformed to coordinate-system coords, fixing dot placement under non-identity transforms.

Tests

Position parity vs matplotlib (incl. non-identity transform), backend selection, no-color fallback, and test_plot_* visual baselines for matplotlib + datashader as_points (continuous, categorical, no-color).

… helper
Infrastructure for an upcoming "render cells as centroid points" fast mode
(no user-facing render option yet).
Phase 0 — shared scatter primitive:
- Extract `_scatter_points(ax, x, y, color_vector, ...)` from `_render_points`'s
matplotlib branch; `_render_points` now calls it. Byte-identical output
(verified vs main on categorical and continuous point renders). This is the
reuse seam the fast mode will draw through.
Phase 1 — centroid + caching core (headless, fully unit-tested):
- `_compute_element_centroids` / `_compute_label_centroids`: per-instance
centroids in a coordinate system. Shapes use spatialdata's vectorized
`get_centroids`; labels use skimage `regionprops` (the per-label reduction is
orders of magnitude faster than `get_centroids` on rasters), mapped onto the
raster's intrinsic coordinate arrays so it reproduces `get_centroids` exactly
(incl. the pixel-center 0.5 offset) then transformed to the target CS.
- `_get_or_compute_centroids`: reuses/persists centroids via the squidpy
convention. A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted
as the cells' locations; otherwise centroids are computed and written back into
the annotating table's `obsm["spatial"]` with a coordinate-system provenance
marker in `uns`, so later renders are instant. Reads run before writes, so a
valid existing cache is reused rather than clobbered; an incompatible existing
`obsm["spatial"]` is never overwritten; the cache is invalidated when the
requested coordinate system differs.
- Tests: shapes/labels centroids match `get_centroids`; cache round-trip +
provenance; CS invalidation; pre-existing obsm trusted; no-table compute path;
`cache=False` writes nothing.
… provenance
Cleanup from /simplify (no behavioral change):
- Extract `_region_mask_and_keys(table, element)` used by both read and write,
removing the duplicated `get_table_keys` + O(n_obs) `region_key`-string-cast
mask that was computed twice per cold call.
- Read path: validate shape on the raw obsm array and cast only the masked
subset to float, instead of casting the whole `obsm["spatial"]` on every
cache hit (the hot path).
- Write path: coerce a non-dict `uns["spatialdata_plot"]` instead of early
returning after `obsm` was already mutated, so obsm and the provenance marker
are always written together (no half-write).
- Drop the dead `"key"` provenance field (constant, never read back).
- Rename the misleading `table` local (held a table *name*) in
`_get_or_compute_centroids`.
Refactor the centroid cache to store element-*intrinsic* coordinates and
transform to the render coordinate system on demand, instead of caching
coords already mapped into one coordinate system. Decisions from design pass:
- Intrinsic storage: one `obsm["spatial"]` cache serves every coordinate
system (proven equivalent to per-CS computation). `_compute_element_centroids`
returns intrinsic coords (shapes via shapely `.centroid`, labels via
`regionprops`); `_centroids_to_coordinate_system` maps them to the requested
CS via the element's transform; `_get_or_compute_centroids` reads/computes
intrinsic then transforms on return.
- Provenance records `{n, scale_level}` (no coordinate system). Cache is
invalidated when the region's instance count changes (cells added/removed),
not on CS change.
- A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted as the
cells' intrinsic locations and transformed to the render CS.
- Exhaustive model dispatch: shapes and 2D labels supported; other element
types raise NotImplementedError.
- Labels are reduced at full resolution (scale0).
Drops the now-unused `get_centroids` import; adds `ShapesModel`. Tests updated:
parametrized shapes/labels match `get_centroids`; new coordinate-system-
independence test (one cache, two CS); staleness-by-instance-count; trusted
pre-existing obsm; unsupported-type rejection.
…e import, shared obsm gate
Cleanup from /simplify:
- `_centroids_to_coordinate_system` ran `PointsModel.parse` + a dask
`transform(...).compute()` round trip on every call, including cache hits
(~19 ms fixed floor, ~100 ms at 1M cells) — defeating the cache. Replace with
`to_affine_matrix` + a plain numpy matmul: numerically identical (verified
against the dask path for multiple coordinate systems), ~80-140x faster, and
it removes the private-API import `spatialdata._core.operations.transform`
(a repo non-negotiable) plus the now-unused `PointsModel`.
- Widen `_transformable_raster` -> `_transform_carrier` to accept any element
(rasters -> scale0, others as-is), dropping the `isinstance` branch in
`_centroids_to_coordinate_system`.
- Extract `_valid_spatial_obsm(arr, n_obs)` shared by the read and write paths,
reconciling their previously divergent obsm-shape checks (read accepted >=2
columns, write required exactly 2) so they cannot drift.
`render_shapes(..., as_points=True)` and `render_labels(..., as_points=True)`
draw one dot per cell at its centroid instead of the full geometry / rasterized
mask — a large speedup when only cell location matters. New `size=` controls the
marker size.
- Shared `_render_centroids_as_points` draws the scatter (via `_scatter_points`)
and the legend/colorbar. The per-cell color vector is the *same* one the
geometry/raster path computes (`_set_color_source_vec`), so colors match the
full rendering exactly; only the apply step (scatter vs patches/imshow) differs.
- Shapes: centroids from shapely `.centroid` of the (filtered) geometry,
positionally aligned to the color vector, drawn in intrinsic coords via the
element transform. Labels: centroids from `_get_or_compute_centroids`
(regionprops, fast) reindexed to `instance_id`. Positions verified identical to
`sd.get_centroids`.
- `as_points` short-circuits before the geometry/raster path; outline_*, shape
(shapes) and contour_px, outline_* (labels) are ignored with an info log.
- Default (`as_points=False`) output is byte-identical to main.
Tests: non-visual checks that centroids land exactly on `get_centroids` for both
element types and that outline/shape are ignored without error.
Note: as_points currently always uses the matplotlib scatter backend; routing
through datashader for very large cell counts (and persisting the obsm cache to
the user's object rather than show()'s working copy) are follow-ups.
@timtreistimtreis changed the title Centroid extraction + squidpy obsm["spatial"] caching; shared scatter helperFast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingJun 8, 2026
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.08046% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.89%. Comparing base (b370b1f) to head (882d208).

Files with missing linesPatch %Lines
src/spatialdata_plot/pl/utils.py81.81%6 Missing and 6 partials ⚠️
src/spatialdata_plot/pl/render.py92.70%4 Missing and 3 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #703 +/- ##
==========================================
+ Coverage 76.28% 76.89% +0.60% 
==========================================
Files 14 14 Lines 4327 4458 +131 Branches 1006 1035 +29 ==========================================
+ Hits 3301 3428 +127 + Misses 667 664 -3 - Partials 359 366 +7 
Files with missing linesCoverage Δ
src/spatialdata_plot/pl/_datashader.py90.50% <100.00%> (ø)
src/spatialdata_plot/pl/basic.py79.61% <100.00%> (+0.19%)⬆️
src/spatialdata_plot/pl/render_params.py89.02% <100.00%> (+0.22%)⬆️
src/spatialdata_plot/pl/render.py88.16% <92.70%> (+1.11%)⬆️
src/spatialdata_plot/pl/utils.py69.40% <81.81%> (+0.51%)⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`render_labels(element, as_points=True)` with no color crashed:
`instance_id` (the raster's unique values) includes the background label `0`,
which has no centroid, and the literal/no-color color vector is sized to the
raster (not per-instance), so `ax.scatter` got mismatched `c` vs `x`/`y`.
Drop the background label from the rendered instances and align the per-cell
color: for data-driven color the vector is already per-instance and is subset to
match; for the literal/no-color path it is replaced with one na/literal color
per centroid. Data-driven (categorical/continuous) renders are unchanged and
still land exactly on `get_centroids`.
Adds a regression test for the no-color labels case.
…ator
Replace the `regionprops` reduction in `_compute_label_centroids` with an
additive bincount aggregator that streams the labels raster block by block —
one dask chunk (or bounded numpy row-block) in memory at a time — accumulating
per-label `count`/`sum_x`/`sum_y`. This is what makes the feature usable at
Xenium scale:
- Out-of-core: peak memory is one chunk + O(n_labels) accumulators, NOT the
whole raster (measured: 9 MB peak streaming a 268 MB mask). `regionprops`
needs the full array materialized and OOMs on large morphology masks.
- Scales in cell count: 500k+ labels are just array indexing (562k labels in
~1.4 s with 13.5 MB of accumulators); `regionprops`' per-label table does not.
- Faster than `regionprops` (~1.3-1.6x) on in-memory rasters.
- Exact across chunk boundaries (additive reduction) — verified numpy ==
dask-chunked, and identical to `sd.get_centroids`.
- `count` is the cell area, a free by-product (ready for footprint-based dot
sizing).
Drops the `regionprops_table` import; adds `slices_from_chunks`. Adds a unit
test locking the chunk-exact, out-of-core, area-correct behavior.
Note: the chunk loop is currently sequential; parallelizing the per-chunk
partials (dask map_blocks + tree-reduce) is a future speedup.
#705 (measure_obs) merged to main, superseding #703's pre-extraction centroid
block. This merge takes main's utils.py + test_utils.py wholesale (zero #703
delta there) and rewires the labels as_points path onto main's primitive:
- render.py labels branch: drop the deleted `_get_or_compute_centroids` + the
whole cache layer; compute full-resolution (scale0) centroids via main's
`_compute_element_measurements`, and draw them with a transform built from the
*same* scale0 element (`_prepare_transformation(_get_top_data_array(...))`) —
so positions are independent of any rasterization applied to the rendered
`label`. Coerce point_ids to the label dtype so str/object instance ids
(e.g. Xenium readers) align instead of silently reindexing to NaN.
- Net: utils.py == main (the 242-line #703 centroid block + cache layer is
gone); the surviving diff vs main is just the as_points feature.
- Tests: added a non-identity-transform regression test asserting the dots land
at the cells' coordinate-system positions in display space (the guard for the
transform pairing); existing as_points position tests still pass.
Shapes branch unchanged (already intrinsic centroid + trans_data, positionally
aligned to its post-filter color vector).
…ked fields
The fast-mode draw wrapper read cmap/size/alpha/zorder/colorbar/colorbar_params
straight off render_params at both call sites. Pass the render_params dataclass
instead and read them internally; only the genuinely per-branch values
(x/y/color vectors, norm, na_color, transform, adata, palette, col_for_color)
stay explicit. Signature 20->15 params; both as_points call sites lose the
restated render_params plumbing.
Collapse the per-centroid color alignment (drop the unconditional asarray
pre-init and the nested None-guard into a single if/else + ternary), and drop a
redundant np.asarray around the already-ndarray point_ids in the reindex.
Behavior unchanged.
Benchmark showed labels as_points was 16-34x SLOWER than the normal render
because it recomputed full-resolution scale0 centroids while the normal path
downsamples + imshows. Compute centroids on the already-rendered (downsampled)
raster and draw with its trans_data instead: 671k cells goes 19.7s -> 0.88s
(now ~parity with imshow). Centroid error is sub-pixel at display resolution;
position tests updated to display-space within a few-px tolerance.
…gned
show()'s get_extent(exact=True) transforms EVERY shapes/points geometry into
the coordinate system just to take a bounding box - the dominant cost for large
shape collections (~85% of a 5.5M-shape render), unrelated to what is drawn.
Add a self-contained get_extent_fast (drop-in for spatialdata.get_extent) that,
for axis-aligned transforms (scale/flip/90deg/swap + translation - all real
Visium/Xenium data), transforms only the 4 bounding-box corners and reads the
intrinsic bounds vectorised (no per-geometry .apply(is_empty)). Proven identical
to the exact extent for such transforms (spatialdata's own get_extent docstring
notes this); falls back to spatialdata's get_extent for rotation/shear, for
anisotropically-scaled circles (radius->ellipse divergence), and for
images/labels (whose get_extent is already a cheap corner transform).
Measured on real Visium HD render_shapes(as_points=True): 351k 8.5s->1.8s
(4.7x), 5.48M 129s->24s (5.5x); the full (non-as_points) render benefits too.
The whole block is isolated so it can be lifted into spatialdata's get_extent
verbatim (see issue #706). Tests assert it matches get_extent across scale/flip/
rotation/shear for circles and polygons.
@timtreis
timtreisforce-pushed the feat/centroid-scatter-helper branch from 35c8f1f to 57bcf8dCompareJune 9, 2026 23:20
…ompute, underscore)
Follow-up cleanups from review of the get_extent fast path:
- fold _intrinsic_xy_bounds into _element_extent_fast (drop duplicate get_model / geom_type)
- geom.bounds -> geom.total_bounds (C-level union, avoids an Nx4 alloc on large collections)
- batch the four points min/max into one dask.compute
- rename get_extent_fast -> _get_extent_fast (internal helper; matches sibling underscored helpers)
No behavior change; output identical for axis-aligned and rotated/sheared elements.
The datashader shapes canvas sized itself via spatialdata's exact get_extent, transforming every
geometry (O(N)) just for a bounding box -- the same cost _get_extent_fast already removed from
show()'s axis limits, in a second code path. Reuse _element_extent_fast (corner transform for
axis-aligned elements; None -> exact get_extent fallback for rotation/shear), so the result is
pixel-identical and the per-geometry pass is skipped for the common case.
Measured on Curio (69,713 colored shapes): render_shapes 3146 ms -> 1675 ms (1.88x), on top of
the get_extent_fast win already in show(). Mirrors _datashader_canvas_from_dataframe, which
already avoids get_extent for the points path.
…de review)
From the perf-stack review of labels as_points:
- No color column collapsed every dot to a single na_color; now each cell gets a distinct random
colour, matching the mask path's _map_color_seg Case C. Adds a color assertion to the regression test.
- The rasterize drop-filter only ran when a color column was set, so as_points could emit dots at NaN
positions (or drop cells) when rasterization removed labels; extend it to as_points so point ids stay
within the rendered raster.
- ax.scatter autoscales the Normalize in place; copy the shared cmap_params.norm (the shapes path already does).
- render_shapes/render_labels(as_points=True) did not validate size; add the points-path numeric/positive
check so size=-5 / 'big' raise an actionable error instead of a raw matplotlib failure.
Extract the duplicated as_points size-validation block from render_shapes
and render_labels into a shared _validate_as_points_size helper. Defer the
np.asarray(color_vector) conversion in the centroid color path to the only
branch that uses it.
…to feat/centroid-scatter-helper
# Conflicts:
#	tests/pl/test_utils.py
_get_extent_fast already reads get_transformation(element, get_all=True) for
the coordinate-system membership check; pass it through so the fast path does
not re-fetch it per element. The optional kwarg leaves the datashader-canvas
call site unchanged.
_element_extent_fast returned a NaN extent for an element whose geometries are
all empty, which silently poisoned the union in _get_extent_fast (blank axes)
instead of raising spatialdata's clear 'empty collection' error. Guard against
non-finite bounds and fall back. Also trims the verbose extent-block comment and
the scatter/centroid docstrings (net -12 LOC, no behavior change).
@timtreistimtreis changed the title Fast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingFast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extentJun 10, 2026
Four PlotTester baselines: render_shapes/labels(as_points=True) at a base size
and a larger size (size = scatter marker area). Baselines to be generated from CI.
…l tests
Rendered on hatch-test.py3.11-stable; verified dots land at shape/label
centroids, colored by instance_id (labels), larger at size=600.
The empty-shape check ran a per-geometry Python lambda (`.apply(lambda g: not
g.is_empty)`) over every geometry on every shapes render; GeoSeries.is_empty is
GEOS-vectorized. `.is_empty.all()` is identical and ~134x faster on this guard
(665ms -> 5ms at 351k shapes, ~10s -> 80ms at 5.5M).
Shared datashader draw primitive (canvas->aggregate->norm->color-key->shade->
image->colorbar), taking explicit primitives instead of render_params so it can
also serve the as_points centroid path (shapes/labels params use fill_alpha and
lack the density fields). Returns the possibly-recomputed color vectors so the
caller's legend/colorbar stays identical. Verified pixel-identical on
render_points datashader (categorical/continuous/plain/density).
- Route as_points centroids through the shared _datashader_points when method=
'datashader' or above ~500k dots (AS_POINTS_DS_AUTO); matplotlib otherwise.
- No-color labels (one random colour per cell) cannot be aggregated by datashader;
force matplotlib there (warn on explicit method='datashader').
- Fix: shapes as_points drew dots at intrinsic centroid coords while the axes are
in coordinate-system coords, so non-identity transforms misplaced them. Transform
centroids to CS via the element->CS affine for both backends (labels too).
- render_labels gains 'method'; LabelsRenderParams gains 'method'. as_points uses a
fixed datashader reduction ('max', closest to matplotlib) — no user knob.
- shapes non-identity transform regression (the coordinate bug fixed in 5854ae6)
- backend selection: method='datashader' -> datashader image; default -> matplotlib
- no-color labels force matplotlib even with method='datashader' (warns)
- _resolve_as_points_method unit test (threshold/explicit/no-color/empty)
- two test_plot_* visual tests for datashader as_points (baselines from CI)
timtreis added 22 commits June 11, 2026 06:40
Rendered on hatch-test.py3.11-stable; verified datashaded centroids coloured by
category with a categorical legend. Only this test failed on stable CI (missing
baseline); existing datashader baselines unchanged.
_datashader_points' two near-identical reindex/align branches (categorical
source vs colour vector) differed only in which vector they materialize; pick
the source once, then share the reindex/construct. render_points datashader
output verified byte-identical (categorical/continuous/plain/density). -9 LOC.
The datashader as_points canvas was sized to the exact centroid bounding box,
so the outermost centroids sat on the canvas edge and their marker spread was
clipped (half-circles), looking nothing like the matplotlib backend. Grow the
canvas by the spread radius on each side (factor unchanged, so placement still
aligns). render_points keeps pad_for_markers=False -> byte-identical.
…padding fix
The padding fix changed the datashader as_points renders within the comparison
tolerance, so the baselines didn't auto-update. Regenerate from CI so the
committed images reflect the un-clipped output.
CI-rendered (py3.11-stable) after the canvas-padding fix; verified the edge
centroids now render as full circles matching the matplotlib backend.
Restructure the as_points visual tests into matplotlib+datashader pairs that
render identical params (shared helper) for shapes (no color), labels
(instance_id), and labels (categorical). Drop the old inconsistent/stale
baselines; all 6 will be regenerated from CI so the two backends are directly
comparable and look maximally similar.
Positions and sizes align after the canvas-padding fix; dots land at the same
centroids at the same size. Remaining difference is color shading on the
categorical path (datashader modulates alpha by per-pixel count).
Datashader faded single-cell dots (count-driven alpha floor + a second user-alpha
multiply ~= alpha^2), making categorical as_points read much paler than the
matplotlib backend. Add uniform_alpha for the as_points marker mode: a full alpha
floor so each dot is one flat colour at fill_alpha, like a matplotlib marker.
Renamed the as_points flag pad_for_markers -> as_markers (pads canvas + uniform
alpha). render_points unchanged (byte-identical).
…h matplotlib)
CI-rendered (py3.11-stable); the datashader as_points dots now match the
matplotlib markers in colour saturation, position, and size.
…matplotlib
Size (derived, no heuristic): datashader previously rasterized over the centroid
bounding box, so dot display size depended on the canvas/axes extent ratio (shapes
~0.9x, labels ~1.7x matplotlib). Now the canvas spans the same extent as the axes
(like render_points), and the spread radius is set to matplotlib's marker radius
sqrt(s)*dpi/144 (its 'o' marker has diameter sqrt(s)*dpi/72). Result: ds/mpl size
ratio 1.02 +/- 0.03 across element type, size, figure, and dpi.
Continuous colorbar: the spread combined overlapping dots with 'add' (ds_reduction
None -> 'sum'), summing ids and inflating reduction_bounds; marker mode now spreads
with 'max' so overlaps overlay and the colorbar keeps the true range.
render_points unchanged (byte-identical). Replaces the canvas-padding workaround
(the axes-extent canvas already gives edge dots their margin).
… colorbar)
CI-rendered (py3.11-stable): datashader dots now match the matplotlib markers in
size, and the continuous colorbar uses the true value range.
The datashader result is a data-coordinate image that scales with the axes,
while matplotlib markers are fixed in display points. The canvas was sized from
fig.get_size_inches()*dpi (the whole figure), but the axes are smaller (margins,
colorbar), so the image - and every dot - was scaled down: labels (with colorbar)
0.81x matplotlib, shapes 0.88x. Size the as_markers canvas to the axes display box
(ax.get_window_extent()) so 1 canvas px == 1 axes-display px and the sqrt(s)*dpi/144
spread radius matches the marker. Now mean 0.99 +/-5% across sizes/figs/dpi/elements;
render_points untouched (byte-identical). Visual tests render at a non-overlapping
size so the engines' overlap handling (stack vs aggregate) doesn't enter the pairs.
…der canvas
CI-rendered (py3.11-stable). matplotlib and datashader pairs now match in dot
size (the canvas-vs-figure scaling bug is fixed) at a non-overlapping size.
Cleanup pass over the as_points feature (no behavior change):
- trim restated comments/docstrings in _datashader_points, keeping the
load-bearing marker-radius (sqrt(s)*dpi/144) and canvas-vs-axes rationale
- collapse the 3-way min_alpha if/elif/else to a ternary
- reuse the already-computed label extent instead of recomputing get_extent
- np.full instead of list*N for the literal na_color vector
- tighten the as_points test-helper comment
render_points output verified byte-identical; as_points renders unchanged.
Datashader pays off for as_points from ~50k cells (~1.66x faster than the
matplotlib scatter backend, rising to ~1.8x at 1M), so switch over there
instead of 500k. Users can still force either backend via method=.
The shapes as_points block computed its datashader canvas extent via
get_extent (exact=True), which transforms every geometry — on 351k Visium HD
shapes that was ~7s/render and dominated the whole as_points cost, so the fast
mode gave no speedup over full-geometry rendering. Use _element_extent_fast
(corner-transform; identical result for axis-aligned transforms, falls back to
get_extent for rotation/shear), which the PR already added for exactly this.
Real overlay (image + shapes color=gene), as_points:
91k: 7.2s -> 2.0s (3.3x)
351k: 24.3s -> 3.1s (7.3x, vs 22.8s full-geometry)
Output is unchanged: the fast extent is byte-identical for axis-aligned data.
- extract _fast_extent(element, cs) helper; the '_element_extent_fast(...) or
get_extent(...)' idiom was duplicated verbatim at the shapes as_points site
and the datashader-canvas helper
- fix docstrings that still said '~500k' after the threshold dropped to 50k
- trim _render_centroids_as_points docstring and a non-actionable comment
No behavior change: _fast_extent is identical to the inline form (verified
== get_extent on blobs), render_points byte-identical.
@timtreis
timtreis merged commit 8730ff4 into mainJun 14, 2026
7 of 8 checks passed
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #702 (per-panel title fix) and #703 (as_points + fast extent)
into the show() decomposition.
Conflict resolution in src/spatialdata_plot/pl/basic.py:
- #702 up-front title-count validation: kept (after num_panels). The
adjacent axes/panel-count check was dropped here because the
decomposition already relocated it into _plan_panels.
- #702 simplified title selection (dropped the per-panel try/except):
applied to the extracted _finalize_panel helper.
- #703 had no show()-level render-dispatch changes (as_points is
param-driven in render.py, handled inside _render_panel already); its
only show()-level change, get_extent -> _get_extent_fast, auto-merged
into the extent block and consumes _render_panel's `wants` dict.
Also sweeps up two pre-existing #703 lint/type nits in utils.py surfaced
by the merge: _fast_extent docstring (ruff D205) and _get_extent_fast
Any-return (mypy).
Verified: ruff + mypy clean; 109 non-visual show/shapes/labels tests pass.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 21 color helpers + 6 color-only format/uniqueness helpers verbatim from
utils.py into a new sibling module pl/_color.py. Imports flow one way:
_color -> utils (downward, for _get_list/to_hex/_build_alignment_dtype_hint/
_MPL_SINGLE_LETTER_COLORS). render.py, basic.py and _datashader.py are repointed
to import color symbols from _color.
The two validators still in utils (_type_check_params, _validate_graph_render_params)
use color symbols; they carry temporary function-local imports of _color (cycle-safe)
until they move to _validate.py in the next commit, where these become top-level imports.
No behavior change (verbatim move; all-private except set_zero_in_cmap_to_transparent,
which is not re-exported). Verified: no import cycle; 410 non-visual tests pass;
ruff + ruff-format clean. Pre-existing #703/#705 mypy/ruff debt in utils.py is
unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 17 validation/type-check functions verbatim from utils.py into a new sibling
module pl/_validate.py. Imports flow one way: _validate -> _color (_is_color_like,
_prepare_cmap_norm, _get_colors_for_categorical_obs, now top-level) and
_validate -> utils (downward). The temporary function-local _color imports added in
the previous commit are hoisted to top-level here and removed from utils.
render.py and basic.py repointed to _validate.
Completes the utils.py split (#696): utils 4918 -> 1311 lines, with
_geometry / _datashader / _color / _validate as single-concern siblings.
No import cycle anywhere. No behavior change (verbatim moves; all-private).
Verified: 410 non-visual tests pass; ruff + ruff-format clean. Pre-existing
#703/#705 mypy debt in utils.py is unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #714 (show() decomposition) into the utils.py split. Only utils.py
conflicted:
- import block: dropped the now-unused `_locate_value` import (it moved to
_color.py with the color code that uses it); kept main's `_locate_value`
out of utils.
- `_fast_extent` docstring: took main's #714 version (D205 fix).
basic.py auto-merged: #714's decomposed show()/helpers now import color and
validation symbols from _color/_validate (the split's repoints), not utils.
Bonus: merging #714 brings its fixes for the pre-existing #703/#705 debt
(_resolve_measure_table str-return, _get_extent_fast Any-return, _fast_extent
D205), so the branch is now fully ruff + mypy clean (no --no-verify).
Verified: no import cycle; ruff + ruff-format + mypy all pass; 410 non-visual
tests pass.
@timtreis
timtreis deleted the feat/centroid-scatter-helper branch July 10, 2026 11:11
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@timtreis@codecov-commenter
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent - #703

Merged
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper
Jun 14, 2026
Merged

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent#703
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper

Conversation

@timtreis

@timtreistimtreis commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

Fast rendering for shapes/labels, in spatialdata-plot's API:

  1. as_points=True — draw each cell as a dot at its centroid instead of its full geometry/mask (the squidpy spatial_scatter idea). matplotlib by default; auto-switches to datashader above ~500k dots, or with method="datashader".
  2. Fast axis-aligned extentpl.show() skips transforming every geometry to size the axes when the element's transform is axis-aligned (falls back to get_extent otherwise).
sdata.pl.render_shapes("cells", color="cell_type", as_points=True).pl.show()
sdata.pl.render_labels("cells", color="leiden", as_points=True, method="datashader").pl.show()

Notes

  • Default (as_points=False) output is unchanged vs main.
  • render_points' datashader pipeline is extracted into a shared _datashader_points (byte-identical) and reused by the centroid path.
  • Datashader can't represent the random-per-cell colours of uncolored labels, so that case stays matplotlib (with a warning).
  • Centroids are transformed to coordinate-system coords, fixing dot placement under non-identity transforms.

Tests

Position parity vs matplotlib (incl. non-identity transform), backend selection, no-color fallback, and test_plot_* visual baselines for matplotlib + datashader as_points (continuous, categorical, no-color).

… helper
Infrastructure for an upcoming "render cells as centroid points" fast mode
(no user-facing render option yet).
Phase 0 — shared scatter primitive:
- Extract `_scatter_points(ax, x, y, color_vector, ...)` from `_render_points`'s
matplotlib branch; `_render_points` now calls it. Byte-identical output
(verified vs main on categorical and continuous point renders). This is the
reuse seam the fast mode will draw through.
Phase 1 — centroid + caching core (headless, fully unit-tested):
- `_compute_element_centroids` / `_compute_label_centroids`: per-instance
centroids in a coordinate system. Shapes use spatialdata's vectorized
`get_centroids`; labels use skimage `regionprops` (the per-label reduction is
orders of magnitude faster than `get_centroids` on rasters), mapped onto the
raster's intrinsic coordinate arrays so it reproduces `get_centroids` exactly
(incl. the pixel-center 0.5 offset) then transformed to the target CS.
- `_get_or_compute_centroids`: reuses/persists centroids via the squidpy
convention. A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted
as the cells' locations; otherwise centroids are computed and written back into
the annotating table's `obsm["spatial"]` with a coordinate-system provenance
marker in `uns`, so later renders are instant. Reads run before writes, so a
valid existing cache is reused rather than clobbered; an incompatible existing
`obsm["spatial"]` is never overwritten; the cache is invalidated when the
requested coordinate system differs.
- Tests: shapes/labels centroids match `get_centroids`; cache round-trip +
provenance; CS invalidation; pre-existing obsm trusted; no-table compute path;
`cache=False` writes nothing.
… provenance
Cleanup from /simplify (no behavioral change):
- Extract `_region_mask_and_keys(table, element)` used by both read and write,
removing the duplicated `get_table_keys` + O(n_obs) `region_key`-string-cast
mask that was computed twice per cold call.
- Read path: validate shape on the raw obsm array and cast only the masked
subset to float, instead of casting the whole `obsm["spatial"]` on every
cache hit (the hot path).
- Write path: coerce a non-dict `uns["spatialdata_plot"]` instead of early
returning after `obsm` was already mutated, so obsm and the provenance marker
are always written together (no half-write).
- Drop the dead `"key"` provenance field (constant, never read back).
- Rename the misleading `table` local (held a table *name*) in
`_get_or_compute_centroids`.
Refactor the centroid cache to store element-*intrinsic* coordinates and
transform to the render coordinate system on demand, instead of caching
coords already mapped into one coordinate system. Decisions from design pass:
- Intrinsic storage: one `obsm["spatial"]` cache serves every coordinate
system (proven equivalent to per-CS computation). `_compute_element_centroids`
returns intrinsic coords (shapes via shapely `.centroid`, labels via
`regionprops`); `_centroids_to_coordinate_system` maps them to the requested
CS via the element's transform; `_get_or_compute_centroids` reads/computes
intrinsic then transforms on return.
- Provenance records `{n, scale_level}` (no coordinate system). Cache is
invalidated when the region's instance count changes (cells added/removed),
not on CS change.
- A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted as the
cells' intrinsic locations and transformed to the render CS.
- Exhaustive model dispatch: shapes and 2D labels supported; other element
types raise NotImplementedError.
- Labels are reduced at full resolution (scale0).
Drops the now-unused `get_centroids` import; adds `ShapesModel`. Tests updated:
parametrized shapes/labels match `get_centroids`; new coordinate-system-
independence test (one cache, two CS); staleness-by-instance-count; trusted
pre-existing obsm; unsupported-type rejection.
…e import, shared obsm gate
Cleanup from /simplify:
- `_centroids_to_coordinate_system` ran `PointsModel.parse` + a dask
`transform(...).compute()` round trip on every call, including cache hits
(~19 ms fixed floor, ~100 ms at 1M cells) — defeating the cache. Replace with
`to_affine_matrix` + a plain numpy matmul: numerically identical (verified
against the dask path for multiple coordinate systems), ~80-140x faster, and
it removes the private-API import `spatialdata._core.operations.transform`
(a repo non-negotiable) plus the now-unused `PointsModel`.
- Widen `_transformable_raster` -> `_transform_carrier` to accept any element
(rasters -> scale0, others as-is), dropping the `isinstance` branch in
`_centroids_to_coordinate_system`.
- Extract `_valid_spatial_obsm(arr, n_obs)` shared by the read and write paths,
reconciling their previously divergent obsm-shape checks (read accepted >=2
columns, write required exactly 2) so they cannot drift.
`render_shapes(..., as_points=True)` and `render_labels(..., as_points=True)`
draw one dot per cell at its centroid instead of the full geometry / rasterized
mask — a large speedup when only cell location matters. New `size=` controls the
marker size.
- Shared `_render_centroids_as_points` draws the scatter (via `_scatter_points`)
and the legend/colorbar. The per-cell color vector is the *same* one the
geometry/raster path computes (`_set_color_source_vec`), so colors match the
full rendering exactly; only the apply step (scatter vs patches/imshow) differs.
- Shapes: centroids from shapely `.centroid` of the (filtered) geometry,
positionally aligned to the color vector, drawn in intrinsic coords via the
element transform. Labels: centroids from `_get_or_compute_centroids`
(regionprops, fast) reindexed to `instance_id`. Positions verified identical to
`sd.get_centroids`.
- `as_points` short-circuits before the geometry/raster path; outline_*, shape
(shapes) and contour_px, outline_* (labels) are ignored with an info log.
- Default (`as_points=False`) output is byte-identical to main.
Tests: non-visual checks that centroids land exactly on `get_centroids` for both
element types and that outline/shape are ignored without error.
Note: as_points currently always uses the matplotlib scatter backend; routing
through datashader for very large cell counts (and persisting the obsm cache to
the user's object rather than show()'s working copy) are follow-ups.
@timtreistimtreis changed the title Centroid extraction + squidpy obsm["spatial"] caching; shared scatter helperFast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingJun 8, 2026
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.08046% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.89%. Comparing base (b370b1f) to head (882d208).

Files with missing linesPatch %Lines
src/spatialdata_plot/pl/utils.py81.81%6 Missing and 6 partials ⚠️
src/spatialdata_plot/pl/render.py92.70%4 Missing and 3 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #703 +/- ##
==========================================
+ Coverage 76.28% 76.89% +0.60% 
==========================================
Files 14 14 Lines 4327 4458 +131 Branches 1006 1035 +29 ==========================================
+ Hits 3301 3428 +127 + Misses 667 664 -3 - Partials 359 366 +7 
Files with missing linesCoverage Δ
src/spatialdata_plot/pl/_datashader.py90.50% <100.00%> (ø)
src/spatialdata_plot/pl/basic.py79.61% <100.00%> (+0.19%)⬆️
src/spatialdata_plot/pl/render_params.py89.02% <100.00%> (+0.22%)⬆️
src/spatialdata_plot/pl/render.py88.16% <92.70%> (+1.11%)⬆️
src/spatialdata_plot/pl/utils.py69.40% <81.81%> (+0.51%)⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`render_labels(element, as_points=True)` with no color crashed:
`instance_id` (the raster's unique values) includes the background label `0`,
which has no centroid, and the literal/no-color color vector is sized to the
raster (not per-instance), so `ax.scatter` got mismatched `c` vs `x`/`y`.
Drop the background label from the rendered instances and align the per-cell
color: for data-driven color the vector is already per-instance and is subset to
match; for the literal/no-color path it is replaced with one na/literal color
per centroid. Data-driven (categorical/continuous) renders are unchanged and
still land exactly on `get_centroids`.
Adds a regression test for the no-color labels case.
…ator
Replace the `regionprops` reduction in `_compute_label_centroids` with an
additive bincount aggregator that streams the labels raster block by block —
one dask chunk (or bounded numpy row-block) in memory at a time — accumulating
per-label `count`/`sum_x`/`sum_y`. This is what makes the feature usable at
Xenium scale:
- Out-of-core: peak memory is one chunk + O(n_labels) accumulators, NOT the
whole raster (measured: 9 MB peak streaming a 268 MB mask). `regionprops`
needs the full array materialized and OOMs on large morphology masks.
- Scales in cell count: 500k+ labels are just array indexing (562k labels in
~1.4 s with 13.5 MB of accumulators); `regionprops`' per-label table does not.
- Faster than `regionprops` (~1.3-1.6x) on in-memory rasters.
- Exact across chunk boundaries (additive reduction) — verified numpy ==
dask-chunked, and identical to `sd.get_centroids`.
- `count` is the cell area, a free by-product (ready for footprint-based dot
sizing).
Drops the `regionprops_table` import; adds `slices_from_chunks`. Adds a unit
test locking the chunk-exact, out-of-core, area-correct behavior.
Note: the chunk loop is currently sequential; parallelizing the per-chunk
partials (dask map_blocks + tree-reduce) is a future speedup.
#705 (measure_obs) merged to main, superseding #703's pre-extraction centroid
block. This merge takes main's utils.py + test_utils.py wholesale (zero #703
delta there) and rewires the labels as_points path onto main's primitive:
- render.py labels branch: drop the deleted `_get_or_compute_centroids` + the
whole cache layer; compute full-resolution (scale0) centroids via main's
`_compute_element_measurements`, and draw them with a transform built from the
*same* scale0 element (`_prepare_transformation(_get_top_data_array(...))`) —
so positions are independent of any rasterization applied to the rendered
`label`. Coerce point_ids to the label dtype so str/object instance ids
(e.g. Xenium readers) align instead of silently reindexing to NaN.
- Net: utils.py == main (the 242-line #703 centroid block + cache layer is
gone); the surviving diff vs main is just the as_points feature.
- Tests: added a non-identity-transform regression test asserting the dots land
at the cells' coordinate-system positions in display space (the guard for the
transform pairing); existing as_points position tests still pass.
Shapes branch unchanged (already intrinsic centroid + trans_data, positionally
aligned to its post-filter color vector).
…ked fields
The fast-mode draw wrapper read cmap/size/alpha/zorder/colorbar/colorbar_params
straight off render_params at both call sites. Pass the render_params dataclass
instead and read them internally; only the genuinely per-branch values
(x/y/color vectors, norm, na_color, transform, adata, palette, col_for_color)
stay explicit. Signature 20->15 params; both as_points call sites lose the
restated render_params plumbing.
Collapse the per-centroid color alignment (drop the unconditional asarray
pre-init and the nested None-guard into a single if/else + ternary), and drop a
redundant np.asarray around the already-ndarray point_ids in the reindex.
Behavior unchanged.
Benchmark showed labels as_points was 16-34x SLOWER than the normal render
because it recomputed full-resolution scale0 centroids while the normal path
downsamples + imshows. Compute centroids on the already-rendered (downsampled)
raster and draw with its trans_data instead: 671k cells goes 19.7s -> 0.88s
(now ~parity with imshow). Centroid error is sub-pixel at display resolution;
position tests updated to display-space within a few-px tolerance.
…gned
show()'s get_extent(exact=True) transforms EVERY shapes/points geometry into
the coordinate system just to take a bounding box - the dominant cost for large
shape collections (~85% of a 5.5M-shape render), unrelated to what is drawn.
Add a self-contained get_extent_fast (drop-in for spatialdata.get_extent) that,
for axis-aligned transforms (scale/flip/90deg/swap + translation - all real
Visium/Xenium data), transforms only the 4 bounding-box corners and reads the
intrinsic bounds vectorised (no per-geometry .apply(is_empty)). Proven identical
to the exact extent for such transforms (spatialdata's own get_extent docstring
notes this); falls back to spatialdata's get_extent for rotation/shear, for
anisotropically-scaled circles (radius->ellipse divergence), and for
images/labels (whose get_extent is already a cheap corner transform).
Measured on real Visium HD render_shapes(as_points=True): 351k 8.5s->1.8s
(4.7x), 5.48M 129s->24s (5.5x); the full (non-as_points) render benefits too.
The whole block is isolated so it can be lifted into spatialdata's get_extent
verbatim (see issue #706). Tests assert it matches get_extent across scale/flip/
rotation/shear for circles and polygons.
@timtreis
timtreisforce-pushed the feat/centroid-scatter-helper branch from 35c8f1f to 57bcf8dCompareJune 9, 2026 23:20
…ompute, underscore)
Follow-up cleanups from review of the get_extent fast path:
- fold _intrinsic_xy_bounds into _element_extent_fast (drop duplicate get_model / geom_type)
- geom.bounds -> geom.total_bounds (C-level union, avoids an Nx4 alloc on large collections)
- batch the four points min/max into one dask.compute
- rename get_extent_fast -> _get_extent_fast (internal helper; matches sibling underscored helpers)
No behavior change; output identical for axis-aligned and rotated/sheared elements.
The datashader shapes canvas sized itself via spatialdata's exact get_extent, transforming every
geometry (O(N)) just for a bounding box -- the same cost _get_extent_fast already removed from
show()'s axis limits, in a second code path. Reuse _element_extent_fast (corner transform for
axis-aligned elements; None -> exact get_extent fallback for rotation/shear), so the result is
pixel-identical and the per-geometry pass is skipped for the common case.
Measured on Curio (69,713 colored shapes): render_shapes 3146 ms -> 1675 ms (1.88x), on top of
the get_extent_fast win already in show(). Mirrors _datashader_canvas_from_dataframe, which
already avoids get_extent for the points path.
…de review)
From the perf-stack review of labels as_points:
- No color column collapsed every dot to a single na_color; now each cell gets a distinct random
colour, matching the mask path's _map_color_seg Case C. Adds a color assertion to the regression test.
- The rasterize drop-filter only ran when a color column was set, so as_points could emit dots at NaN
positions (or drop cells) when rasterization removed labels; extend it to as_points so point ids stay
within the rendered raster.
- ax.scatter autoscales the Normalize in place; copy the shared cmap_params.norm (the shapes path already does).
- render_shapes/render_labels(as_points=True) did not validate size; add the points-path numeric/positive
check so size=-5 / 'big' raise an actionable error instead of a raw matplotlib failure.
Extract the duplicated as_points size-validation block from render_shapes
and render_labels into a shared _validate_as_points_size helper. Defer the
np.asarray(color_vector) conversion in the centroid color path to the only
branch that uses it.
…to feat/centroid-scatter-helper
# Conflicts:
#	tests/pl/test_utils.py
_get_extent_fast already reads get_transformation(element, get_all=True) for
the coordinate-system membership check; pass it through so the fast path does
not re-fetch it per element. The optional kwarg leaves the datashader-canvas
call site unchanged.
_element_extent_fast returned a NaN extent for an element whose geometries are
all empty, which silently poisoned the union in _get_extent_fast (blank axes)
instead of raising spatialdata's clear 'empty collection' error. Guard against
non-finite bounds and fall back. Also trims the verbose extent-block comment and
the scatter/centroid docstrings (net -12 LOC, no behavior change).
@timtreistimtreis changed the title Fast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingFast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extentJun 10, 2026
Four PlotTester baselines: render_shapes/labels(as_points=True) at a base size
and a larger size (size = scatter marker area). Baselines to be generated from CI.
…l tests
Rendered on hatch-test.py3.11-stable; verified dots land at shape/label
centroids, colored by instance_id (labels), larger at size=600.
The empty-shape check ran a per-geometry Python lambda (`.apply(lambda g: not
g.is_empty)`) over every geometry on every shapes render; GeoSeries.is_empty is
GEOS-vectorized. `.is_empty.all()` is identical and ~134x faster on this guard
(665ms -> 5ms at 351k shapes, ~10s -> 80ms at 5.5M).
Shared datashader draw primitive (canvas->aggregate->norm->color-key->shade->
image->colorbar), taking explicit primitives instead of render_params so it can
also serve the as_points centroid path (shapes/labels params use fill_alpha and
lack the density fields). Returns the possibly-recomputed color vectors so the
caller's legend/colorbar stays identical. Verified pixel-identical on
render_points datashader (categorical/continuous/plain/density).
- Route as_points centroids through the shared _datashader_points when method=
'datashader' or above ~500k dots (AS_POINTS_DS_AUTO); matplotlib otherwise.
- No-color labels (one random colour per cell) cannot be aggregated by datashader;
force matplotlib there (warn on explicit method='datashader').
- Fix: shapes as_points drew dots at intrinsic centroid coords while the axes are
in coordinate-system coords, so non-identity transforms misplaced them. Transform
centroids to CS via the element->CS affine for both backends (labels too).
- render_labels gains 'method'; LabelsRenderParams gains 'method'. as_points uses a
fixed datashader reduction ('max', closest to matplotlib) — no user knob.
- shapes non-identity transform regression (the coordinate bug fixed in 5854ae6)
- backend selection: method='datashader' -> datashader image; default -> matplotlib
- no-color labels force matplotlib even with method='datashader' (warns)
- _resolve_as_points_method unit test (threshold/explicit/no-color/empty)
- two test_plot_* visual tests for datashader as_points (baselines from CI)
timtreis added 22 commits June 11, 2026 06:40
Rendered on hatch-test.py3.11-stable; verified datashaded centroids coloured by
category with a categorical legend. Only this test failed on stable CI (missing
baseline); existing datashader baselines unchanged.
_datashader_points' two near-identical reindex/align branches (categorical
source vs colour vector) differed only in which vector they materialize; pick
the source once, then share the reindex/construct. render_points datashader
output verified byte-identical (categorical/continuous/plain/density). -9 LOC.
The datashader as_points canvas was sized to the exact centroid bounding box,
so the outermost centroids sat on the canvas edge and their marker spread was
clipped (half-circles), looking nothing like the matplotlib backend. Grow the
canvas by the spread radius on each side (factor unchanged, so placement still
aligns). render_points keeps pad_for_markers=False -> byte-identical.
…padding fix
The padding fix changed the datashader as_points renders within the comparison
tolerance, so the baselines didn't auto-update. Regenerate from CI so the
committed images reflect the un-clipped output.
CI-rendered (py3.11-stable) after the canvas-padding fix; verified the edge
centroids now render as full circles matching the matplotlib backend.
Restructure the as_points visual tests into matplotlib+datashader pairs that
render identical params (shared helper) for shapes (no color), labels
(instance_id), and labels (categorical). Drop the old inconsistent/stale
baselines; all 6 will be regenerated from CI so the two backends are directly
comparable and look maximally similar.
Positions and sizes align after the canvas-padding fix; dots land at the same
centroids at the same size. Remaining difference is color shading on the
categorical path (datashader modulates alpha by per-pixel count).
Datashader faded single-cell dots (count-driven alpha floor + a second user-alpha
multiply ~= alpha^2), making categorical as_points read much paler than the
matplotlib backend. Add uniform_alpha for the as_points marker mode: a full alpha
floor so each dot is one flat colour at fill_alpha, like a matplotlib marker.
Renamed the as_points flag pad_for_markers -> as_markers (pads canvas + uniform
alpha). render_points unchanged (byte-identical).
…h matplotlib)
CI-rendered (py3.11-stable); the datashader as_points dots now match the
matplotlib markers in colour saturation, position, and size.
…matplotlib
Size (derived, no heuristic): datashader previously rasterized over the centroid
bounding box, so dot display size depended on the canvas/axes extent ratio (shapes
~0.9x, labels ~1.7x matplotlib). Now the canvas spans the same extent as the axes
(like render_points), and the spread radius is set to matplotlib's marker radius
sqrt(s)*dpi/144 (its 'o' marker has diameter sqrt(s)*dpi/72). Result: ds/mpl size
ratio 1.02 +/- 0.03 across element type, size, figure, and dpi.
Continuous colorbar: the spread combined overlapping dots with 'add' (ds_reduction
None -> 'sum'), summing ids and inflating reduction_bounds; marker mode now spreads
with 'max' so overlaps overlay and the colorbar keeps the true range.
render_points unchanged (byte-identical). Replaces the canvas-padding workaround
(the axes-extent canvas already gives edge dots their margin).
… colorbar)
CI-rendered (py3.11-stable): datashader dots now match the matplotlib markers in
size, and the continuous colorbar uses the true value range.
The datashader result is a data-coordinate image that scales with the axes,
while matplotlib markers are fixed in display points. The canvas was sized from
fig.get_size_inches()*dpi (the whole figure), but the axes are smaller (margins,
colorbar), so the image - and every dot - was scaled down: labels (with colorbar)
0.81x matplotlib, shapes 0.88x. Size the as_markers canvas to the axes display box
(ax.get_window_extent()) so 1 canvas px == 1 axes-display px and the sqrt(s)*dpi/144
spread radius matches the marker. Now mean 0.99 +/-5% across sizes/figs/dpi/elements;
render_points untouched (byte-identical). Visual tests render at a non-overlapping
size so the engines' overlap handling (stack vs aggregate) doesn't enter the pairs.
…der canvas
CI-rendered (py3.11-stable). matplotlib and datashader pairs now match in dot
size (the canvas-vs-figure scaling bug is fixed) at a non-overlapping size.
Cleanup pass over the as_points feature (no behavior change):
- trim restated comments/docstrings in _datashader_points, keeping the
load-bearing marker-radius (sqrt(s)*dpi/144) and canvas-vs-axes rationale
- collapse the 3-way min_alpha if/elif/else to a ternary
- reuse the already-computed label extent instead of recomputing get_extent
- np.full instead of list*N for the literal na_color vector
- tighten the as_points test-helper comment
render_points output verified byte-identical; as_points renders unchanged.
Datashader pays off for as_points from ~50k cells (~1.66x faster than the
matplotlib scatter backend, rising to ~1.8x at 1M), so switch over there
instead of 500k. Users can still force either backend via method=.
The shapes as_points block computed its datashader canvas extent via
get_extent (exact=True), which transforms every geometry — on 351k Visium HD
shapes that was ~7s/render and dominated the whole as_points cost, so the fast
mode gave no speedup over full-geometry rendering. Use _element_extent_fast
(corner-transform; identical result for axis-aligned transforms, falls back to
get_extent for rotation/shear), which the PR already added for exactly this.
Real overlay (image + shapes color=gene), as_points:
91k: 7.2s -> 2.0s (3.3x)
351k: 24.3s -> 3.1s (7.3x, vs 22.8s full-geometry)
Output is unchanged: the fast extent is byte-identical for axis-aligned data.
- extract _fast_extent(element, cs) helper; the '_element_extent_fast(...) or
get_extent(...)' idiom was duplicated verbatim at the shapes as_points site
and the datashader-canvas helper
- fix docstrings that still said '~500k' after the threshold dropped to 50k
- trim _render_centroids_as_points docstring and a non-actionable comment
No behavior change: _fast_extent is identical to the inline form (verified
== get_extent on blobs), render_points byte-identical.
@timtreis
timtreis merged commit 8730ff4 into mainJun 14, 2026
7 of 8 checks passed
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #702 (per-panel title fix) and #703 (as_points + fast extent)
into the show() decomposition.
Conflict resolution in src/spatialdata_plot/pl/basic.py:
- #702 up-front title-count validation: kept (after num_panels). The
adjacent axes/panel-count check was dropped here because the
decomposition already relocated it into _plan_panels.
- #702 simplified title selection (dropped the per-panel try/except):
applied to the extracted _finalize_panel helper.
- #703 had no show()-level render-dispatch changes (as_points is
param-driven in render.py, handled inside _render_panel already); its
only show()-level change, get_extent -> _get_extent_fast, auto-merged
into the extent block and consumes _render_panel's `wants` dict.
Also sweeps up two pre-existing #703 lint/type nits in utils.py surfaced
by the merge: _fast_extent docstring (ruff D205) and _get_extent_fast
Any-return (mypy).
Verified: ruff + mypy clean; 109 non-visual show/shapes/labels tests pass.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 21 color helpers + 6 color-only format/uniqueness helpers verbatim from
utils.py into a new sibling module pl/_color.py. Imports flow one way:
_color -> utils (downward, for _get_list/to_hex/_build_alignment_dtype_hint/
_MPL_SINGLE_LETTER_COLORS). render.py, basic.py and _datashader.py are repointed
to import color symbols from _color.
The two validators still in utils (_type_check_params, _validate_graph_render_params)
use color symbols; they carry temporary function-local imports of _color (cycle-safe)
until they move to _validate.py in the next commit, where these become top-level imports.
No behavior change (verbatim move; all-private except set_zero_in_cmap_to_transparent,
which is not re-exported). Verified: no import cycle; 410 non-visual tests pass;
ruff + ruff-format clean. Pre-existing #703/#705 mypy/ruff debt in utils.py is
unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 17 validation/type-check functions verbatim from utils.py into a new sibling
module pl/_validate.py. Imports flow one way: _validate -> _color (_is_color_like,
_prepare_cmap_norm, _get_colors_for_categorical_obs, now top-level) and
_validate -> utils (downward). The temporary function-local _color imports added in
the previous commit are hoisted to top-level here and removed from utils.
render.py and basic.py repointed to _validate.
Completes the utils.py split (#696): utils 4918 -> 1311 lines, with
_geometry / _datashader / _color / _validate as single-concern siblings.
No import cycle anywhere. No behavior change (verbatim moves; all-private).
Verified: 410 non-visual tests pass; ruff + ruff-format clean. Pre-existing
#703/#705 mypy debt in utils.py is unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #714 (show() decomposition) into the utils.py split. Only utils.py
conflicted:
- import block: dropped the now-unused `_locate_value` import (it moved to
_color.py with the color code that uses it); kept main's `_locate_value`
out of utils.
- `_fast_extent` docstring: took main's #714 version (D205 fix).
basic.py auto-merged: #714's decomposed show()/helpers now import color and
validation symbols from _color/_validate (the split's repoints), not utils.
Bonus: merging #714 brings its fixes for the pre-existing #703/#705 debt
(_resolve_measure_table str-return, _get_extent_fast Any-return, _fast_extent
D205), so the branch is now fully ruff + mypy clean (no --no-verify).
Verified: no import cycle; ruff + ruff-format + mypy all pass; 410 non-visual
tests pass.
@timtreis
timtreis deleted the feat/centroid-scatter-helper branch July 10, 2026 11:11
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@timtreis@codecov-commenter
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent - #703

Merged
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper
Jun 14, 2026
Merged

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent#703
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper

Conversation

@timtreis

@timtreistimtreis commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

Fast rendering for shapes/labels, in spatialdata-plot's API:

  1. as_points=True — draw each cell as a dot at its centroid instead of its full geometry/mask (the squidpy spatial_scatter idea). matplotlib by default; auto-switches to datashader above ~500k dots, or with method="datashader".
  2. Fast axis-aligned extentpl.show() skips transforming every geometry to size the axes when the element's transform is axis-aligned (falls back to get_extent otherwise).
sdata.pl.render_shapes("cells", color="cell_type", as_points=True).pl.show()
sdata.pl.render_labels("cells", color="leiden", as_points=True, method="datashader").pl.show()

Notes

  • Default (as_points=False) output is unchanged vs main.
  • render_points' datashader pipeline is extracted into a shared _datashader_points (byte-identical) and reused by the centroid path.
  • Datashader can't represent the random-per-cell colours of uncolored labels, so that case stays matplotlib (with a warning).
  • Centroids are transformed to coordinate-system coords, fixing dot placement under non-identity transforms.

Tests

Position parity vs matplotlib (incl. non-identity transform), backend selection, no-color fallback, and test_plot_* visual baselines for matplotlib + datashader as_points (continuous, categorical, no-color).

… helper
Infrastructure for an upcoming "render cells as centroid points" fast mode
(no user-facing render option yet).
Phase 0 — shared scatter primitive:
- Extract `_scatter_points(ax, x, y, color_vector, ...)` from `_render_points`'s
matplotlib branch; `_render_points` now calls it. Byte-identical output
(verified vs main on categorical and continuous point renders). This is the
reuse seam the fast mode will draw through.
Phase 1 — centroid + caching core (headless, fully unit-tested):
- `_compute_element_centroids` / `_compute_label_centroids`: per-instance
centroids in a coordinate system. Shapes use spatialdata's vectorized
`get_centroids`; labels use skimage `regionprops` (the per-label reduction is
orders of magnitude faster than `get_centroids` on rasters), mapped onto the
raster's intrinsic coordinate arrays so it reproduces `get_centroids` exactly
(incl. the pixel-center 0.5 offset) then transformed to the target CS.
- `_get_or_compute_centroids`: reuses/persists centroids via the squidpy
convention. A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted
as the cells' locations; otherwise centroids are computed and written back into
the annotating table's `obsm["spatial"]` with a coordinate-system provenance
marker in `uns`, so later renders are instant. Reads run before writes, so a
valid existing cache is reused rather than clobbered; an incompatible existing
`obsm["spatial"]` is never overwritten; the cache is invalidated when the
requested coordinate system differs.
- Tests: shapes/labels centroids match `get_centroids`; cache round-trip +
provenance; CS invalidation; pre-existing obsm trusted; no-table compute path;
`cache=False` writes nothing.
… provenance
Cleanup from /simplify (no behavioral change):
- Extract `_region_mask_and_keys(table, element)` used by both read and write,
removing the duplicated `get_table_keys` + O(n_obs) `region_key`-string-cast
mask that was computed twice per cold call.
- Read path: validate shape on the raw obsm array and cast only the masked
subset to float, instead of casting the whole `obsm["spatial"]` on every
cache hit (the hot path).
- Write path: coerce a non-dict `uns["spatialdata_plot"]` instead of early
returning after `obsm` was already mutated, so obsm and the provenance marker
are always written together (no half-write).
- Drop the dead `"key"` provenance field (constant, never read back).
- Rename the misleading `table` local (held a table *name*) in
`_get_or_compute_centroids`.
Refactor the centroid cache to store element-*intrinsic* coordinates and
transform to the render coordinate system on demand, instead of caching
coords already mapped into one coordinate system. Decisions from design pass:
- Intrinsic storage: one `obsm["spatial"]` cache serves every coordinate
system (proven equivalent to per-CS computation). `_compute_element_centroids`
returns intrinsic coords (shapes via shapely `.centroid`, labels via
`regionprops`); `_centroids_to_coordinate_system` maps them to the requested
CS via the element's transform; `_get_or_compute_centroids` reads/computes
intrinsic then transforms on return.
- Provenance records `{n, scale_level}` (no coordinate system). Cache is
invalidated when the region's instance count changes (cells added/removed),
not on CS change.
- A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted as the
cells' intrinsic locations and transformed to the render CS.
- Exhaustive model dispatch: shapes and 2D labels supported; other element
types raise NotImplementedError.
- Labels are reduced at full resolution (scale0).
Drops the now-unused `get_centroids` import; adds `ShapesModel`. Tests updated:
parametrized shapes/labels match `get_centroids`; new coordinate-system-
independence test (one cache, two CS); staleness-by-instance-count; trusted
pre-existing obsm; unsupported-type rejection.
…e import, shared obsm gate
Cleanup from /simplify:
- `_centroids_to_coordinate_system` ran `PointsModel.parse` + a dask
`transform(...).compute()` round trip on every call, including cache hits
(~19 ms fixed floor, ~100 ms at 1M cells) — defeating the cache. Replace with
`to_affine_matrix` + a plain numpy matmul: numerically identical (verified
against the dask path for multiple coordinate systems), ~80-140x faster, and
it removes the private-API import `spatialdata._core.operations.transform`
(a repo non-negotiable) plus the now-unused `PointsModel`.
- Widen `_transformable_raster` -> `_transform_carrier` to accept any element
(rasters -> scale0, others as-is), dropping the `isinstance` branch in
`_centroids_to_coordinate_system`.
- Extract `_valid_spatial_obsm(arr, n_obs)` shared by the read and write paths,
reconciling their previously divergent obsm-shape checks (read accepted >=2
columns, write required exactly 2) so they cannot drift.
`render_shapes(..., as_points=True)` and `render_labels(..., as_points=True)`
draw one dot per cell at its centroid instead of the full geometry / rasterized
mask — a large speedup when only cell location matters. New `size=` controls the
marker size.
- Shared `_render_centroids_as_points` draws the scatter (via `_scatter_points`)
and the legend/colorbar. The per-cell color vector is the *same* one the
geometry/raster path computes (`_set_color_source_vec`), so colors match the
full rendering exactly; only the apply step (scatter vs patches/imshow) differs.
- Shapes: centroids from shapely `.centroid` of the (filtered) geometry,
positionally aligned to the color vector, drawn in intrinsic coords via the
element transform. Labels: centroids from `_get_or_compute_centroids`
(regionprops, fast) reindexed to `instance_id`. Positions verified identical to
`sd.get_centroids`.
- `as_points` short-circuits before the geometry/raster path; outline_*, shape
(shapes) and contour_px, outline_* (labels) are ignored with an info log.
- Default (`as_points=False`) output is byte-identical to main.
Tests: non-visual checks that centroids land exactly on `get_centroids` for both
element types and that outline/shape are ignored without error.
Note: as_points currently always uses the matplotlib scatter backend; routing
through datashader for very large cell counts (and persisting the obsm cache to
the user's object rather than show()'s working copy) are follow-ups.
@timtreistimtreis changed the title Centroid extraction + squidpy obsm["spatial"] caching; shared scatter helperFast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingJun 8, 2026
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.08046% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.89%. Comparing base (b370b1f) to head (882d208).

Files with missing linesPatch %Lines
src/spatialdata_plot/pl/utils.py81.81%6 Missing and 6 partials ⚠️
src/spatialdata_plot/pl/render.py92.70%4 Missing and 3 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #703 +/- ##
==========================================
+ Coverage 76.28% 76.89% +0.60% 
==========================================
Files 14 14 Lines 4327 4458 +131 Branches 1006 1035 +29 ==========================================
+ Hits 3301 3428 +127 + Misses 667 664 -3 - Partials 359 366 +7 
Files with missing linesCoverage Δ
src/spatialdata_plot/pl/_datashader.py90.50% <100.00%> (ø)
src/spatialdata_plot/pl/basic.py79.61% <100.00%> (+0.19%)⬆️
src/spatialdata_plot/pl/render_params.py89.02% <100.00%> (+0.22%)⬆️
src/spatialdata_plot/pl/render.py88.16% <92.70%> (+1.11%)⬆️
src/spatialdata_plot/pl/utils.py69.40% <81.81%> (+0.51%)⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`render_labels(element, as_points=True)` with no color crashed:
`instance_id` (the raster's unique values) includes the background label `0`,
which has no centroid, and the literal/no-color color vector is sized to the
raster (not per-instance), so `ax.scatter` got mismatched `c` vs `x`/`y`.
Drop the background label from the rendered instances and align the per-cell
color: for data-driven color the vector is already per-instance and is subset to
match; for the literal/no-color path it is replaced with one na/literal color
per centroid. Data-driven (categorical/continuous) renders are unchanged and
still land exactly on `get_centroids`.
Adds a regression test for the no-color labels case.
…ator
Replace the `regionprops` reduction in `_compute_label_centroids` with an
additive bincount aggregator that streams the labels raster block by block —
one dask chunk (or bounded numpy row-block) in memory at a time — accumulating
per-label `count`/`sum_x`/`sum_y`. This is what makes the feature usable at
Xenium scale:
- Out-of-core: peak memory is one chunk + O(n_labels) accumulators, NOT the
whole raster (measured: 9 MB peak streaming a 268 MB mask). `regionprops`
needs the full array materialized and OOMs on large morphology masks.
- Scales in cell count: 500k+ labels are just array indexing (562k labels in
~1.4 s with 13.5 MB of accumulators); `regionprops`' per-label table does not.
- Faster than `regionprops` (~1.3-1.6x) on in-memory rasters.
- Exact across chunk boundaries (additive reduction) — verified numpy ==
dask-chunked, and identical to `sd.get_centroids`.
- `count` is the cell area, a free by-product (ready for footprint-based dot
sizing).
Drops the `regionprops_table` import; adds `slices_from_chunks`. Adds a unit
test locking the chunk-exact, out-of-core, area-correct behavior.
Note: the chunk loop is currently sequential; parallelizing the per-chunk
partials (dask map_blocks + tree-reduce) is a future speedup.
#705 (measure_obs) merged to main, superseding #703's pre-extraction centroid
block. This merge takes main's utils.py + test_utils.py wholesale (zero #703
delta there) and rewires the labels as_points path onto main's primitive:
- render.py labels branch: drop the deleted `_get_or_compute_centroids` + the
whole cache layer; compute full-resolution (scale0) centroids via main's
`_compute_element_measurements`, and draw them with a transform built from the
*same* scale0 element (`_prepare_transformation(_get_top_data_array(...))`) —
so positions are independent of any rasterization applied to the rendered
`label`. Coerce point_ids to the label dtype so str/object instance ids
(e.g. Xenium readers) align instead of silently reindexing to NaN.
- Net: utils.py == main (the 242-line #703 centroid block + cache layer is
gone); the surviving diff vs main is just the as_points feature.
- Tests: added a non-identity-transform regression test asserting the dots land
at the cells' coordinate-system positions in display space (the guard for the
transform pairing); existing as_points position tests still pass.
Shapes branch unchanged (already intrinsic centroid + trans_data, positionally
aligned to its post-filter color vector).
…ked fields
The fast-mode draw wrapper read cmap/size/alpha/zorder/colorbar/colorbar_params
straight off render_params at both call sites. Pass the render_params dataclass
instead and read them internally; only the genuinely per-branch values
(x/y/color vectors, norm, na_color, transform, adata, palette, col_for_color)
stay explicit. Signature 20->15 params; both as_points call sites lose the
restated render_params plumbing.
Collapse the per-centroid color alignment (drop the unconditional asarray
pre-init and the nested None-guard into a single if/else + ternary), and drop a
redundant np.asarray around the already-ndarray point_ids in the reindex.
Behavior unchanged.
Benchmark showed labels as_points was 16-34x SLOWER than the normal render
because it recomputed full-resolution scale0 centroids while the normal path
downsamples + imshows. Compute centroids on the already-rendered (downsampled)
raster and draw with its trans_data instead: 671k cells goes 19.7s -> 0.88s
(now ~parity with imshow). Centroid error is sub-pixel at display resolution;
position tests updated to display-space within a few-px tolerance.
…gned
show()'s get_extent(exact=True) transforms EVERY shapes/points geometry into
the coordinate system just to take a bounding box - the dominant cost for large
shape collections (~85% of a 5.5M-shape render), unrelated to what is drawn.
Add a self-contained get_extent_fast (drop-in for spatialdata.get_extent) that,
for axis-aligned transforms (scale/flip/90deg/swap + translation - all real
Visium/Xenium data), transforms only the 4 bounding-box corners and reads the
intrinsic bounds vectorised (no per-geometry .apply(is_empty)). Proven identical
to the exact extent for such transforms (spatialdata's own get_extent docstring
notes this); falls back to spatialdata's get_extent for rotation/shear, for
anisotropically-scaled circles (radius->ellipse divergence), and for
images/labels (whose get_extent is already a cheap corner transform).
Measured on real Visium HD render_shapes(as_points=True): 351k 8.5s->1.8s
(4.7x), 5.48M 129s->24s (5.5x); the full (non-as_points) render benefits too.
The whole block is isolated so it can be lifted into spatialdata's get_extent
verbatim (see issue #706). Tests assert it matches get_extent across scale/flip/
rotation/shear for circles and polygons.
@timtreis
timtreisforce-pushed the feat/centroid-scatter-helper branch from 35c8f1f to 57bcf8dCompareJune 9, 2026 23:20
…ompute, underscore)
Follow-up cleanups from review of the get_extent fast path:
- fold _intrinsic_xy_bounds into _element_extent_fast (drop duplicate get_model / geom_type)
- geom.bounds -> geom.total_bounds (C-level union, avoids an Nx4 alloc on large collections)
- batch the four points min/max into one dask.compute
- rename get_extent_fast -> _get_extent_fast (internal helper; matches sibling underscored helpers)
No behavior change; output identical for axis-aligned and rotated/sheared elements.
The datashader shapes canvas sized itself via spatialdata's exact get_extent, transforming every
geometry (O(N)) just for a bounding box -- the same cost _get_extent_fast already removed from
show()'s axis limits, in a second code path. Reuse _element_extent_fast (corner transform for
axis-aligned elements; None -> exact get_extent fallback for rotation/shear), so the result is
pixel-identical and the per-geometry pass is skipped for the common case.
Measured on Curio (69,713 colored shapes): render_shapes 3146 ms -> 1675 ms (1.88x), on top of
the get_extent_fast win already in show(). Mirrors _datashader_canvas_from_dataframe, which
already avoids get_extent for the points path.
…de review)
From the perf-stack review of labels as_points:
- No color column collapsed every dot to a single na_color; now each cell gets a distinct random
colour, matching the mask path's _map_color_seg Case C. Adds a color assertion to the regression test.
- The rasterize drop-filter only ran when a color column was set, so as_points could emit dots at NaN
positions (or drop cells) when rasterization removed labels; extend it to as_points so point ids stay
within the rendered raster.
- ax.scatter autoscales the Normalize in place; copy the shared cmap_params.norm (the shapes path already does).
- render_shapes/render_labels(as_points=True) did not validate size; add the points-path numeric/positive
check so size=-5 / 'big' raise an actionable error instead of a raw matplotlib failure.
Extract the duplicated as_points size-validation block from render_shapes
and render_labels into a shared _validate_as_points_size helper. Defer the
np.asarray(color_vector) conversion in the centroid color path to the only
branch that uses it.
…to feat/centroid-scatter-helper
# Conflicts:
#	tests/pl/test_utils.py
_get_extent_fast already reads get_transformation(element, get_all=True) for
the coordinate-system membership check; pass it through so the fast path does
not re-fetch it per element. The optional kwarg leaves the datashader-canvas
call site unchanged.
_element_extent_fast returned a NaN extent for an element whose geometries are
all empty, which silently poisoned the union in _get_extent_fast (blank axes)
instead of raising spatialdata's clear 'empty collection' error. Guard against
non-finite bounds and fall back. Also trims the verbose extent-block comment and
the scatter/centroid docstrings (net -12 LOC, no behavior change).
@timtreistimtreis changed the title Fast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingFast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extentJun 10, 2026
Four PlotTester baselines: render_shapes/labels(as_points=True) at a base size
and a larger size (size = scatter marker area). Baselines to be generated from CI.
…l tests
Rendered on hatch-test.py3.11-stable; verified dots land at shape/label
centroids, colored by instance_id (labels), larger at size=600.
The empty-shape check ran a per-geometry Python lambda (`.apply(lambda g: not
g.is_empty)`) over every geometry on every shapes render; GeoSeries.is_empty is
GEOS-vectorized. `.is_empty.all()` is identical and ~134x faster on this guard
(665ms -> 5ms at 351k shapes, ~10s -> 80ms at 5.5M).
Shared datashader draw primitive (canvas->aggregate->norm->color-key->shade->
image->colorbar), taking explicit primitives instead of render_params so it can
also serve the as_points centroid path (shapes/labels params use fill_alpha and
lack the density fields). Returns the possibly-recomputed color vectors so the
caller's legend/colorbar stays identical. Verified pixel-identical on
render_points datashader (categorical/continuous/plain/density).
- Route as_points centroids through the shared _datashader_points when method=
'datashader' or above ~500k dots (AS_POINTS_DS_AUTO); matplotlib otherwise.
- No-color labels (one random colour per cell) cannot be aggregated by datashader;
force matplotlib there (warn on explicit method='datashader').
- Fix: shapes as_points drew dots at intrinsic centroid coords while the axes are
in coordinate-system coords, so non-identity transforms misplaced them. Transform
centroids to CS via the element->CS affine for both backends (labels too).
- render_labels gains 'method'; LabelsRenderParams gains 'method'. as_points uses a
fixed datashader reduction ('max', closest to matplotlib) — no user knob.
- shapes non-identity transform regression (the coordinate bug fixed in 5854ae6)
- backend selection: method='datashader' -> datashader image; default -> matplotlib
- no-color labels force matplotlib even with method='datashader' (warns)
- _resolve_as_points_method unit test (threshold/explicit/no-color/empty)
- two test_plot_* visual tests for datashader as_points (baselines from CI)
timtreis added 22 commits June 11, 2026 06:40
Rendered on hatch-test.py3.11-stable; verified datashaded centroids coloured by
category with a categorical legend. Only this test failed on stable CI (missing
baseline); existing datashader baselines unchanged.
_datashader_points' two near-identical reindex/align branches (categorical
source vs colour vector) differed only in which vector they materialize; pick
the source once, then share the reindex/construct. render_points datashader
output verified byte-identical (categorical/continuous/plain/density). -9 LOC.
The datashader as_points canvas was sized to the exact centroid bounding box,
so the outermost centroids sat on the canvas edge and their marker spread was
clipped (half-circles), looking nothing like the matplotlib backend. Grow the
canvas by the spread radius on each side (factor unchanged, so placement still
aligns). render_points keeps pad_for_markers=False -> byte-identical.
…padding fix
The padding fix changed the datashader as_points renders within the comparison
tolerance, so the baselines didn't auto-update. Regenerate from CI so the
committed images reflect the un-clipped output.
CI-rendered (py3.11-stable) after the canvas-padding fix; verified the edge
centroids now render as full circles matching the matplotlib backend.
Restructure the as_points visual tests into matplotlib+datashader pairs that
render identical params (shared helper) for shapes (no color), labels
(instance_id), and labels (categorical). Drop the old inconsistent/stale
baselines; all 6 will be regenerated from CI so the two backends are directly
comparable and look maximally similar.
Positions and sizes align after the canvas-padding fix; dots land at the same
centroids at the same size. Remaining difference is color shading on the
categorical path (datashader modulates alpha by per-pixel count).
Datashader faded single-cell dots (count-driven alpha floor + a second user-alpha
multiply ~= alpha^2), making categorical as_points read much paler than the
matplotlib backend. Add uniform_alpha for the as_points marker mode: a full alpha
floor so each dot is one flat colour at fill_alpha, like a matplotlib marker.
Renamed the as_points flag pad_for_markers -> as_markers (pads canvas + uniform
alpha). render_points unchanged (byte-identical).
…h matplotlib)
CI-rendered (py3.11-stable); the datashader as_points dots now match the
matplotlib markers in colour saturation, position, and size.
…matplotlib
Size (derived, no heuristic): datashader previously rasterized over the centroid
bounding box, so dot display size depended on the canvas/axes extent ratio (shapes
~0.9x, labels ~1.7x matplotlib). Now the canvas spans the same extent as the axes
(like render_points), and the spread radius is set to matplotlib's marker radius
sqrt(s)*dpi/144 (its 'o' marker has diameter sqrt(s)*dpi/72). Result: ds/mpl size
ratio 1.02 +/- 0.03 across element type, size, figure, and dpi.
Continuous colorbar: the spread combined overlapping dots with 'add' (ds_reduction
None -> 'sum'), summing ids and inflating reduction_bounds; marker mode now spreads
with 'max' so overlaps overlay and the colorbar keeps the true range.
render_points unchanged (byte-identical). Replaces the canvas-padding workaround
(the axes-extent canvas already gives edge dots their margin).
… colorbar)
CI-rendered (py3.11-stable): datashader dots now match the matplotlib markers in
size, and the continuous colorbar uses the true value range.
The datashader result is a data-coordinate image that scales with the axes,
while matplotlib markers are fixed in display points. The canvas was sized from
fig.get_size_inches()*dpi (the whole figure), but the axes are smaller (margins,
colorbar), so the image - and every dot - was scaled down: labels (with colorbar)
0.81x matplotlib, shapes 0.88x. Size the as_markers canvas to the axes display box
(ax.get_window_extent()) so 1 canvas px == 1 axes-display px and the sqrt(s)*dpi/144
spread radius matches the marker. Now mean 0.99 +/-5% across sizes/figs/dpi/elements;
render_points untouched (byte-identical). Visual tests render at a non-overlapping
size so the engines' overlap handling (stack vs aggregate) doesn't enter the pairs.
…der canvas
CI-rendered (py3.11-stable). matplotlib and datashader pairs now match in dot
size (the canvas-vs-figure scaling bug is fixed) at a non-overlapping size.
Cleanup pass over the as_points feature (no behavior change):
- trim restated comments/docstrings in _datashader_points, keeping the
load-bearing marker-radius (sqrt(s)*dpi/144) and canvas-vs-axes rationale
- collapse the 3-way min_alpha if/elif/else to a ternary
- reuse the already-computed label extent instead of recomputing get_extent
- np.full instead of list*N for the literal na_color vector
- tighten the as_points test-helper comment
render_points output verified byte-identical; as_points renders unchanged.
Datashader pays off for as_points from ~50k cells (~1.66x faster than the
matplotlib scatter backend, rising to ~1.8x at 1M), so switch over there
instead of 500k. Users can still force either backend via method=.
The shapes as_points block computed its datashader canvas extent via
get_extent (exact=True), which transforms every geometry — on 351k Visium HD
shapes that was ~7s/render and dominated the whole as_points cost, so the fast
mode gave no speedup over full-geometry rendering. Use _element_extent_fast
(corner-transform; identical result for axis-aligned transforms, falls back to
get_extent for rotation/shear), which the PR already added for exactly this.
Real overlay (image + shapes color=gene), as_points:
91k: 7.2s -> 2.0s (3.3x)
351k: 24.3s -> 3.1s (7.3x, vs 22.8s full-geometry)
Output is unchanged: the fast extent is byte-identical for axis-aligned data.
- extract _fast_extent(element, cs) helper; the '_element_extent_fast(...) or
get_extent(...)' idiom was duplicated verbatim at the shapes as_points site
and the datashader-canvas helper
- fix docstrings that still said '~500k' after the threshold dropped to 50k
- trim _render_centroids_as_points docstring and a non-actionable comment
No behavior change: _fast_extent is identical to the inline form (verified
== get_extent on blobs), render_points byte-identical.
@timtreis
timtreis merged commit 8730ff4 into mainJun 14, 2026
7 of 8 checks passed
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #702 (per-panel title fix) and #703 (as_points + fast extent)
into the show() decomposition.
Conflict resolution in src/spatialdata_plot/pl/basic.py:
- #702 up-front title-count validation: kept (after num_panels). The
adjacent axes/panel-count check was dropped here because the
decomposition already relocated it into _plan_panels.
- #702 simplified title selection (dropped the per-panel try/except):
applied to the extracted _finalize_panel helper.
- #703 had no show()-level render-dispatch changes (as_points is
param-driven in render.py, handled inside _render_panel already); its
only show()-level change, get_extent -> _get_extent_fast, auto-merged
into the extent block and consumes _render_panel's `wants` dict.
Also sweeps up two pre-existing #703 lint/type nits in utils.py surfaced
by the merge: _fast_extent docstring (ruff D205) and _get_extent_fast
Any-return (mypy).
Verified: ruff + mypy clean; 109 non-visual show/shapes/labels tests pass.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 21 color helpers + 6 color-only format/uniqueness helpers verbatim from
utils.py into a new sibling module pl/_color.py. Imports flow one way:
_color -> utils (downward, for _get_list/to_hex/_build_alignment_dtype_hint/
_MPL_SINGLE_LETTER_COLORS). render.py, basic.py and _datashader.py are repointed
to import color symbols from _color.
The two validators still in utils (_type_check_params, _validate_graph_render_params)
use color symbols; they carry temporary function-local imports of _color (cycle-safe)
until they move to _validate.py in the next commit, where these become top-level imports.
No behavior change (verbatim move; all-private except set_zero_in_cmap_to_transparent,
which is not re-exported). Verified: no import cycle; 410 non-visual tests pass;
ruff + ruff-format clean. Pre-existing #703/#705 mypy/ruff debt in utils.py is
unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 17 validation/type-check functions verbatim from utils.py into a new sibling
module pl/_validate.py. Imports flow one way: _validate -> _color (_is_color_like,
_prepare_cmap_norm, _get_colors_for_categorical_obs, now top-level) and
_validate -> utils (downward). The temporary function-local _color imports added in
the previous commit are hoisted to top-level here and removed from utils.
render.py and basic.py repointed to _validate.
Completes the utils.py split (#696): utils 4918 -> 1311 lines, with
_geometry / _datashader / _color / _validate as single-concern siblings.
No import cycle anywhere. No behavior change (verbatim moves; all-private).
Verified: 410 non-visual tests pass; ruff + ruff-format clean. Pre-existing
#703/#705 mypy debt in utils.py is unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #714 (show() decomposition) into the utils.py split. Only utils.py
conflicted:
- import block: dropped the now-unused `_locate_value` import (it moved to
_color.py with the color code that uses it); kept main's `_locate_value`
out of utils.
- `_fast_extent` docstring: took main's #714 version (D205 fix).
basic.py auto-merged: #714's decomposed show()/helpers now import color and
validation symbols from _color/_validate (the split's repoints), not utils.
Bonus: merging #714 brings its fixes for the pre-existing #703/#705 debt
(_resolve_measure_table str-return, _get_extent_fast Any-return, _fast_extent
D205), so the branch is now fully ruff + mypy clean (no --no-verify).
Verified: no import cycle; ruff + ruff-format + mypy all pass; 410 non-visual
tests pass.
@timtreis
timtreis deleted the feat/centroid-scatter-helper branch July 10, 2026 11:11
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@timtreis@codecov-commenter
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent - #703

Merged
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper
Jun 14, 2026
Merged

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent#703
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper

Conversation

@timtreis

@timtreistimtreis commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

Fast rendering for shapes/labels, in spatialdata-plot's API:

  1. as_points=True — draw each cell as a dot at its centroid instead of its full geometry/mask (the squidpy spatial_scatter idea). matplotlib by default; auto-switches to datashader above ~500k dots, or with method="datashader".
  2. Fast axis-aligned extentpl.show() skips transforming every geometry to size the axes when the element's transform is axis-aligned (falls back to get_extent otherwise).
sdata.pl.render_shapes("cells", color="cell_type", as_points=True).pl.show()
sdata.pl.render_labels("cells", color="leiden", as_points=True, method="datashader").pl.show()

Notes

  • Default (as_points=False) output is unchanged vs main.
  • render_points' datashader pipeline is extracted into a shared _datashader_points (byte-identical) and reused by the centroid path.
  • Datashader can't represent the random-per-cell colours of uncolored labels, so that case stays matplotlib (with a warning).
  • Centroids are transformed to coordinate-system coords, fixing dot placement under non-identity transforms.

Tests

Position parity vs matplotlib (incl. non-identity transform), backend selection, no-color fallback, and test_plot_* visual baselines for matplotlib + datashader as_points (continuous, categorical, no-color).

… helper
Infrastructure for an upcoming "render cells as centroid points" fast mode
(no user-facing render option yet).
Phase 0 — shared scatter primitive:
- Extract `_scatter_points(ax, x, y, color_vector, ...)` from `_render_points`'s
matplotlib branch; `_render_points` now calls it. Byte-identical output
(verified vs main on categorical and continuous point renders). This is the
reuse seam the fast mode will draw through.
Phase 1 — centroid + caching core (headless, fully unit-tested):
- `_compute_element_centroids` / `_compute_label_centroids`: per-instance
centroids in a coordinate system. Shapes use spatialdata's vectorized
`get_centroids`; labels use skimage `regionprops` (the per-label reduction is
orders of magnitude faster than `get_centroids` on rasters), mapped onto the
raster's intrinsic coordinate arrays so it reproduces `get_centroids` exactly
(incl. the pixel-center 0.5 offset) then transformed to the target CS.
- `_get_or_compute_centroids`: reuses/persists centroids via the squidpy
convention. A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted
as the cells' locations; otherwise centroids are computed and written back into
the annotating table's `obsm["spatial"]` with a coordinate-system provenance
marker in `uns`, so later renders are instant. Reads run before writes, so a
valid existing cache is reused rather than clobbered; an incompatible existing
`obsm["spatial"]` is never overwritten; the cache is invalidated when the
requested coordinate system differs.
- Tests: shapes/labels centroids match `get_centroids`; cache round-trip +
provenance; CS invalidation; pre-existing obsm trusted; no-table compute path;
`cache=False` writes nothing.
… provenance
Cleanup from /simplify (no behavioral change):
- Extract `_region_mask_and_keys(table, element)` used by both read and write,
removing the duplicated `get_table_keys` + O(n_obs) `region_key`-string-cast
mask that was computed twice per cold call.
- Read path: validate shape on the raw obsm array and cast only the masked
subset to float, instead of casting the whole `obsm["spatial"]` on every
cache hit (the hot path).
- Write path: coerce a non-dict `uns["spatialdata_plot"]` instead of early
returning after `obsm` was already mutated, so obsm and the provenance marker
are always written together (no half-write).
- Drop the dead `"key"` provenance field (constant, never read back).
- Rename the misleading `table` local (held a table *name*) in
`_get_or_compute_centroids`.
Refactor the centroid cache to store element-*intrinsic* coordinates and
transform to the render coordinate system on demand, instead of caching
coords already mapped into one coordinate system. Decisions from design pass:
- Intrinsic storage: one `obsm["spatial"]` cache serves every coordinate
system (proven equivalent to per-CS computation). `_compute_element_centroids`
returns intrinsic coords (shapes via shapely `.centroid`, labels via
`regionprops`); `_centroids_to_coordinate_system` maps them to the requested
CS via the element's transform; `_get_or_compute_centroids` reads/computes
intrinsic then transforms on return.
- Provenance records `{n, scale_level}` (no coordinate system). Cache is
invalidated when the region's instance count changes (cells added/removed),
not on CS change.
- A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted as the
cells' intrinsic locations and transformed to the render CS.
- Exhaustive model dispatch: shapes and 2D labels supported; other element
types raise NotImplementedError.
- Labels are reduced at full resolution (scale0).
Drops the now-unused `get_centroids` import; adds `ShapesModel`. Tests updated:
parametrized shapes/labels match `get_centroids`; new coordinate-system-
independence test (one cache, two CS); staleness-by-instance-count; trusted
pre-existing obsm; unsupported-type rejection.
…e import, shared obsm gate
Cleanup from /simplify:
- `_centroids_to_coordinate_system` ran `PointsModel.parse` + a dask
`transform(...).compute()` round trip on every call, including cache hits
(~19 ms fixed floor, ~100 ms at 1M cells) — defeating the cache. Replace with
`to_affine_matrix` + a plain numpy matmul: numerically identical (verified
against the dask path for multiple coordinate systems), ~80-140x faster, and
it removes the private-API import `spatialdata._core.operations.transform`
(a repo non-negotiable) plus the now-unused `PointsModel`.
- Widen `_transformable_raster` -> `_transform_carrier` to accept any element
(rasters -> scale0, others as-is), dropping the `isinstance` branch in
`_centroids_to_coordinate_system`.
- Extract `_valid_spatial_obsm(arr, n_obs)` shared by the read and write paths,
reconciling their previously divergent obsm-shape checks (read accepted >=2
columns, write required exactly 2) so they cannot drift.
`render_shapes(..., as_points=True)` and `render_labels(..., as_points=True)`
draw one dot per cell at its centroid instead of the full geometry / rasterized
mask — a large speedup when only cell location matters. New `size=` controls the
marker size.
- Shared `_render_centroids_as_points` draws the scatter (via `_scatter_points`)
and the legend/colorbar. The per-cell color vector is the *same* one the
geometry/raster path computes (`_set_color_source_vec`), so colors match the
full rendering exactly; only the apply step (scatter vs patches/imshow) differs.
- Shapes: centroids from shapely `.centroid` of the (filtered) geometry,
positionally aligned to the color vector, drawn in intrinsic coords via the
element transform. Labels: centroids from `_get_or_compute_centroids`
(regionprops, fast) reindexed to `instance_id`. Positions verified identical to
`sd.get_centroids`.
- `as_points` short-circuits before the geometry/raster path; outline_*, shape
(shapes) and contour_px, outline_* (labels) are ignored with an info log.
- Default (`as_points=False`) output is byte-identical to main.
Tests: non-visual checks that centroids land exactly on `get_centroids` for both
element types and that outline/shape are ignored without error.
Note: as_points currently always uses the matplotlib scatter backend; routing
through datashader for very large cell counts (and persisting the obsm cache to
the user's object rather than show()'s working copy) are follow-ups.
@timtreistimtreis changed the title Centroid extraction + squidpy obsm["spatial"] caching; shared scatter helperFast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingJun 8, 2026
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.08046% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.89%. Comparing base (b370b1f) to head (882d208).

Files with missing linesPatch %Lines
src/spatialdata_plot/pl/utils.py81.81%6 Missing and 6 partials ⚠️
src/spatialdata_plot/pl/render.py92.70%4 Missing and 3 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #703 +/- ##
==========================================
+ Coverage 76.28% 76.89% +0.60% 
==========================================
Files 14 14 Lines 4327 4458 +131 Branches 1006 1035 +29 ==========================================
+ Hits 3301 3428 +127 + Misses 667 664 -3 - Partials 359 366 +7 
Files with missing linesCoverage Δ
src/spatialdata_plot/pl/_datashader.py90.50% <100.00%> (ø)
src/spatialdata_plot/pl/basic.py79.61% <100.00%> (+0.19%)⬆️
src/spatialdata_plot/pl/render_params.py89.02% <100.00%> (+0.22%)⬆️
src/spatialdata_plot/pl/render.py88.16% <92.70%> (+1.11%)⬆️
src/spatialdata_plot/pl/utils.py69.40% <81.81%> (+0.51%)⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`render_labels(element, as_points=True)` with no color crashed:
`instance_id` (the raster's unique values) includes the background label `0`,
which has no centroid, and the literal/no-color color vector is sized to the
raster (not per-instance), so `ax.scatter` got mismatched `c` vs `x`/`y`.
Drop the background label from the rendered instances and align the per-cell
color: for data-driven color the vector is already per-instance and is subset to
match; for the literal/no-color path it is replaced with one na/literal color
per centroid. Data-driven (categorical/continuous) renders are unchanged and
still land exactly on `get_centroids`.
Adds a regression test for the no-color labels case.
…ator
Replace the `regionprops` reduction in `_compute_label_centroids` with an
additive bincount aggregator that streams the labels raster block by block —
one dask chunk (or bounded numpy row-block) in memory at a time — accumulating
per-label `count`/`sum_x`/`sum_y`. This is what makes the feature usable at
Xenium scale:
- Out-of-core: peak memory is one chunk + O(n_labels) accumulators, NOT the
whole raster (measured: 9 MB peak streaming a 268 MB mask). `regionprops`
needs the full array materialized and OOMs on large morphology masks.
- Scales in cell count: 500k+ labels are just array indexing (562k labels in
~1.4 s with 13.5 MB of accumulators); `regionprops`' per-label table does not.
- Faster than `regionprops` (~1.3-1.6x) on in-memory rasters.
- Exact across chunk boundaries (additive reduction) — verified numpy ==
dask-chunked, and identical to `sd.get_centroids`.
- `count` is the cell area, a free by-product (ready for footprint-based dot
sizing).
Drops the `regionprops_table` import; adds `slices_from_chunks`. Adds a unit
test locking the chunk-exact, out-of-core, area-correct behavior.
Note: the chunk loop is currently sequential; parallelizing the per-chunk
partials (dask map_blocks + tree-reduce) is a future speedup.
#705 (measure_obs) merged to main, superseding #703's pre-extraction centroid
block. This merge takes main's utils.py + test_utils.py wholesale (zero #703
delta there) and rewires the labels as_points path onto main's primitive:
- render.py labels branch: drop the deleted `_get_or_compute_centroids` + the
whole cache layer; compute full-resolution (scale0) centroids via main's
`_compute_element_measurements`, and draw them with a transform built from the
*same* scale0 element (`_prepare_transformation(_get_top_data_array(...))`) —
so positions are independent of any rasterization applied to the rendered
`label`. Coerce point_ids to the label dtype so str/object instance ids
(e.g. Xenium readers) align instead of silently reindexing to NaN.
- Net: utils.py == main (the 242-line #703 centroid block + cache layer is
gone); the surviving diff vs main is just the as_points feature.
- Tests: added a non-identity-transform regression test asserting the dots land
at the cells' coordinate-system positions in display space (the guard for the
transform pairing); existing as_points position tests still pass.
Shapes branch unchanged (already intrinsic centroid + trans_data, positionally
aligned to its post-filter color vector).
…ked fields
The fast-mode draw wrapper read cmap/size/alpha/zorder/colorbar/colorbar_params
straight off render_params at both call sites. Pass the render_params dataclass
instead and read them internally; only the genuinely per-branch values
(x/y/color vectors, norm, na_color, transform, adata, palette, col_for_color)
stay explicit. Signature 20->15 params; both as_points call sites lose the
restated render_params plumbing.
Collapse the per-centroid color alignment (drop the unconditional asarray
pre-init and the nested None-guard into a single if/else + ternary), and drop a
redundant np.asarray around the already-ndarray point_ids in the reindex.
Behavior unchanged.
Benchmark showed labels as_points was 16-34x SLOWER than the normal render
because it recomputed full-resolution scale0 centroids while the normal path
downsamples + imshows. Compute centroids on the already-rendered (downsampled)
raster and draw with its trans_data instead: 671k cells goes 19.7s -> 0.88s
(now ~parity with imshow). Centroid error is sub-pixel at display resolution;
position tests updated to display-space within a few-px tolerance.
…gned
show()'s get_extent(exact=True) transforms EVERY shapes/points geometry into
the coordinate system just to take a bounding box - the dominant cost for large
shape collections (~85% of a 5.5M-shape render), unrelated to what is drawn.
Add a self-contained get_extent_fast (drop-in for spatialdata.get_extent) that,
for axis-aligned transforms (scale/flip/90deg/swap + translation - all real
Visium/Xenium data), transforms only the 4 bounding-box corners and reads the
intrinsic bounds vectorised (no per-geometry .apply(is_empty)). Proven identical
to the exact extent for such transforms (spatialdata's own get_extent docstring
notes this); falls back to spatialdata's get_extent for rotation/shear, for
anisotropically-scaled circles (radius->ellipse divergence), and for
images/labels (whose get_extent is already a cheap corner transform).
Measured on real Visium HD render_shapes(as_points=True): 351k 8.5s->1.8s
(4.7x), 5.48M 129s->24s (5.5x); the full (non-as_points) render benefits too.
The whole block is isolated so it can be lifted into spatialdata's get_extent
verbatim (see issue #706). Tests assert it matches get_extent across scale/flip/
rotation/shear for circles and polygons.
@timtreis
timtreisforce-pushed the feat/centroid-scatter-helper branch from 35c8f1f to 57bcf8dCompareJune 9, 2026 23:20
…ompute, underscore)
Follow-up cleanups from review of the get_extent fast path:
- fold _intrinsic_xy_bounds into _element_extent_fast (drop duplicate get_model / geom_type)
- geom.bounds -> geom.total_bounds (C-level union, avoids an Nx4 alloc on large collections)
- batch the four points min/max into one dask.compute
- rename get_extent_fast -> _get_extent_fast (internal helper; matches sibling underscored helpers)
No behavior change; output identical for axis-aligned and rotated/sheared elements.
The datashader shapes canvas sized itself via spatialdata's exact get_extent, transforming every
geometry (O(N)) just for a bounding box -- the same cost _get_extent_fast already removed from
show()'s axis limits, in a second code path. Reuse _element_extent_fast (corner transform for
axis-aligned elements; None -> exact get_extent fallback for rotation/shear), so the result is
pixel-identical and the per-geometry pass is skipped for the common case.
Measured on Curio (69,713 colored shapes): render_shapes 3146 ms -> 1675 ms (1.88x), on top of
the get_extent_fast win already in show(). Mirrors _datashader_canvas_from_dataframe, which
already avoids get_extent for the points path.
…de review)
From the perf-stack review of labels as_points:
- No color column collapsed every dot to a single na_color; now each cell gets a distinct random
colour, matching the mask path's _map_color_seg Case C. Adds a color assertion to the regression test.
- The rasterize drop-filter only ran when a color column was set, so as_points could emit dots at NaN
positions (or drop cells) when rasterization removed labels; extend it to as_points so point ids stay
within the rendered raster.
- ax.scatter autoscales the Normalize in place; copy the shared cmap_params.norm (the shapes path already does).
- render_shapes/render_labels(as_points=True) did not validate size; add the points-path numeric/positive
check so size=-5 / 'big' raise an actionable error instead of a raw matplotlib failure.
Extract the duplicated as_points size-validation block from render_shapes
and render_labels into a shared _validate_as_points_size helper. Defer the
np.asarray(color_vector) conversion in the centroid color path to the only
branch that uses it.
…to feat/centroid-scatter-helper
# Conflicts:
#	tests/pl/test_utils.py
_get_extent_fast already reads get_transformation(element, get_all=True) for
the coordinate-system membership check; pass it through so the fast path does
not re-fetch it per element. The optional kwarg leaves the datashader-canvas
call site unchanged.
_element_extent_fast returned a NaN extent for an element whose geometries are
all empty, which silently poisoned the union in _get_extent_fast (blank axes)
instead of raising spatialdata's clear 'empty collection' error. Guard against
non-finite bounds and fall back. Also trims the verbose extent-block comment and
the scatter/centroid docstrings (net -12 LOC, no behavior change).
@timtreistimtreis changed the title Fast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingFast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extentJun 10, 2026
Four PlotTester baselines: render_shapes/labels(as_points=True) at a base size
and a larger size (size = scatter marker area). Baselines to be generated from CI.
…l tests
Rendered on hatch-test.py3.11-stable; verified dots land at shape/label
centroids, colored by instance_id (labels), larger at size=600.
The empty-shape check ran a per-geometry Python lambda (`.apply(lambda g: not
g.is_empty)`) over every geometry on every shapes render; GeoSeries.is_empty is
GEOS-vectorized. `.is_empty.all()` is identical and ~134x faster on this guard
(665ms -> 5ms at 351k shapes, ~10s -> 80ms at 5.5M).
Shared datashader draw primitive (canvas->aggregate->norm->color-key->shade->
image->colorbar), taking explicit primitives instead of render_params so it can
also serve the as_points centroid path (shapes/labels params use fill_alpha and
lack the density fields). Returns the possibly-recomputed color vectors so the
caller's legend/colorbar stays identical. Verified pixel-identical on
render_points datashader (categorical/continuous/plain/density).
- Route as_points centroids through the shared _datashader_points when method=
'datashader' or above ~500k dots (AS_POINTS_DS_AUTO); matplotlib otherwise.
- No-color labels (one random colour per cell) cannot be aggregated by datashader;
force matplotlib there (warn on explicit method='datashader').
- Fix: shapes as_points drew dots at intrinsic centroid coords while the axes are
in coordinate-system coords, so non-identity transforms misplaced them. Transform
centroids to CS via the element->CS affine for both backends (labels too).
- render_labels gains 'method'; LabelsRenderParams gains 'method'. as_points uses a
fixed datashader reduction ('max', closest to matplotlib) — no user knob.
- shapes non-identity transform regression (the coordinate bug fixed in 5854ae6)
- backend selection: method='datashader' -> datashader image; default -> matplotlib
- no-color labels force matplotlib even with method='datashader' (warns)
- _resolve_as_points_method unit test (threshold/explicit/no-color/empty)
- two test_plot_* visual tests for datashader as_points (baselines from CI)
timtreis added 22 commits June 11, 2026 06:40
Rendered on hatch-test.py3.11-stable; verified datashaded centroids coloured by
category with a categorical legend. Only this test failed on stable CI (missing
baseline); existing datashader baselines unchanged.
_datashader_points' two near-identical reindex/align branches (categorical
source vs colour vector) differed only in which vector they materialize; pick
the source once, then share the reindex/construct. render_points datashader
output verified byte-identical (categorical/continuous/plain/density). -9 LOC.
The datashader as_points canvas was sized to the exact centroid bounding box,
so the outermost centroids sat on the canvas edge and their marker spread was
clipped (half-circles), looking nothing like the matplotlib backend. Grow the
canvas by the spread radius on each side (factor unchanged, so placement still
aligns). render_points keeps pad_for_markers=False -> byte-identical.
…padding fix
The padding fix changed the datashader as_points renders within the comparison
tolerance, so the baselines didn't auto-update. Regenerate from CI so the
committed images reflect the un-clipped output.
CI-rendered (py3.11-stable) after the canvas-padding fix; verified the edge
centroids now render as full circles matching the matplotlib backend.
Restructure the as_points visual tests into matplotlib+datashader pairs that
render identical params (shared helper) for shapes (no color), labels
(instance_id), and labels (categorical). Drop the old inconsistent/stale
baselines; all 6 will be regenerated from CI so the two backends are directly
comparable and look maximally similar.
Positions and sizes align after the canvas-padding fix; dots land at the same
centroids at the same size. Remaining difference is color shading on the
categorical path (datashader modulates alpha by per-pixel count).
Datashader faded single-cell dots (count-driven alpha floor + a second user-alpha
multiply ~= alpha^2), making categorical as_points read much paler than the
matplotlib backend. Add uniform_alpha for the as_points marker mode: a full alpha
floor so each dot is one flat colour at fill_alpha, like a matplotlib marker.
Renamed the as_points flag pad_for_markers -> as_markers (pads canvas + uniform
alpha). render_points unchanged (byte-identical).
…h matplotlib)
CI-rendered (py3.11-stable); the datashader as_points dots now match the
matplotlib markers in colour saturation, position, and size.
…matplotlib
Size (derived, no heuristic): datashader previously rasterized over the centroid
bounding box, so dot display size depended on the canvas/axes extent ratio (shapes
~0.9x, labels ~1.7x matplotlib). Now the canvas spans the same extent as the axes
(like render_points), and the spread radius is set to matplotlib's marker radius
sqrt(s)*dpi/144 (its 'o' marker has diameter sqrt(s)*dpi/72). Result: ds/mpl size
ratio 1.02 +/- 0.03 across element type, size, figure, and dpi.
Continuous colorbar: the spread combined overlapping dots with 'add' (ds_reduction
None -> 'sum'), summing ids and inflating reduction_bounds; marker mode now spreads
with 'max' so overlaps overlay and the colorbar keeps the true range.
render_points unchanged (byte-identical). Replaces the canvas-padding workaround
(the axes-extent canvas already gives edge dots their margin).
… colorbar)
CI-rendered (py3.11-stable): datashader dots now match the matplotlib markers in
size, and the continuous colorbar uses the true value range.
The datashader result is a data-coordinate image that scales with the axes,
while matplotlib markers are fixed in display points. The canvas was sized from
fig.get_size_inches()*dpi (the whole figure), but the axes are smaller (margins,
colorbar), so the image - and every dot - was scaled down: labels (with colorbar)
0.81x matplotlib, shapes 0.88x. Size the as_markers canvas to the axes display box
(ax.get_window_extent()) so 1 canvas px == 1 axes-display px and the sqrt(s)*dpi/144
spread radius matches the marker. Now mean 0.99 +/-5% across sizes/figs/dpi/elements;
render_points untouched (byte-identical). Visual tests render at a non-overlapping
size so the engines' overlap handling (stack vs aggregate) doesn't enter the pairs.
…der canvas
CI-rendered (py3.11-stable). matplotlib and datashader pairs now match in dot
size (the canvas-vs-figure scaling bug is fixed) at a non-overlapping size.
Cleanup pass over the as_points feature (no behavior change):
- trim restated comments/docstrings in _datashader_points, keeping the
load-bearing marker-radius (sqrt(s)*dpi/144) and canvas-vs-axes rationale
- collapse the 3-way min_alpha if/elif/else to a ternary
- reuse the already-computed label extent instead of recomputing get_extent
- np.full instead of list*N for the literal na_color vector
- tighten the as_points test-helper comment
render_points output verified byte-identical; as_points renders unchanged.
Datashader pays off for as_points from ~50k cells (~1.66x faster than the
matplotlib scatter backend, rising to ~1.8x at 1M), so switch over there
instead of 500k. Users can still force either backend via method=.
The shapes as_points block computed its datashader canvas extent via
get_extent (exact=True), which transforms every geometry — on 351k Visium HD
shapes that was ~7s/render and dominated the whole as_points cost, so the fast
mode gave no speedup over full-geometry rendering. Use _element_extent_fast
(corner-transform; identical result for axis-aligned transforms, falls back to
get_extent for rotation/shear), which the PR already added for exactly this.
Real overlay (image + shapes color=gene), as_points:
91k: 7.2s -> 2.0s (3.3x)
351k: 24.3s -> 3.1s (7.3x, vs 22.8s full-geometry)
Output is unchanged: the fast extent is byte-identical for axis-aligned data.
- extract _fast_extent(element, cs) helper; the '_element_extent_fast(...) or
get_extent(...)' idiom was duplicated verbatim at the shapes as_points site
and the datashader-canvas helper
- fix docstrings that still said '~500k' after the threshold dropped to 50k
- trim _render_centroids_as_points docstring and a non-actionable comment
No behavior change: _fast_extent is identical to the inline form (verified
== get_extent on blobs), render_points byte-identical.
@timtreis
timtreis merged commit 8730ff4 into mainJun 14, 2026
7 of 8 checks passed
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #702 (per-panel title fix) and #703 (as_points + fast extent)
into the show() decomposition.
Conflict resolution in src/spatialdata_plot/pl/basic.py:
- #702 up-front title-count validation: kept (after num_panels). The
adjacent axes/panel-count check was dropped here because the
decomposition already relocated it into _plan_panels.
- #702 simplified title selection (dropped the per-panel try/except):
applied to the extracted _finalize_panel helper.
- #703 had no show()-level render-dispatch changes (as_points is
param-driven in render.py, handled inside _render_panel already); its
only show()-level change, get_extent -> _get_extent_fast, auto-merged
into the extent block and consumes _render_panel's `wants` dict.
Also sweeps up two pre-existing #703 lint/type nits in utils.py surfaced
by the merge: _fast_extent docstring (ruff D205) and _get_extent_fast
Any-return (mypy).
Verified: ruff + mypy clean; 109 non-visual show/shapes/labels tests pass.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 21 color helpers + 6 color-only format/uniqueness helpers verbatim from
utils.py into a new sibling module pl/_color.py. Imports flow one way:
_color -> utils (downward, for _get_list/to_hex/_build_alignment_dtype_hint/
_MPL_SINGLE_LETTER_COLORS). render.py, basic.py and _datashader.py are repointed
to import color symbols from _color.
The two validators still in utils (_type_check_params, _validate_graph_render_params)
use color symbols; they carry temporary function-local imports of _color (cycle-safe)
until they move to _validate.py in the next commit, where these become top-level imports.
No behavior change (verbatim move; all-private except set_zero_in_cmap_to_transparent,
which is not re-exported). Verified: no import cycle; 410 non-visual tests pass;
ruff + ruff-format clean. Pre-existing #703/#705 mypy/ruff debt in utils.py is
unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 17 validation/type-check functions verbatim from utils.py into a new sibling
module pl/_validate.py. Imports flow one way: _validate -> _color (_is_color_like,
_prepare_cmap_norm, _get_colors_for_categorical_obs, now top-level) and
_validate -> utils (downward). The temporary function-local _color imports added in
the previous commit are hoisted to top-level here and removed from utils.
render.py and basic.py repointed to _validate.
Completes the utils.py split (#696): utils 4918 -> 1311 lines, with
_geometry / _datashader / _color / _validate as single-concern siblings.
No import cycle anywhere. No behavior change (verbatim moves; all-private).
Verified: 410 non-visual tests pass; ruff + ruff-format clean. Pre-existing
#703/#705 mypy debt in utils.py is unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #714 (show() decomposition) into the utils.py split. Only utils.py
conflicted:
- import block: dropped the now-unused `_locate_value` import (it moved to
_color.py with the color code that uses it); kept main's `_locate_value`
out of utils.
- `_fast_extent` docstring: took main's #714 version (D205 fix).
basic.py auto-merged: #714's decomposed show()/helpers now import color and
validation symbols from _color/_validate (the split's repoints), not utils.
Bonus: merging #714 brings its fixes for the pre-existing #703/#705 debt
(_resolve_measure_table str-return, _get_extent_fast Any-return, _fast_extent
D205), so the branch is now fully ruff + mypy clean (no --no-verify).
Verified: no import cycle; ruff + ruff-format + mypy all pass; 410 non-visual
tests pass.
@timtreis
timtreis deleted the feat/centroid-scatter-helper branch July 10, 2026 11:11
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@timtreis@codecov-commenter
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent - #703

Merged
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper
Jun 14, 2026
Merged

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent#703
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper

Conversation

@timtreis

@timtreistimtreis commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

Fast rendering for shapes/labels, in spatialdata-plot's API:

  1. as_points=True — draw each cell as a dot at its centroid instead of its full geometry/mask (the squidpy spatial_scatter idea). matplotlib by default; auto-switches to datashader above ~500k dots, or with method="datashader".
  2. Fast axis-aligned extentpl.show() skips transforming every geometry to size the axes when the element's transform is axis-aligned (falls back to get_extent otherwise).
sdata.pl.render_shapes("cells", color="cell_type", as_points=True).pl.show()
sdata.pl.render_labels("cells", color="leiden", as_points=True, method="datashader").pl.show()

Notes

  • Default (as_points=False) output is unchanged vs main.
  • render_points' datashader pipeline is extracted into a shared _datashader_points (byte-identical) and reused by the centroid path.
  • Datashader can't represent the random-per-cell colours of uncolored labels, so that case stays matplotlib (with a warning).
  • Centroids are transformed to coordinate-system coords, fixing dot placement under non-identity transforms.

Tests

Position parity vs matplotlib (incl. non-identity transform), backend selection, no-color fallback, and test_plot_* visual baselines for matplotlib + datashader as_points (continuous, categorical, no-color).

… helper
Infrastructure for an upcoming "render cells as centroid points" fast mode
(no user-facing render option yet).
Phase 0 — shared scatter primitive:
- Extract `_scatter_points(ax, x, y, color_vector, ...)` from `_render_points`'s
matplotlib branch; `_render_points` now calls it. Byte-identical output
(verified vs main on categorical and continuous point renders). This is the
reuse seam the fast mode will draw through.
Phase 1 — centroid + caching core (headless, fully unit-tested):
- `_compute_element_centroids` / `_compute_label_centroids`: per-instance
centroids in a coordinate system. Shapes use spatialdata's vectorized
`get_centroids`; labels use skimage `regionprops` (the per-label reduction is
orders of magnitude faster than `get_centroids` on rasters), mapped onto the
raster's intrinsic coordinate arrays so it reproduces `get_centroids` exactly
(incl. the pixel-center 0.5 offset) then transformed to the target CS.
- `_get_or_compute_centroids`: reuses/persists centroids via the squidpy
convention. A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted
as the cells' locations; otherwise centroids are computed and written back into
the annotating table's `obsm["spatial"]` with a coordinate-system provenance
marker in `uns`, so later renders are instant. Reads run before writes, so a
valid existing cache is reused rather than clobbered; an incompatible existing
`obsm["spatial"]` is never overwritten; the cache is invalidated when the
requested coordinate system differs.
- Tests: shapes/labels centroids match `get_centroids`; cache round-trip +
provenance; CS invalidation; pre-existing obsm trusted; no-table compute path;
`cache=False` writes nothing.
… provenance
Cleanup from /simplify (no behavioral change):
- Extract `_region_mask_and_keys(table, element)` used by both read and write,
removing the duplicated `get_table_keys` + O(n_obs) `region_key`-string-cast
mask that was computed twice per cold call.
- Read path: validate shape on the raw obsm array and cast only the masked
subset to float, instead of casting the whole `obsm["spatial"]` on every
cache hit (the hot path).
- Write path: coerce a non-dict `uns["spatialdata_plot"]` instead of early
returning after `obsm` was already mutated, so obsm and the provenance marker
are always written together (no half-write).
- Drop the dead `"key"` provenance field (constant, never read back).
- Rename the misleading `table` local (held a table *name*) in
`_get_or_compute_centroids`.
Refactor the centroid cache to store element-*intrinsic* coordinates and
transform to the render coordinate system on demand, instead of caching
coords already mapped into one coordinate system. Decisions from design pass:
- Intrinsic storage: one `obsm["spatial"]` cache serves every coordinate
system (proven equivalent to per-CS computation). `_compute_element_centroids`
returns intrinsic coords (shapes via shapely `.centroid`, labels via
`regionprops`); `_centroids_to_coordinate_system` maps them to the requested
CS via the element's transform; `_get_or_compute_centroids` reads/computes
intrinsic then transforms on return.
- Provenance records `{n, scale_level}` (no coordinate system). Cache is
invalidated when the region's instance count changes (cells added/removed),
not on CS change.
- A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted as the
cells' intrinsic locations and transformed to the render CS.
- Exhaustive model dispatch: shapes and 2D labels supported; other element
types raise NotImplementedError.
- Labels are reduced at full resolution (scale0).
Drops the now-unused `get_centroids` import; adds `ShapesModel`. Tests updated:
parametrized shapes/labels match `get_centroids`; new coordinate-system-
independence test (one cache, two CS); staleness-by-instance-count; trusted
pre-existing obsm; unsupported-type rejection.
…e import, shared obsm gate
Cleanup from /simplify:
- `_centroids_to_coordinate_system` ran `PointsModel.parse` + a dask
`transform(...).compute()` round trip on every call, including cache hits
(~19 ms fixed floor, ~100 ms at 1M cells) — defeating the cache. Replace with
`to_affine_matrix` + a plain numpy matmul: numerically identical (verified
against the dask path for multiple coordinate systems), ~80-140x faster, and
it removes the private-API import `spatialdata._core.operations.transform`
(a repo non-negotiable) plus the now-unused `PointsModel`.
- Widen `_transformable_raster` -> `_transform_carrier` to accept any element
(rasters -> scale0, others as-is), dropping the `isinstance` branch in
`_centroids_to_coordinate_system`.
- Extract `_valid_spatial_obsm(arr, n_obs)` shared by the read and write paths,
reconciling their previously divergent obsm-shape checks (read accepted >=2
columns, write required exactly 2) so they cannot drift.
`render_shapes(..., as_points=True)` and `render_labels(..., as_points=True)`
draw one dot per cell at its centroid instead of the full geometry / rasterized
mask — a large speedup when only cell location matters. New `size=` controls the
marker size.
- Shared `_render_centroids_as_points` draws the scatter (via `_scatter_points`)
and the legend/colorbar. The per-cell color vector is the *same* one the
geometry/raster path computes (`_set_color_source_vec`), so colors match the
full rendering exactly; only the apply step (scatter vs patches/imshow) differs.
- Shapes: centroids from shapely `.centroid` of the (filtered) geometry,
positionally aligned to the color vector, drawn in intrinsic coords via the
element transform. Labels: centroids from `_get_or_compute_centroids`
(regionprops, fast) reindexed to `instance_id`. Positions verified identical to
`sd.get_centroids`.
- `as_points` short-circuits before the geometry/raster path; outline_*, shape
(shapes) and contour_px, outline_* (labels) are ignored with an info log.
- Default (`as_points=False`) output is byte-identical to main.
Tests: non-visual checks that centroids land exactly on `get_centroids` for both
element types and that outline/shape are ignored without error.
Note: as_points currently always uses the matplotlib scatter backend; routing
through datashader for very large cell counts (and persisting the obsm cache to
the user's object rather than show()'s working copy) are follow-ups.
@timtreistimtreis changed the title Centroid extraction + squidpy obsm["spatial"] caching; shared scatter helperFast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingJun 8, 2026
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.08046% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.89%. Comparing base (b370b1f) to head (882d208).

Files with missing linesPatch %Lines
src/spatialdata_plot/pl/utils.py81.81%6 Missing and 6 partials ⚠️
src/spatialdata_plot/pl/render.py92.70%4 Missing and 3 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #703 +/- ##
==========================================
+ Coverage 76.28% 76.89% +0.60% 
==========================================
Files 14 14 Lines 4327 4458 +131 Branches 1006 1035 +29 ==========================================
+ Hits 3301 3428 +127 + Misses 667 664 -3 - Partials 359 366 +7 
Files with missing linesCoverage Δ
src/spatialdata_plot/pl/_datashader.py90.50% <100.00%> (ø)
src/spatialdata_plot/pl/basic.py79.61% <100.00%> (+0.19%)⬆️
src/spatialdata_plot/pl/render_params.py89.02% <100.00%> (+0.22%)⬆️
src/spatialdata_plot/pl/render.py88.16% <92.70%> (+1.11%)⬆️
src/spatialdata_plot/pl/utils.py69.40% <81.81%> (+0.51%)⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`render_labels(element, as_points=True)` with no color crashed:
`instance_id` (the raster's unique values) includes the background label `0`,
which has no centroid, and the literal/no-color color vector is sized to the
raster (not per-instance), so `ax.scatter` got mismatched `c` vs `x`/`y`.
Drop the background label from the rendered instances and align the per-cell
color: for data-driven color the vector is already per-instance and is subset to
match; for the literal/no-color path it is replaced with one na/literal color
per centroid. Data-driven (categorical/continuous) renders are unchanged and
still land exactly on `get_centroids`.
Adds a regression test for the no-color labels case.
…ator
Replace the `regionprops` reduction in `_compute_label_centroids` with an
additive bincount aggregator that streams the labels raster block by block —
one dask chunk (or bounded numpy row-block) in memory at a time — accumulating
per-label `count`/`sum_x`/`sum_y`. This is what makes the feature usable at
Xenium scale:
- Out-of-core: peak memory is one chunk + O(n_labels) accumulators, NOT the
whole raster (measured: 9 MB peak streaming a 268 MB mask). `regionprops`
needs the full array materialized and OOMs on large morphology masks.
- Scales in cell count: 500k+ labels are just array indexing (562k labels in
~1.4 s with 13.5 MB of accumulators); `regionprops`' per-label table does not.
- Faster than `regionprops` (~1.3-1.6x) on in-memory rasters.
- Exact across chunk boundaries (additive reduction) — verified numpy ==
dask-chunked, and identical to `sd.get_centroids`.
- `count` is the cell area, a free by-product (ready for footprint-based dot
sizing).
Drops the `regionprops_table` import; adds `slices_from_chunks`. Adds a unit
test locking the chunk-exact, out-of-core, area-correct behavior.
Note: the chunk loop is currently sequential; parallelizing the per-chunk
partials (dask map_blocks + tree-reduce) is a future speedup.
#705 (measure_obs) merged to main, superseding #703's pre-extraction centroid
block. This merge takes main's utils.py + test_utils.py wholesale (zero #703
delta there) and rewires the labels as_points path onto main's primitive:
- render.py labels branch: drop the deleted `_get_or_compute_centroids` + the
whole cache layer; compute full-resolution (scale0) centroids via main's
`_compute_element_measurements`, and draw them with a transform built from the
*same* scale0 element (`_prepare_transformation(_get_top_data_array(...))`) —
so positions are independent of any rasterization applied to the rendered
`label`. Coerce point_ids to the label dtype so str/object instance ids
(e.g. Xenium readers) align instead of silently reindexing to NaN.
- Net: utils.py == main (the 242-line #703 centroid block + cache layer is
gone); the surviving diff vs main is just the as_points feature.
- Tests: added a non-identity-transform regression test asserting the dots land
at the cells' coordinate-system positions in display space (the guard for the
transform pairing); existing as_points position tests still pass.
Shapes branch unchanged (already intrinsic centroid + trans_data, positionally
aligned to its post-filter color vector).
…ked fields
The fast-mode draw wrapper read cmap/size/alpha/zorder/colorbar/colorbar_params
straight off render_params at both call sites. Pass the render_params dataclass
instead and read them internally; only the genuinely per-branch values
(x/y/color vectors, norm, na_color, transform, adata, palette, col_for_color)
stay explicit. Signature 20->15 params; both as_points call sites lose the
restated render_params plumbing.
Collapse the per-centroid color alignment (drop the unconditional asarray
pre-init and the nested None-guard into a single if/else + ternary), and drop a
redundant np.asarray around the already-ndarray point_ids in the reindex.
Behavior unchanged.
Benchmark showed labels as_points was 16-34x SLOWER than the normal render
because it recomputed full-resolution scale0 centroids while the normal path
downsamples + imshows. Compute centroids on the already-rendered (downsampled)
raster and draw with its trans_data instead: 671k cells goes 19.7s -> 0.88s
(now ~parity with imshow). Centroid error is sub-pixel at display resolution;
position tests updated to display-space within a few-px tolerance.
…gned
show()'s get_extent(exact=True) transforms EVERY shapes/points geometry into
the coordinate system just to take a bounding box - the dominant cost for large
shape collections (~85% of a 5.5M-shape render), unrelated to what is drawn.
Add a self-contained get_extent_fast (drop-in for spatialdata.get_extent) that,
for axis-aligned transforms (scale/flip/90deg/swap + translation - all real
Visium/Xenium data), transforms only the 4 bounding-box corners and reads the
intrinsic bounds vectorised (no per-geometry .apply(is_empty)). Proven identical
to the exact extent for such transforms (spatialdata's own get_extent docstring
notes this); falls back to spatialdata's get_extent for rotation/shear, for
anisotropically-scaled circles (radius->ellipse divergence), and for
images/labels (whose get_extent is already a cheap corner transform).
Measured on real Visium HD render_shapes(as_points=True): 351k 8.5s->1.8s
(4.7x), 5.48M 129s->24s (5.5x); the full (non-as_points) render benefits too.
The whole block is isolated so it can be lifted into spatialdata's get_extent
verbatim (see issue #706). Tests assert it matches get_extent across scale/flip/
rotation/shear for circles and polygons.
@timtreis
timtreisforce-pushed the feat/centroid-scatter-helper branch from 35c8f1f to 57bcf8dCompareJune 9, 2026 23:20
…ompute, underscore)
Follow-up cleanups from review of the get_extent fast path:
- fold _intrinsic_xy_bounds into _element_extent_fast (drop duplicate get_model / geom_type)
- geom.bounds -> geom.total_bounds (C-level union, avoids an Nx4 alloc on large collections)
- batch the four points min/max into one dask.compute
- rename get_extent_fast -> _get_extent_fast (internal helper; matches sibling underscored helpers)
No behavior change; output identical for axis-aligned and rotated/sheared elements.
The datashader shapes canvas sized itself via spatialdata's exact get_extent, transforming every
geometry (O(N)) just for a bounding box -- the same cost _get_extent_fast already removed from
show()'s axis limits, in a second code path. Reuse _element_extent_fast (corner transform for
axis-aligned elements; None -> exact get_extent fallback for rotation/shear), so the result is
pixel-identical and the per-geometry pass is skipped for the common case.
Measured on Curio (69,713 colored shapes): render_shapes 3146 ms -> 1675 ms (1.88x), on top of
the get_extent_fast win already in show(). Mirrors _datashader_canvas_from_dataframe, which
already avoids get_extent for the points path.
…de review)
From the perf-stack review of labels as_points:
- No color column collapsed every dot to a single na_color; now each cell gets a distinct random
colour, matching the mask path's _map_color_seg Case C. Adds a color assertion to the regression test.
- The rasterize drop-filter only ran when a color column was set, so as_points could emit dots at NaN
positions (or drop cells) when rasterization removed labels; extend it to as_points so point ids stay
within the rendered raster.
- ax.scatter autoscales the Normalize in place; copy the shared cmap_params.norm (the shapes path already does).
- render_shapes/render_labels(as_points=True) did not validate size; add the points-path numeric/positive
check so size=-5 / 'big' raise an actionable error instead of a raw matplotlib failure.
Extract the duplicated as_points size-validation block from render_shapes
and render_labels into a shared _validate_as_points_size helper. Defer the
np.asarray(color_vector) conversion in the centroid color path to the only
branch that uses it.
…to feat/centroid-scatter-helper
# Conflicts:
#	tests/pl/test_utils.py
_get_extent_fast already reads get_transformation(element, get_all=True) for
the coordinate-system membership check; pass it through so the fast path does
not re-fetch it per element. The optional kwarg leaves the datashader-canvas
call site unchanged.
_element_extent_fast returned a NaN extent for an element whose geometries are
all empty, which silently poisoned the union in _get_extent_fast (blank axes)
instead of raising spatialdata's clear 'empty collection' error. Guard against
non-finite bounds and fall back. Also trims the verbose extent-block comment and
the scatter/centroid docstrings (net -12 LOC, no behavior change).
@timtreistimtreis changed the title Fast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingFast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extentJun 10, 2026
Four PlotTester baselines: render_shapes/labels(as_points=True) at a base size
and a larger size (size = scatter marker area). Baselines to be generated from CI.
…l tests
Rendered on hatch-test.py3.11-stable; verified dots land at shape/label
centroids, colored by instance_id (labels), larger at size=600.
The empty-shape check ran a per-geometry Python lambda (`.apply(lambda g: not
g.is_empty)`) over every geometry on every shapes render; GeoSeries.is_empty is
GEOS-vectorized. `.is_empty.all()` is identical and ~134x faster on this guard
(665ms -> 5ms at 351k shapes, ~10s -> 80ms at 5.5M).
Shared datashader draw primitive (canvas->aggregate->norm->color-key->shade->
image->colorbar), taking explicit primitives instead of render_params so it can
also serve the as_points centroid path (shapes/labels params use fill_alpha and
lack the density fields). Returns the possibly-recomputed color vectors so the
caller's legend/colorbar stays identical. Verified pixel-identical on
render_points datashader (categorical/continuous/plain/density).
- Route as_points centroids through the shared _datashader_points when method=
'datashader' or above ~500k dots (AS_POINTS_DS_AUTO); matplotlib otherwise.
- No-color labels (one random colour per cell) cannot be aggregated by datashader;
force matplotlib there (warn on explicit method='datashader').
- Fix: shapes as_points drew dots at intrinsic centroid coords while the axes are
in coordinate-system coords, so non-identity transforms misplaced them. Transform
centroids to CS via the element->CS affine for both backends (labels too).
- render_labels gains 'method'; LabelsRenderParams gains 'method'. as_points uses a
fixed datashader reduction ('max', closest to matplotlib) — no user knob.
- shapes non-identity transform regression (the coordinate bug fixed in 5854ae6)
- backend selection: method='datashader' -> datashader image; default -> matplotlib
- no-color labels force matplotlib even with method='datashader' (warns)
- _resolve_as_points_method unit test (threshold/explicit/no-color/empty)
- two test_plot_* visual tests for datashader as_points (baselines from CI)
timtreis added 22 commits June 11, 2026 06:40
Rendered on hatch-test.py3.11-stable; verified datashaded centroids coloured by
category with a categorical legend. Only this test failed on stable CI (missing
baseline); existing datashader baselines unchanged.
_datashader_points' two near-identical reindex/align branches (categorical
source vs colour vector) differed only in which vector they materialize; pick
the source once, then share the reindex/construct. render_points datashader
output verified byte-identical (categorical/continuous/plain/density). -9 LOC.
The datashader as_points canvas was sized to the exact centroid bounding box,
so the outermost centroids sat on the canvas edge and their marker spread was
clipped (half-circles), looking nothing like the matplotlib backend. Grow the
canvas by the spread radius on each side (factor unchanged, so placement still
aligns). render_points keeps pad_for_markers=False -> byte-identical.
…padding fix
The padding fix changed the datashader as_points renders within the comparison
tolerance, so the baselines didn't auto-update. Regenerate from CI so the
committed images reflect the un-clipped output.
CI-rendered (py3.11-stable) after the canvas-padding fix; verified the edge
centroids now render as full circles matching the matplotlib backend.
Restructure the as_points visual tests into matplotlib+datashader pairs that
render identical params (shared helper) for shapes (no color), labels
(instance_id), and labels (categorical). Drop the old inconsistent/stale
baselines; all 6 will be regenerated from CI so the two backends are directly
comparable and look maximally similar.
Positions and sizes align after the canvas-padding fix; dots land at the same
centroids at the same size. Remaining difference is color shading on the
categorical path (datashader modulates alpha by per-pixel count).
Datashader faded single-cell dots (count-driven alpha floor + a second user-alpha
multiply ~= alpha^2), making categorical as_points read much paler than the
matplotlib backend. Add uniform_alpha for the as_points marker mode: a full alpha
floor so each dot is one flat colour at fill_alpha, like a matplotlib marker.
Renamed the as_points flag pad_for_markers -> as_markers (pads canvas + uniform
alpha). render_points unchanged (byte-identical).
…h matplotlib)
CI-rendered (py3.11-stable); the datashader as_points dots now match the
matplotlib markers in colour saturation, position, and size.
…matplotlib
Size (derived, no heuristic): datashader previously rasterized over the centroid
bounding box, so dot display size depended on the canvas/axes extent ratio (shapes
~0.9x, labels ~1.7x matplotlib). Now the canvas spans the same extent as the axes
(like render_points), and the spread radius is set to matplotlib's marker radius
sqrt(s)*dpi/144 (its 'o' marker has diameter sqrt(s)*dpi/72). Result: ds/mpl size
ratio 1.02 +/- 0.03 across element type, size, figure, and dpi.
Continuous colorbar: the spread combined overlapping dots with 'add' (ds_reduction
None -> 'sum'), summing ids and inflating reduction_bounds; marker mode now spreads
with 'max' so overlaps overlay and the colorbar keeps the true range.
render_points unchanged (byte-identical). Replaces the canvas-padding workaround
(the axes-extent canvas already gives edge dots their margin).
… colorbar)
CI-rendered (py3.11-stable): datashader dots now match the matplotlib markers in
size, and the continuous colorbar uses the true value range.
The datashader result is a data-coordinate image that scales with the axes,
while matplotlib markers are fixed in display points. The canvas was sized from
fig.get_size_inches()*dpi (the whole figure), but the axes are smaller (margins,
colorbar), so the image - and every dot - was scaled down: labels (with colorbar)
0.81x matplotlib, shapes 0.88x. Size the as_markers canvas to the axes display box
(ax.get_window_extent()) so 1 canvas px == 1 axes-display px and the sqrt(s)*dpi/144
spread radius matches the marker. Now mean 0.99 +/-5% across sizes/figs/dpi/elements;
render_points untouched (byte-identical). Visual tests render at a non-overlapping
size so the engines' overlap handling (stack vs aggregate) doesn't enter the pairs.
…der canvas
CI-rendered (py3.11-stable). matplotlib and datashader pairs now match in dot
size (the canvas-vs-figure scaling bug is fixed) at a non-overlapping size.
Cleanup pass over the as_points feature (no behavior change):
- trim restated comments/docstrings in _datashader_points, keeping the
load-bearing marker-radius (sqrt(s)*dpi/144) and canvas-vs-axes rationale
- collapse the 3-way min_alpha if/elif/else to a ternary
- reuse the already-computed label extent instead of recomputing get_extent
- np.full instead of list*N for the literal na_color vector
- tighten the as_points test-helper comment
render_points output verified byte-identical; as_points renders unchanged.
Datashader pays off for as_points from ~50k cells (~1.66x faster than the
matplotlib scatter backend, rising to ~1.8x at 1M), so switch over there
instead of 500k. Users can still force either backend via method=.
The shapes as_points block computed its datashader canvas extent via
get_extent (exact=True), which transforms every geometry — on 351k Visium HD
shapes that was ~7s/render and dominated the whole as_points cost, so the fast
mode gave no speedup over full-geometry rendering. Use _element_extent_fast
(corner-transform; identical result for axis-aligned transforms, falls back to
get_extent for rotation/shear), which the PR already added for exactly this.
Real overlay (image + shapes color=gene), as_points:
91k: 7.2s -> 2.0s (3.3x)
351k: 24.3s -> 3.1s (7.3x, vs 22.8s full-geometry)
Output is unchanged: the fast extent is byte-identical for axis-aligned data.
- extract _fast_extent(element, cs) helper; the '_element_extent_fast(...) or
get_extent(...)' idiom was duplicated verbatim at the shapes as_points site
and the datashader-canvas helper
- fix docstrings that still said '~500k' after the threshold dropped to 50k
- trim _render_centroids_as_points docstring and a non-actionable comment
No behavior change: _fast_extent is identical to the inline form (verified
== get_extent on blobs), render_points byte-identical.
@timtreis
timtreis merged commit 8730ff4 into mainJun 14, 2026
7 of 8 checks passed
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #702 (per-panel title fix) and #703 (as_points + fast extent)
into the show() decomposition.
Conflict resolution in src/spatialdata_plot/pl/basic.py:
- #702 up-front title-count validation: kept (after num_panels). The
adjacent axes/panel-count check was dropped here because the
decomposition already relocated it into _plan_panels.
- #702 simplified title selection (dropped the per-panel try/except):
applied to the extracted _finalize_panel helper.
- #703 had no show()-level render-dispatch changes (as_points is
param-driven in render.py, handled inside _render_panel already); its
only show()-level change, get_extent -> _get_extent_fast, auto-merged
into the extent block and consumes _render_panel's `wants` dict.
Also sweeps up two pre-existing #703 lint/type nits in utils.py surfaced
by the merge: _fast_extent docstring (ruff D205) and _get_extent_fast
Any-return (mypy).
Verified: ruff + mypy clean; 109 non-visual show/shapes/labels tests pass.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 21 color helpers + 6 color-only format/uniqueness helpers verbatim from
utils.py into a new sibling module pl/_color.py. Imports flow one way:
_color -> utils (downward, for _get_list/to_hex/_build_alignment_dtype_hint/
_MPL_SINGLE_LETTER_COLORS). render.py, basic.py and _datashader.py are repointed
to import color symbols from _color.
The two validators still in utils (_type_check_params, _validate_graph_render_params)
use color symbols; they carry temporary function-local imports of _color (cycle-safe)
until they move to _validate.py in the next commit, where these become top-level imports.
No behavior change (verbatim move; all-private except set_zero_in_cmap_to_transparent,
which is not re-exported). Verified: no import cycle; 410 non-visual tests pass;
ruff + ruff-format clean. Pre-existing #703/#705 mypy/ruff debt in utils.py is
unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 17 validation/type-check functions verbatim from utils.py into a new sibling
module pl/_validate.py. Imports flow one way: _validate -> _color (_is_color_like,
_prepare_cmap_norm, _get_colors_for_categorical_obs, now top-level) and
_validate -> utils (downward). The temporary function-local _color imports added in
the previous commit are hoisted to top-level here and removed from utils.
render.py and basic.py repointed to _validate.
Completes the utils.py split (#696): utils 4918 -> 1311 lines, with
_geometry / _datashader / _color / _validate as single-concern siblings.
No import cycle anywhere. No behavior change (verbatim moves; all-private).
Verified: 410 non-visual tests pass; ruff + ruff-format clean. Pre-existing
#703/#705 mypy debt in utils.py is unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #714 (show() decomposition) into the utils.py split. Only utils.py
conflicted:
- import block: dropped the now-unused `_locate_value` import (it moved to
_color.py with the color code that uses it); kept main's `_locate_value`
out of utils.
- `_fast_extent` docstring: took main's #714 version (D205 fix).
basic.py auto-merged: #714's decomposed show()/helpers now import color and
validation symbols from _color/_validate (the split's repoints), not utils.
Bonus: merging #714 brings its fixes for the pre-existing #703/#705 debt
(_resolve_measure_table str-return, _get_extent_fast Any-return, _fast_extent
D205), so the branch is now fully ruff + mypy clean (no --no-verify).
Verified: no import cycle; ruff + ruff-format + mypy all pass; 410 non-visual
tests pass.
@timtreis
timtreis deleted the feat/centroid-scatter-helper branch July 10, 2026 11:11
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@timtreis@codecov-commenter
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent - #703

Merged
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper
Jun 14, 2026
Merged

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent#703
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper

Conversation

@timtreis

@timtreistimtreis commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

Fast rendering for shapes/labels, in spatialdata-plot's API:

  1. as_points=True — draw each cell as a dot at its centroid instead of its full geometry/mask (the squidpy spatial_scatter idea). matplotlib by default; auto-switches to datashader above ~500k dots, or with method="datashader".
  2. Fast axis-aligned extentpl.show() skips transforming every geometry to size the axes when the element's transform is axis-aligned (falls back to get_extent otherwise).
sdata.pl.render_shapes("cells", color="cell_type", as_points=True).pl.show()
sdata.pl.render_labels("cells", color="leiden", as_points=True, method="datashader").pl.show()

Notes

  • Default (as_points=False) output is unchanged vs main.
  • render_points' datashader pipeline is extracted into a shared _datashader_points (byte-identical) and reused by the centroid path.
  • Datashader can't represent the random-per-cell colours of uncolored labels, so that case stays matplotlib (with a warning).
  • Centroids are transformed to coordinate-system coords, fixing dot placement under non-identity transforms.

Tests

Position parity vs matplotlib (incl. non-identity transform), backend selection, no-color fallback, and test_plot_* visual baselines for matplotlib + datashader as_points (continuous, categorical, no-color).

… helper
Infrastructure for an upcoming "render cells as centroid points" fast mode
(no user-facing render option yet).
Phase 0 — shared scatter primitive:
- Extract `_scatter_points(ax, x, y, color_vector, ...)` from `_render_points`'s
matplotlib branch; `_render_points` now calls it. Byte-identical output
(verified vs main on categorical and continuous point renders). This is the
reuse seam the fast mode will draw through.
Phase 1 — centroid + caching core (headless, fully unit-tested):
- `_compute_element_centroids` / `_compute_label_centroids`: per-instance
centroids in a coordinate system. Shapes use spatialdata's vectorized
`get_centroids`; labels use skimage `regionprops` (the per-label reduction is
orders of magnitude faster than `get_centroids` on rasters), mapped onto the
raster's intrinsic coordinate arrays so it reproduces `get_centroids` exactly
(incl. the pixel-center 0.5 offset) then transformed to the target CS.
- `_get_or_compute_centroids`: reuses/persists centroids via the squidpy
convention. A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted
as the cells' locations; otherwise centroids are computed and written back into
the annotating table's `obsm["spatial"]` with a coordinate-system provenance
marker in `uns`, so later renders are instant. Reads run before writes, so a
valid existing cache is reused rather than clobbered; an incompatible existing
`obsm["spatial"]` is never overwritten; the cache is invalidated when the
requested coordinate system differs.
- Tests: shapes/labels centroids match `get_centroids`; cache round-trip +
provenance; CS invalidation; pre-existing obsm trusted; no-table compute path;
`cache=False` writes nothing.
… provenance
Cleanup from /simplify (no behavioral change):
- Extract `_region_mask_and_keys(table, element)` used by both read and write,
removing the duplicated `get_table_keys` + O(n_obs) `region_key`-string-cast
mask that was computed twice per cold call.
- Read path: validate shape on the raw obsm array and cast only the masked
subset to float, instead of casting the whole `obsm["spatial"]` on every
cache hit (the hot path).
- Write path: coerce a non-dict `uns["spatialdata_plot"]` instead of early
returning after `obsm` was already mutated, so obsm and the provenance marker
are always written together (no half-write).
- Drop the dead `"key"` provenance field (constant, never read back).
- Rename the misleading `table` local (held a table *name*) in
`_get_or_compute_centroids`.
Refactor the centroid cache to store element-*intrinsic* coordinates and
transform to the render coordinate system on demand, instead of caching
coords already mapped into one coordinate system. Decisions from design pass:
- Intrinsic storage: one `obsm["spatial"]` cache serves every coordinate
system (proven equivalent to per-CS computation). `_compute_element_centroids`
returns intrinsic coords (shapes via shapely `.centroid`, labels via
`regionprops`); `_centroids_to_coordinate_system` maps them to the requested
CS via the element's transform; `_get_or_compute_centroids` reads/computes
intrinsic then transforms on return.
- Provenance records `{n, scale_level}` (no coordinate system). Cache is
invalidated when the region's instance count changes (cells added/removed),
not on CS change.
- A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted as the
cells' intrinsic locations and transformed to the render CS.
- Exhaustive model dispatch: shapes and 2D labels supported; other element
types raise NotImplementedError.
- Labels are reduced at full resolution (scale0).
Drops the now-unused `get_centroids` import; adds `ShapesModel`. Tests updated:
parametrized shapes/labels match `get_centroids`; new coordinate-system-
independence test (one cache, two CS); staleness-by-instance-count; trusted
pre-existing obsm; unsupported-type rejection.
…e import, shared obsm gate
Cleanup from /simplify:
- `_centroids_to_coordinate_system` ran `PointsModel.parse` + a dask
`transform(...).compute()` round trip on every call, including cache hits
(~19 ms fixed floor, ~100 ms at 1M cells) — defeating the cache. Replace with
`to_affine_matrix` + a plain numpy matmul: numerically identical (verified
against the dask path for multiple coordinate systems), ~80-140x faster, and
it removes the private-API import `spatialdata._core.operations.transform`
(a repo non-negotiable) plus the now-unused `PointsModel`.
- Widen `_transformable_raster` -> `_transform_carrier` to accept any element
(rasters -> scale0, others as-is), dropping the `isinstance` branch in
`_centroids_to_coordinate_system`.
- Extract `_valid_spatial_obsm(arr, n_obs)` shared by the read and write paths,
reconciling their previously divergent obsm-shape checks (read accepted >=2
columns, write required exactly 2) so they cannot drift.
`render_shapes(..., as_points=True)` and `render_labels(..., as_points=True)`
draw one dot per cell at its centroid instead of the full geometry / rasterized
mask — a large speedup when only cell location matters. New `size=` controls the
marker size.
- Shared `_render_centroids_as_points` draws the scatter (via `_scatter_points`)
and the legend/colorbar. The per-cell color vector is the *same* one the
geometry/raster path computes (`_set_color_source_vec`), so colors match the
full rendering exactly; only the apply step (scatter vs patches/imshow) differs.
- Shapes: centroids from shapely `.centroid` of the (filtered) geometry,
positionally aligned to the color vector, drawn in intrinsic coords via the
element transform. Labels: centroids from `_get_or_compute_centroids`
(regionprops, fast) reindexed to `instance_id`. Positions verified identical to
`sd.get_centroids`.
- `as_points` short-circuits before the geometry/raster path; outline_*, shape
(shapes) and contour_px, outline_* (labels) are ignored with an info log.
- Default (`as_points=False`) output is byte-identical to main.
Tests: non-visual checks that centroids land exactly on `get_centroids` for both
element types and that outline/shape are ignored without error.
Note: as_points currently always uses the matplotlib scatter backend; routing
through datashader for very large cell counts (and persisting the obsm cache to
the user's object rather than show()'s working copy) are follow-ups.
@timtreistimtreis changed the title Centroid extraction + squidpy obsm["spatial"] caching; shared scatter helperFast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingJun 8, 2026
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.08046% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.89%. Comparing base (b370b1f) to head (882d208).

Files with missing linesPatch %Lines
src/spatialdata_plot/pl/utils.py81.81%6 Missing and 6 partials ⚠️
src/spatialdata_plot/pl/render.py92.70%4 Missing and 3 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #703 +/- ##
==========================================
+ Coverage 76.28% 76.89% +0.60% 
==========================================
Files 14 14 Lines 4327 4458 +131 Branches 1006 1035 +29 ==========================================
+ Hits 3301 3428 +127 + Misses 667 664 -3 - Partials 359 366 +7 
Files with missing linesCoverage Δ
src/spatialdata_plot/pl/_datashader.py90.50% <100.00%> (ø)
src/spatialdata_plot/pl/basic.py79.61% <100.00%> (+0.19%)⬆️
src/spatialdata_plot/pl/render_params.py89.02% <100.00%> (+0.22%)⬆️
src/spatialdata_plot/pl/render.py88.16% <92.70%> (+1.11%)⬆️
src/spatialdata_plot/pl/utils.py69.40% <81.81%> (+0.51%)⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`render_labels(element, as_points=True)` with no color crashed:
`instance_id` (the raster's unique values) includes the background label `0`,
which has no centroid, and the literal/no-color color vector is sized to the
raster (not per-instance), so `ax.scatter` got mismatched `c` vs `x`/`y`.
Drop the background label from the rendered instances and align the per-cell
color: for data-driven color the vector is already per-instance and is subset to
match; for the literal/no-color path it is replaced with one na/literal color
per centroid. Data-driven (categorical/continuous) renders are unchanged and
still land exactly on `get_centroids`.
Adds a regression test for the no-color labels case.
…ator
Replace the `regionprops` reduction in `_compute_label_centroids` with an
additive bincount aggregator that streams the labels raster block by block —
one dask chunk (or bounded numpy row-block) in memory at a time — accumulating
per-label `count`/`sum_x`/`sum_y`. This is what makes the feature usable at
Xenium scale:
- Out-of-core: peak memory is one chunk + O(n_labels) accumulators, NOT the
whole raster (measured: 9 MB peak streaming a 268 MB mask). `regionprops`
needs the full array materialized and OOMs on large morphology masks.
- Scales in cell count: 500k+ labels are just array indexing (562k labels in
~1.4 s with 13.5 MB of accumulators); `regionprops`' per-label table does not.
- Faster than `regionprops` (~1.3-1.6x) on in-memory rasters.
- Exact across chunk boundaries (additive reduction) — verified numpy ==
dask-chunked, and identical to `sd.get_centroids`.
- `count` is the cell area, a free by-product (ready for footprint-based dot
sizing).
Drops the `regionprops_table` import; adds `slices_from_chunks`. Adds a unit
test locking the chunk-exact, out-of-core, area-correct behavior.
Note: the chunk loop is currently sequential; parallelizing the per-chunk
partials (dask map_blocks + tree-reduce) is a future speedup.
#705 (measure_obs) merged to main, superseding #703's pre-extraction centroid
block. This merge takes main's utils.py + test_utils.py wholesale (zero #703
delta there) and rewires the labels as_points path onto main's primitive:
- render.py labels branch: drop the deleted `_get_or_compute_centroids` + the
whole cache layer; compute full-resolution (scale0) centroids via main's
`_compute_element_measurements`, and draw them with a transform built from the
*same* scale0 element (`_prepare_transformation(_get_top_data_array(...))`) —
so positions are independent of any rasterization applied to the rendered
`label`. Coerce point_ids to the label dtype so str/object instance ids
(e.g. Xenium readers) align instead of silently reindexing to NaN.
- Net: utils.py == main (the 242-line #703 centroid block + cache layer is
gone); the surviving diff vs main is just the as_points feature.
- Tests: added a non-identity-transform regression test asserting the dots land
at the cells' coordinate-system positions in display space (the guard for the
transform pairing); existing as_points position tests still pass.
Shapes branch unchanged (already intrinsic centroid + trans_data, positionally
aligned to its post-filter color vector).
…ked fields
The fast-mode draw wrapper read cmap/size/alpha/zorder/colorbar/colorbar_params
straight off render_params at both call sites. Pass the render_params dataclass
instead and read them internally; only the genuinely per-branch values
(x/y/color vectors, norm, na_color, transform, adata, palette, col_for_color)
stay explicit. Signature 20->15 params; both as_points call sites lose the
restated render_params plumbing.
Collapse the per-centroid color alignment (drop the unconditional asarray
pre-init and the nested None-guard into a single if/else + ternary), and drop a
redundant np.asarray around the already-ndarray point_ids in the reindex.
Behavior unchanged.
Benchmark showed labels as_points was 16-34x SLOWER than the normal render
because it recomputed full-resolution scale0 centroids while the normal path
downsamples + imshows. Compute centroids on the already-rendered (downsampled)
raster and draw with its trans_data instead: 671k cells goes 19.7s -> 0.88s
(now ~parity with imshow). Centroid error is sub-pixel at display resolution;
position tests updated to display-space within a few-px tolerance.
…gned
show()'s get_extent(exact=True) transforms EVERY shapes/points geometry into
the coordinate system just to take a bounding box - the dominant cost for large
shape collections (~85% of a 5.5M-shape render), unrelated to what is drawn.
Add a self-contained get_extent_fast (drop-in for spatialdata.get_extent) that,
for axis-aligned transforms (scale/flip/90deg/swap + translation - all real
Visium/Xenium data), transforms only the 4 bounding-box corners and reads the
intrinsic bounds vectorised (no per-geometry .apply(is_empty)). Proven identical
to the exact extent for such transforms (spatialdata's own get_extent docstring
notes this); falls back to spatialdata's get_extent for rotation/shear, for
anisotropically-scaled circles (radius->ellipse divergence), and for
images/labels (whose get_extent is already a cheap corner transform).
Measured on real Visium HD render_shapes(as_points=True): 351k 8.5s->1.8s
(4.7x), 5.48M 129s->24s (5.5x); the full (non-as_points) render benefits too.
The whole block is isolated so it can be lifted into spatialdata's get_extent
verbatim (see issue #706). Tests assert it matches get_extent across scale/flip/
rotation/shear for circles and polygons.
@timtreis
timtreisforce-pushed the feat/centroid-scatter-helper branch from 35c8f1f to 57bcf8dCompareJune 9, 2026 23:20
…ompute, underscore)
Follow-up cleanups from review of the get_extent fast path:
- fold _intrinsic_xy_bounds into _element_extent_fast (drop duplicate get_model / geom_type)
- geom.bounds -> geom.total_bounds (C-level union, avoids an Nx4 alloc on large collections)
- batch the four points min/max into one dask.compute
- rename get_extent_fast -> _get_extent_fast (internal helper; matches sibling underscored helpers)
No behavior change; output identical for axis-aligned and rotated/sheared elements.
The datashader shapes canvas sized itself via spatialdata's exact get_extent, transforming every
geometry (O(N)) just for a bounding box -- the same cost _get_extent_fast already removed from
show()'s axis limits, in a second code path. Reuse _element_extent_fast (corner transform for
axis-aligned elements; None -> exact get_extent fallback for rotation/shear), so the result is
pixel-identical and the per-geometry pass is skipped for the common case.
Measured on Curio (69,713 colored shapes): render_shapes 3146 ms -> 1675 ms (1.88x), on top of
the get_extent_fast win already in show(). Mirrors _datashader_canvas_from_dataframe, which
already avoids get_extent for the points path.
…de review)
From the perf-stack review of labels as_points:
- No color column collapsed every dot to a single na_color; now each cell gets a distinct random
colour, matching the mask path's _map_color_seg Case C. Adds a color assertion to the regression test.
- The rasterize drop-filter only ran when a color column was set, so as_points could emit dots at NaN
positions (or drop cells) when rasterization removed labels; extend it to as_points so point ids stay
within the rendered raster.
- ax.scatter autoscales the Normalize in place; copy the shared cmap_params.norm (the shapes path already does).
- render_shapes/render_labels(as_points=True) did not validate size; add the points-path numeric/positive
check so size=-5 / 'big' raise an actionable error instead of a raw matplotlib failure.
Extract the duplicated as_points size-validation block from render_shapes
and render_labels into a shared _validate_as_points_size helper. Defer the
np.asarray(color_vector) conversion in the centroid color path to the only
branch that uses it.
…to feat/centroid-scatter-helper
# Conflicts:
#	tests/pl/test_utils.py
_get_extent_fast already reads get_transformation(element, get_all=True) for
the coordinate-system membership check; pass it through so the fast path does
not re-fetch it per element. The optional kwarg leaves the datashader-canvas
call site unchanged.
_element_extent_fast returned a NaN extent for an element whose geometries are
all empty, which silently poisoned the union in _get_extent_fast (blank axes)
instead of raising spatialdata's clear 'empty collection' error. Guard against
non-finite bounds and fall back. Also trims the verbose extent-block comment and
the scatter/centroid docstrings (net -12 LOC, no behavior change).
@timtreistimtreis changed the title Fast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingFast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extentJun 10, 2026
Four PlotTester baselines: render_shapes/labels(as_points=True) at a base size
and a larger size (size = scatter marker area). Baselines to be generated from CI.
…l tests
Rendered on hatch-test.py3.11-stable; verified dots land at shape/label
centroids, colored by instance_id (labels), larger at size=600.
The empty-shape check ran a per-geometry Python lambda (`.apply(lambda g: not
g.is_empty)`) over every geometry on every shapes render; GeoSeries.is_empty is
GEOS-vectorized. `.is_empty.all()` is identical and ~134x faster on this guard
(665ms -> 5ms at 351k shapes, ~10s -> 80ms at 5.5M).
Shared datashader draw primitive (canvas->aggregate->norm->color-key->shade->
image->colorbar), taking explicit primitives instead of render_params so it can
also serve the as_points centroid path (shapes/labels params use fill_alpha and
lack the density fields). Returns the possibly-recomputed color vectors so the
caller's legend/colorbar stays identical. Verified pixel-identical on
render_points datashader (categorical/continuous/plain/density).
- Route as_points centroids through the shared _datashader_points when method=
'datashader' or above ~500k dots (AS_POINTS_DS_AUTO); matplotlib otherwise.
- No-color labels (one random colour per cell) cannot be aggregated by datashader;
force matplotlib there (warn on explicit method='datashader').
- Fix: shapes as_points drew dots at intrinsic centroid coords while the axes are
in coordinate-system coords, so non-identity transforms misplaced them. Transform
centroids to CS via the element->CS affine for both backends (labels too).
- render_labels gains 'method'; LabelsRenderParams gains 'method'. as_points uses a
fixed datashader reduction ('max', closest to matplotlib) — no user knob.
- shapes non-identity transform regression (the coordinate bug fixed in 5854ae6)
- backend selection: method='datashader' -> datashader image; default -> matplotlib
- no-color labels force matplotlib even with method='datashader' (warns)
- _resolve_as_points_method unit test (threshold/explicit/no-color/empty)
- two test_plot_* visual tests for datashader as_points (baselines from CI)
timtreis added 22 commits June 11, 2026 06:40
Rendered on hatch-test.py3.11-stable; verified datashaded centroids coloured by
category with a categorical legend. Only this test failed on stable CI (missing
baseline); existing datashader baselines unchanged.
_datashader_points' two near-identical reindex/align branches (categorical
source vs colour vector) differed only in which vector they materialize; pick
the source once, then share the reindex/construct. render_points datashader
output verified byte-identical (categorical/continuous/plain/density). -9 LOC.
The datashader as_points canvas was sized to the exact centroid bounding box,
so the outermost centroids sat on the canvas edge and their marker spread was
clipped (half-circles), looking nothing like the matplotlib backend. Grow the
canvas by the spread radius on each side (factor unchanged, so placement still
aligns). render_points keeps pad_for_markers=False -> byte-identical.
…padding fix
The padding fix changed the datashader as_points renders within the comparison
tolerance, so the baselines didn't auto-update. Regenerate from CI so the
committed images reflect the un-clipped output.
CI-rendered (py3.11-stable) after the canvas-padding fix; verified the edge
centroids now render as full circles matching the matplotlib backend.
Restructure the as_points visual tests into matplotlib+datashader pairs that
render identical params (shared helper) for shapes (no color), labels
(instance_id), and labels (categorical). Drop the old inconsistent/stale
baselines; all 6 will be regenerated from CI so the two backends are directly
comparable and look maximally similar.
Positions and sizes align after the canvas-padding fix; dots land at the same
centroids at the same size. Remaining difference is color shading on the
categorical path (datashader modulates alpha by per-pixel count).
Datashader faded single-cell dots (count-driven alpha floor + a second user-alpha
multiply ~= alpha^2), making categorical as_points read much paler than the
matplotlib backend. Add uniform_alpha for the as_points marker mode: a full alpha
floor so each dot is one flat colour at fill_alpha, like a matplotlib marker.
Renamed the as_points flag pad_for_markers -> as_markers (pads canvas + uniform
alpha). render_points unchanged (byte-identical).
…h matplotlib)
CI-rendered (py3.11-stable); the datashader as_points dots now match the
matplotlib markers in colour saturation, position, and size.
…matplotlib
Size (derived, no heuristic): datashader previously rasterized over the centroid
bounding box, so dot display size depended on the canvas/axes extent ratio (shapes
~0.9x, labels ~1.7x matplotlib). Now the canvas spans the same extent as the axes
(like render_points), and the spread radius is set to matplotlib's marker radius
sqrt(s)*dpi/144 (its 'o' marker has diameter sqrt(s)*dpi/72). Result: ds/mpl size
ratio 1.02 +/- 0.03 across element type, size, figure, and dpi.
Continuous colorbar: the spread combined overlapping dots with 'add' (ds_reduction
None -> 'sum'), summing ids and inflating reduction_bounds; marker mode now spreads
with 'max' so overlaps overlay and the colorbar keeps the true range.
render_points unchanged (byte-identical). Replaces the canvas-padding workaround
(the axes-extent canvas already gives edge dots their margin).
… colorbar)
CI-rendered (py3.11-stable): datashader dots now match the matplotlib markers in
size, and the continuous colorbar uses the true value range.
The datashader result is a data-coordinate image that scales with the axes,
while matplotlib markers are fixed in display points. The canvas was sized from
fig.get_size_inches()*dpi (the whole figure), but the axes are smaller (margins,
colorbar), so the image - and every dot - was scaled down: labels (with colorbar)
0.81x matplotlib, shapes 0.88x. Size the as_markers canvas to the axes display box
(ax.get_window_extent()) so 1 canvas px == 1 axes-display px and the sqrt(s)*dpi/144
spread radius matches the marker. Now mean 0.99 +/-5% across sizes/figs/dpi/elements;
render_points untouched (byte-identical). Visual tests render at a non-overlapping
size so the engines' overlap handling (stack vs aggregate) doesn't enter the pairs.
…der canvas
CI-rendered (py3.11-stable). matplotlib and datashader pairs now match in dot
size (the canvas-vs-figure scaling bug is fixed) at a non-overlapping size.
Cleanup pass over the as_points feature (no behavior change):
- trim restated comments/docstrings in _datashader_points, keeping the
load-bearing marker-radius (sqrt(s)*dpi/144) and canvas-vs-axes rationale
- collapse the 3-way min_alpha if/elif/else to a ternary
- reuse the already-computed label extent instead of recomputing get_extent
- np.full instead of list*N for the literal na_color vector
- tighten the as_points test-helper comment
render_points output verified byte-identical; as_points renders unchanged.
Datashader pays off for as_points from ~50k cells (~1.66x faster than the
matplotlib scatter backend, rising to ~1.8x at 1M), so switch over there
instead of 500k. Users can still force either backend via method=.
The shapes as_points block computed its datashader canvas extent via
get_extent (exact=True), which transforms every geometry — on 351k Visium HD
shapes that was ~7s/render and dominated the whole as_points cost, so the fast
mode gave no speedup over full-geometry rendering. Use _element_extent_fast
(corner-transform; identical result for axis-aligned transforms, falls back to
get_extent for rotation/shear), which the PR already added for exactly this.
Real overlay (image + shapes color=gene), as_points:
91k: 7.2s -> 2.0s (3.3x)
351k: 24.3s -> 3.1s (7.3x, vs 22.8s full-geometry)
Output is unchanged: the fast extent is byte-identical for axis-aligned data.
- extract _fast_extent(element, cs) helper; the '_element_extent_fast(...) or
get_extent(...)' idiom was duplicated verbatim at the shapes as_points site
and the datashader-canvas helper
- fix docstrings that still said '~500k' after the threshold dropped to 50k
- trim _render_centroids_as_points docstring and a non-actionable comment
No behavior change: _fast_extent is identical to the inline form (verified
== get_extent on blobs), render_points byte-identical.
@timtreis
timtreis merged commit 8730ff4 into mainJun 14, 2026
7 of 8 checks passed
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #702 (per-panel title fix) and #703 (as_points + fast extent)
into the show() decomposition.
Conflict resolution in src/spatialdata_plot/pl/basic.py:
- #702 up-front title-count validation: kept (after num_panels). The
adjacent axes/panel-count check was dropped here because the
decomposition already relocated it into _plan_panels.
- #702 simplified title selection (dropped the per-panel try/except):
applied to the extracted _finalize_panel helper.
- #703 had no show()-level render-dispatch changes (as_points is
param-driven in render.py, handled inside _render_panel already); its
only show()-level change, get_extent -> _get_extent_fast, auto-merged
into the extent block and consumes _render_panel's `wants` dict.
Also sweeps up two pre-existing #703 lint/type nits in utils.py surfaced
by the merge: _fast_extent docstring (ruff D205) and _get_extent_fast
Any-return (mypy).
Verified: ruff + mypy clean; 109 non-visual show/shapes/labels tests pass.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 21 color helpers + 6 color-only format/uniqueness helpers verbatim from
utils.py into a new sibling module pl/_color.py. Imports flow one way:
_color -> utils (downward, for _get_list/to_hex/_build_alignment_dtype_hint/
_MPL_SINGLE_LETTER_COLORS). render.py, basic.py and _datashader.py are repointed
to import color symbols from _color.
The two validators still in utils (_type_check_params, _validate_graph_render_params)
use color symbols; they carry temporary function-local imports of _color (cycle-safe)
until they move to _validate.py in the next commit, where these become top-level imports.
No behavior change (verbatim move; all-private except set_zero_in_cmap_to_transparent,
which is not re-exported). Verified: no import cycle; 410 non-visual tests pass;
ruff + ruff-format clean. Pre-existing #703/#705 mypy/ruff debt in utils.py is
unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 17 validation/type-check functions verbatim from utils.py into a new sibling
module pl/_validate.py. Imports flow one way: _validate -> _color (_is_color_like,
_prepare_cmap_norm, _get_colors_for_categorical_obs, now top-level) and
_validate -> utils (downward). The temporary function-local _color imports added in
the previous commit are hoisted to top-level here and removed from utils.
render.py and basic.py repointed to _validate.
Completes the utils.py split (#696): utils 4918 -> 1311 lines, with
_geometry / _datashader / _color / _validate as single-concern siblings.
No import cycle anywhere. No behavior change (verbatim moves; all-private).
Verified: 410 non-visual tests pass; ruff + ruff-format clean. Pre-existing
#703/#705 mypy debt in utils.py is unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #714 (show() decomposition) into the utils.py split. Only utils.py
conflicted:
- import block: dropped the now-unused `_locate_value` import (it moved to
_color.py with the color code that uses it); kept main's `_locate_value`
out of utils.
- `_fast_extent` docstring: took main's #714 version (D205 fix).
basic.py auto-merged: #714's decomposed show()/helpers now import color and
validation symbols from _color/_validate (the split's repoints), not utils.
Bonus: merging #714 brings its fixes for the pre-existing #703/#705 debt
(_resolve_measure_table str-return, _get_extent_fast Any-return, _fast_extent
D205), so the branch is now fully ruff + mypy clean (no --no-verify).
Verified: no import cycle; ruff + ruff-format + mypy all pass; 410 non-visual
tests pass.
@timtreis
timtreis deleted the feat/centroid-scatter-helper branch July 10, 2026 11:11
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@timtreis@codecov-commenter
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent - #703

Merged
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper
Jun 14, 2026
Merged

Fast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extent#703
timtreis merged 51 commits into
mainfrom
feat/centroid-scatter-helper

Conversation

@timtreis

@timtreistimtreis commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

Fast rendering for shapes/labels, in spatialdata-plot's API:

  1. as_points=True — draw each cell as a dot at its centroid instead of its full geometry/mask (the squidpy spatial_scatter idea). matplotlib by default; auto-switches to datashader above ~500k dots, or with method="datashader".
  2. Fast axis-aligned extentpl.show() skips transforming every geometry to size the axes when the element's transform is axis-aligned (falls back to get_extent otherwise).
sdata.pl.render_shapes("cells", color="cell_type", as_points=True).pl.show()
sdata.pl.render_labels("cells", color="leiden", as_points=True, method="datashader").pl.show()

Notes

  • Default (as_points=False) output is unchanged vs main.
  • render_points' datashader pipeline is extracted into a shared _datashader_points (byte-identical) and reused by the centroid path.
  • Datashader can't represent the random-per-cell colours of uncolored labels, so that case stays matplotlib (with a warning).
  • Centroids are transformed to coordinate-system coords, fixing dot placement under non-identity transforms.

Tests

Position parity vs matplotlib (incl. non-identity transform), backend selection, no-color fallback, and test_plot_* visual baselines for matplotlib + datashader as_points (continuous, categorical, no-color).

… helper
Infrastructure for an upcoming "render cells as centroid points" fast mode
(no user-facing render option yet).
Phase 0 — shared scatter primitive:
- Extract `_scatter_points(ax, x, y, color_vector, ...)` from `_render_points`'s
matplotlib branch; `_render_points` now calls it. Byte-identical output
(verified vs main on categorical and continuous point renders). This is the
reuse seam the fast mode will draw through.
Phase 1 — centroid + caching core (headless, fully unit-tested):
- `_compute_element_centroids` / `_compute_label_centroids`: per-instance
centroids in a coordinate system. Shapes use spatialdata's vectorized
`get_centroids`; labels use skimage `regionprops` (the per-label reduction is
orders of magnitude faster than `get_centroids` on rasters), mapped onto the
raster's intrinsic coordinate arrays so it reproduces `get_centroids` exactly
(incl. the pixel-center 0.5 offset) then transformed to the target CS.
- `_get_or_compute_centroids`: reuses/persists centroids via the squidpy
convention. A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted
as the cells' locations; otherwise centroids are computed and written back into
the annotating table's `obsm["spatial"]` with a coordinate-system provenance
marker in `uns`, so later renders are instant. Reads run before writes, so a
valid existing cache is reused rather than clobbered; an incompatible existing
`obsm["spatial"]` is never overwritten; the cache is invalidated when the
requested coordinate system differs.
- Tests: shapes/labels centroids match `get_centroids`; cache round-trip +
provenance; CS invalidation; pre-existing obsm trusted; no-table compute path;
`cache=False` writes nothing.
… provenance
Cleanup from /simplify (no behavioral change):
- Extract `_region_mask_and_keys(table, element)` used by both read and write,
removing the duplicated `get_table_keys` + O(n_obs) `region_key`-string-cast
mask that was computed twice per cold call.
- Read path: validate shape on the raw obsm array and cast only the masked
subset to float, instead of casting the whole `obsm["spatial"]` on every
cache hit (the hot path).
- Write path: coerce a non-dict `uns["spatialdata_plot"]` instead of early
returning after `obsm` was already mutated, so obsm and the provenance marker
are always written together (no half-write).
- Drop the dead `"key"` provenance field (constant, never read back).
- Rename the misleading `table` local (held a table *name*) in
`_get_or_compute_centroids`.
Refactor the centroid cache to store element-*intrinsic* coordinates and
transform to the render coordinate system on demand, instead of caching
coords already mapped into one coordinate system. Decisions from design pass:
- Intrinsic storage: one `obsm["spatial"]` cache serves every coordinate
system (proven equivalent to per-CS computation). `_compute_element_centroids`
returns intrinsic coords (shapes via shapely `.centroid`, labels via
`regionprops`); `_centroids_to_coordinate_system` maps them to the requested
CS via the element's transform; `_get_or_compute_centroids` reads/computes
intrinsic then transforms on return.
- Provenance records `{n, scale_level}` (no coordinate system). Cache is
invalidated when the region's instance count changes (cells added/removed),
not on CS change.
- A pre-existing `obsm["spatial"]` (loader/user-provided) is trusted as the
cells' intrinsic locations and transformed to the render CS.
- Exhaustive model dispatch: shapes and 2D labels supported; other element
types raise NotImplementedError.
- Labels are reduced at full resolution (scale0).
Drops the now-unused `get_centroids` import; adds `ShapesModel`. Tests updated:
parametrized shapes/labels match `get_centroids`; new coordinate-system-
independence test (one cache, two CS); staleness-by-instance-count; trusted
pre-existing obsm; unsupported-type rejection.
…e import, shared obsm gate
Cleanup from /simplify:
- `_centroids_to_coordinate_system` ran `PointsModel.parse` + a dask
`transform(...).compute()` round trip on every call, including cache hits
(~19 ms fixed floor, ~100 ms at 1M cells) — defeating the cache. Replace with
`to_affine_matrix` + a plain numpy matmul: numerically identical (verified
against the dask path for multiple coordinate systems), ~80-140x faster, and
it removes the private-API import `spatialdata._core.operations.transform`
(a repo non-negotiable) plus the now-unused `PointsModel`.
- Widen `_transformable_raster` -> `_transform_carrier` to accept any element
(rasters -> scale0, others as-is), dropping the `isinstance` branch in
`_centroids_to_coordinate_system`.
- Extract `_valid_spatial_obsm(arr, n_obs)` shared by the read and write paths,
reconciling their previously divergent obsm-shape checks (read accepted >=2
columns, write required exactly 2) so they cannot drift.
`render_shapes(..., as_points=True)` and `render_labels(..., as_points=True)`
draw one dot per cell at its centroid instead of the full geometry / rasterized
mask — a large speedup when only cell location matters. New `size=` controls the
marker size.
- Shared `_render_centroids_as_points` draws the scatter (via `_scatter_points`)
and the legend/colorbar. The per-cell color vector is the *same* one the
geometry/raster path computes (`_set_color_source_vec`), so colors match the
full rendering exactly; only the apply step (scatter vs patches/imshow) differs.
- Shapes: centroids from shapely `.centroid` of the (filtered) geometry,
positionally aligned to the color vector, drawn in intrinsic coords via the
element transform. Labels: centroids from `_get_or_compute_centroids`
(regionprops, fast) reindexed to `instance_id`. Positions verified identical to
`sd.get_centroids`.
- `as_points` short-circuits before the geometry/raster path; outline_*, shape
(shapes) and contour_px, outline_* (labels) are ignored with an info log.
- Default (`as_points=False`) output is byte-identical to main.
Tests: non-visual checks that centroids land exactly on `get_centroids` for both
element types and that outline/shape are ignored without error.
Note: as_points currently always uses the matplotlib scatter backend; routing
through datashader for very large cell counts (and persisting the obsm cache to
the user's object rather than show()'s working copy) are follow-ups.
@timtreistimtreis changed the title Centroid extraction + squidpy obsm["spatial"] caching; shared scatter helperFast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingJun 8, 2026
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.08046% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.89%. Comparing base (b370b1f) to head (882d208).

Files with missing linesPatch %Lines
src/spatialdata_plot/pl/utils.py81.81%6 Missing and 6 partials ⚠️
src/spatialdata_plot/pl/render.py92.70%4 Missing and 3 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #703 +/- ##
==========================================
+ Coverage 76.28% 76.89% +0.60% 
==========================================
Files 14 14 Lines 4327 4458 +131 Branches 1006 1035 +29 ==========================================
+ Hits 3301 3428 +127 + Misses 667 664 -3 - Partials 359 366 +7 
Files with missing linesCoverage Δ
src/spatialdata_plot/pl/_datashader.py90.50% <100.00%> (ø)
src/spatialdata_plot/pl/basic.py79.61% <100.00%> (+0.19%)⬆️
src/spatialdata_plot/pl/render_params.py89.02% <100.00%> (+0.22%)⬆️
src/spatialdata_plot/pl/render.py88.16% <92.70%> (+1.11%)⬆️
src/spatialdata_plot/pl/utils.py69.40% <81.81%> (+0.51%)⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

`render_labels(element, as_points=True)` with no color crashed:
`instance_id` (the raster's unique values) includes the background label `0`,
which has no centroid, and the literal/no-color color vector is sized to the
raster (not per-instance), so `ax.scatter` got mismatched `c` vs `x`/`y`.
Drop the background label from the rendered instances and align the per-cell
color: for data-driven color the vector is already per-instance and is subset to
match; for the literal/no-color path it is replaced with one na/literal color
per centroid. Data-driven (categorical/continuous) renders are unchanged and
still land exactly on `get_centroids`.
Adds a regression test for the no-color labels case.
…ator
Replace the `regionprops` reduction in `_compute_label_centroids` with an
additive bincount aggregator that streams the labels raster block by block —
one dask chunk (or bounded numpy row-block) in memory at a time — accumulating
per-label `count`/`sum_x`/`sum_y`. This is what makes the feature usable at
Xenium scale:
- Out-of-core: peak memory is one chunk + O(n_labels) accumulators, NOT the
whole raster (measured: 9 MB peak streaming a 268 MB mask). `regionprops`
needs the full array materialized and OOMs on large morphology masks.
- Scales in cell count: 500k+ labels are just array indexing (562k labels in
~1.4 s with 13.5 MB of accumulators); `regionprops`' per-label table does not.
- Faster than `regionprops` (~1.3-1.6x) on in-memory rasters.
- Exact across chunk boundaries (additive reduction) — verified numpy ==
dask-chunked, and identical to `sd.get_centroids`.
- `count` is the cell area, a free by-product (ready for footprint-based dot
sizing).
Drops the `regionprops_table` import; adds `slices_from_chunks`. Adds a unit
test locking the chunk-exact, out-of-core, area-correct behavior.
Note: the chunk loop is currently sequential; parallelizing the per-chunk
partials (dask map_blocks + tree-reduce) is a future speedup.
#705 (measure_obs) merged to main, superseding #703's pre-extraction centroid
block. This merge takes main's utils.py + test_utils.py wholesale (zero #703
delta there) and rewires the labels as_points path onto main's primitive:
- render.py labels branch: drop the deleted `_get_or_compute_centroids` + the
whole cache layer; compute full-resolution (scale0) centroids via main's
`_compute_element_measurements`, and draw them with a transform built from the
*same* scale0 element (`_prepare_transformation(_get_top_data_array(...))`) —
so positions are independent of any rasterization applied to the rendered
`label`. Coerce point_ids to the label dtype so str/object instance ids
(e.g. Xenium readers) align instead of silently reindexing to NaN.
- Net: utils.py == main (the 242-line #703 centroid block + cache layer is
gone); the surviving diff vs main is just the as_points feature.
- Tests: added a non-identity-transform regression test asserting the dots land
at the cells' coordinate-system positions in display space (the guard for the
transform pairing); existing as_points position tests still pass.
Shapes branch unchanged (already intrinsic centroid + trans_data, positionally
aligned to its post-filter color vector).
…ked fields
The fast-mode draw wrapper read cmap/size/alpha/zorder/colorbar/colorbar_params
straight off render_params at both call sites. Pass the render_params dataclass
instead and read them internally; only the genuinely per-branch values
(x/y/color vectors, norm, na_color, transform, adata, palette, col_for_color)
stay explicit. Signature 20->15 params; both as_points call sites lose the
restated render_params plumbing.
Collapse the per-centroid color alignment (drop the unconditional asarray
pre-init and the nested None-guard into a single if/else + ternary), and drop a
redundant np.asarray around the already-ndarray point_ids in the reindex.
Behavior unchanged.
Benchmark showed labels as_points was 16-34x SLOWER than the normal render
because it recomputed full-resolution scale0 centroids while the normal path
downsamples + imshows. Compute centroids on the already-rendered (downsampled)
raster and draw with its trans_data instead: 671k cells goes 19.7s -> 0.88s
(now ~parity with imshow). Centroid error is sub-pixel at display resolution;
position tests updated to display-space within a few-px tolerance.
…gned
show()'s get_extent(exact=True) transforms EVERY shapes/points geometry into
the coordinate system just to take a bounding box - the dominant cost for large
shape collections (~85% of a 5.5M-shape render), unrelated to what is drawn.
Add a self-contained get_extent_fast (drop-in for spatialdata.get_extent) that,
for axis-aligned transforms (scale/flip/90deg/swap + translation - all real
Visium/Xenium data), transforms only the 4 bounding-box corners and reads the
intrinsic bounds vectorised (no per-geometry .apply(is_empty)). Proven identical
to the exact extent for such transforms (spatialdata's own get_extent docstring
notes this); falls back to spatialdata's get_extent for rotation/shear, for
anisotropically-scaled circles (radius->ellipse divergence), and for
images/labels (whose get_extent is already a cheap corner transform).
Measured on real Visium HD render_shapes(as_points=True): 351k 8.5s->1.8s
(4.7x), 5.48M 129s->24s (5.5x); the full (non-as_points) render benefits too.
The whole block is isolated so it can be lifted into spatialdata's get_extent
verbatim (see issue #706). Tests assert it matches get_extent across scale/flip/
rotation/shear for circles and polygons.
@timtreis
timtreisforce-pushed the feat/centroid-scatter-helper branch from 35c8f1f to 57bcf8dCompareJune 9, 2026 23:20
…ompute, underscore)
Follow-up cleanups from review of the get_extent fast path:
- fold _intrinsic_xy_bounds into _element_extent_fast (drop duplicate get_model / geom_type)
- geom.bounds -> geom.total_bounds (C-level union, avoids an Nx4 alloc on large collections)
- batch the four points min/max into one dask.compute
- rename get_extent_fast -> _get_extent_fast (internal helper; matches sibling underscored helpers)
No behavior change; output identical for axis-aligned and rotated/sheared elements.
The datashader shapes canvas sized itself via spatialdata's exact get_extent, transforming every
geometry (O(N)) just for a bounding box -- the same cost _get_extent_fast already removed from
show()'s axis limits, in a second code path. Reuse _element_extent_fast (corner transform for
axis-aligned elements; None -> exact get_extent fallback for rotation/shear), so the result is
pixel-identical and the per-geometry pass is skipped for the common case.
Measured on Curio (69,713 colored shapes): render_shapes 3146 ms -> 1675 ms (1.88x), on top of
the get_extent_fast win already in show(). Mirrors _datashader_canvas_from_dataframe, which
already avoids get_extent for the points path.
…de review)
From the perf-stack review of labels as_points:
- No color column collapsed every dot to a single na_color; now each cell gets a distinct random
colour, matching the mask path's _map_color_seg Case C. Adds a color assertion to the regression test.
- The rasterize drop-filter only ran when a color column was set, so as_points could emit dots at NaN
positions (or drop cells) when rasterization removed labels; extend it to as_points so point ids stay
within the rendered raster.
- ax.scatter autoscales the Normalize in place; copy the shared cmap_params.norm (the shapes path already does).
- render_shapes/render_labels(as_points=True) did not validate size; add the points-path numeric/positive
check so size=-5 / 'big' raise an actionable error instead of a raw matplotlib failure.
Extract the duplicated as_points size-validation block from render_shapes
and render_labels into a shared _validate_as_points_size helper. Defer the
np.asarray(color_vector) conversion in the centroid color path to the only
branch that uses it.
…to feat/centroid-scatter-helper
# Conflicts:
#	tests/pl/test_utils.py
_get_extent_fast already reads get_transformation(element, get_all=True) for
the coordinate-system membership check; pass it through so the fast path does
not re-fetch it per element. The optional kwarg leaves the datashader-canvas
call site unchanged.
_element_extent_fast returned a NaN extent for an element whose geometries are
all empty, which silently poisoned the union in _get_extent_fast (blank axes)
instead of raising spatialdata's clear 'empty collection' error. Guard against
non-finite bounds and fall back. Also trims the verbose extent-block comment and
the scatter/centroid docstrings (net -12 LOC, no behavior change).
@timtreistimtreis changed the title Fast cell rendering: render_shapes/labels(as_points=True) + squidpy centroid cachingFast cell rendering: render_shapes/labels(as_points=True) + axis-aligned extentJun 10, 2026
Four PlotTester baselines: render_shapes/labels(as_points=True) at a base size
and a larger size (size = scatter marker area). Baselines to be generated from CI.
…l tests
Rendered on hatch-test.py3.11-stable; verified dots land at shape/label
centroids, colored by instance_id (labels), larger at size=600.
The empty-shape check ran a per-geometry Python lambda (`.apply(lambda g: not
g.is_empty)`) over every geometry on every shapes render; GeoSeries.is_empty is
GEOS-vectorized. `.is_empty.all()` is identical and ~134x faster on this guard
(665ms -> 5ms at 351k shapes, ~10s -> 80ms at 5.5M).
Shared datashader draw primitive (canvas->aggregate->norm->color-key->shade->
image->colorbar), taking explicit primitives instead of render_params so it can
also serve the as_points centroid path (shapes/labels params use fill_alpha and
lack the density fields). Returns the possibly-recomputed color vectors so the
caller's legend/colorbar stays identical. Verified pixel-identical on
render_points datashader (categorical/continuous/plain/density).
- Route as_points centroids through the shared _datashader_points when method=
'datashader' or above ~500k dots (AS_POINTS_DS_AUTO); matplotlib otherwise.
- No-color labels (one random colour per cell) cannot be aggregated by datashader;
force matplotlib there (warn on explicit method='datashader').
- Fix: shapes as_points drew dots at intrinsic centroid coords while the axes are
in coordinate-system coords, so non-identity transforms misplaced them. Transform
centroids to CS via the element->CS affine for both backends (labels too).
- render_labels gains 'method'; LabelsRenderParams gains 'method'. as_points uses a
fixed datashader reduction ('max', closest to matplotlib) — no user knob.
- shapes non-identity transform regression (the coordinate bug fixed in 5854ae6)
- backend selection: method='datashader' -> datashader image; default -> matplotlib
- no-color labels force matplotlib even with method='datashader' (warns)
- _resolve_as_points_method unit test (threshold/explicit/no-color/empty)
- two test_plot_* visual tests for datashader as_points (baselines from CI)
timtreis added 22 commits June 11, 2026 06:40
Rendered on hatch-test.py3.11-stable; verified datashaded centroids coloured by
category with a categorical legend. Only this test failed on stable CI (missing
baseline); existing datashader baselines unchanged.
_datashader_points' two near-identical reindex/align branches (categorical
source vs colour vector) differed only in which vector they materialize; pick
the source once, then share the reindex/construct. render_points datashader
output verified byte-identical (categorical/continuous/plain/density). -9 LOC.
The datashader as_points canvas was sized to the exact centroid bounding box,
so the outermost centroids sat on the canvas edge and their marker spread was
clipped (half-circles), looking nothing like the matplotlib backend. Grow the
canvas by the spread radius on each side (factor unchanged, so placement still
aligns). render_points keeps pad_for_markers=False -> byte-identical.
…padding fix
The padding fix changed the datashader as_points renders within the comparison
tolerance, so the baselines didn't auto-update. Regenerate from CI so the
committed images reflect the un-clipped output.
CI-rendered (py3.11-stable) after the canvas-padding fix; verified the edge
centroids now render as full circles matching the matplotlib backend.
Restructure the as_points visual tests into matplotlib+datashader pairs that
render identical params (shared helper) for shapes (no color), labels
(instance_id), and labels (categorical). Drop the old inconsistent/stale
baselines; all 6 will be regenerated from CI so the two backends are directly
comparable and look maximally similar.
Positions and sizes align after the canvas-padding fix; dots land at the same
centroids at the same size. Remaining difference is color shading on the
categorical path (datashader modulates alpha by per-pixel count).
Datashader faded single-cell dots (count-driven alpha floor + a second user-alpha
multiply ~= alpha^2), making categorical as_points read much paler than the
matplotlib backend. Add uniform_alpha for the as_points marker mode: a full alpha
floor so each dot is one flat colour at fill_alpha, like a matplotlib marker.
Renamed the as_points flag pad_for_markers -> as_markers (pads canvas + uniform
alpha). render_points unchanged (byte-identical).
…h matplotlib)
CI-rendered (py3.11-stable); the datashader as_points dots now match the
matplotlib markers in colour saturation, position, and size.
…matplotlib
Size (derived, no heuristic): datashader previously rasterized over the centroid
bounding box, so dot display size depended on the canvas/axes extent ratio (shapes
~0.9x, labels ~1.7x matplotlib). Now the canvas spans the same extent as the axes
(like render_points), and the spread radius is set to matplotlib's marker radius
sqrt(s)*dpi/144 (its 'o' marker has diameter sqrt(s)*dpi/72). Result: ds/mpl size
ratio 1.02 +/- 0.03 across element type, size, figure, and dpi.
Continuous colorbar: the spread combined overlapping dots with 'add' (ds_reduction
None -> 'sum'), summing ids and inflating reduction_bounds; marker mode now spreads
with 'max' so overlaps overlay and the colorbar keeps the true range.
render_points unchanged (byte-identical). Replaces the canvas-padding workaround
(the axes-extent canvas already gives edge dots their margin).
… colorbar)
CI-rendered (py3.11-stable): datashader dots now match the matplotlib markers in
size, and the continuous colorbar uses the true value range.
The datashader result is a data-coordinate image that scales with the axes,
while matplotlib markers are fixed in display points. The canvas was sized from
fig.get_size_inches()*dpi (the whole figure), but the axes are smaller (margins,
colorbar), so the image - and every dot - was scaled down: labels (with colorbar)
0.81x matplotlib, shapes 0.88x. Size the as_markers canvas to the axes display box
(ax.get_window_extent()) so 1 canvas px == 1 axes-display px and the sqrt(s)*dpi/144
spread radius matches the marker. Now mean 0.99 +/-5% across sizes/figs/dpi/elements;
render_points untouched (byte-identical). Visual tests render at a non-overlapping
size so the engines' overlap handling (stack vs aggregate) doesn't enter the pairs.
…der canvas
CI-rendered (py3.11-stable). matplotlib and datashader pairs now match in dot
size (the canvas-vs-figure scaling bug is fixed) at a non-overlapping size.
Cleanup pass over the as_points feature (no behavior change):
- trim restated comments/docstrings in _datashader_points, keeping the
load-bearing marker-radius (sqrt(s)*dpi/144) and canvas-vs-axes rationale
- collapse the 3-way min_alpha if/elif/else to a ternary
- reuse the already-computed label extent instead of recomputing get_extent
- np.full instead of list*N for the literal na_color vector
- tighten the as_points test-helper comment
render_points output verified byte-identical; as_points renders unchanged.
Datashader pays off for as_points from ~50k cells (~1.66x faster than the
matplotlib scatter backend, rising to ~1.8x at 1M), so switch over there
instead of 500k. Users can still force either backend via method=.
The shapes as_points block computed its datashader canvas extent via
get_extent (exact=True), which transforms every geometry — on 351k Visium HD
shapes that was ~7s/render and dominated the whole as_points cost, so the fast
mode gave no speedup over full-geometry rendering. Use _element_extent_fast
(corner-transform; identical result for axis-aligned transforms, falls back to
get_extent for rotation/shear), which the PR already added for exactly this.
Real overlay (image + shapes color=gene), as_points:
91k: 7.2s -> 2.0s (3.3x)
351k: 24.3s -> 3.1s (7.3x, vs 22.8s full-geometry)
Output is unchanged: the fast extent is byte-identical for axis-aligned data.
- extract _fast_extent(element, cs) helper; the '_element_extent_fast(...) or
get_extent(...)' idiom was duplicated verbatim at the shapes as_points site
and the datashader-canvas helper
- fix docstrings that still said '~500k' after the threshold dropped to 50k
- trim _render_centroids_as_points docstring and a non-actionable comment
No behavior change: _fast_extent is identical to the inline form (verified
== get_extent on blobs), render_points byte-identical.
@timtreis
timtreis merged commit 8730ff4 into mainJun 14, 2026
7 of 8 checks passed
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #702 (per-panel title fix) and #703 (as_points + fast extent)
into the show() decomposition.
Conflict resolution in src/spatialdata_plot/pl/basic.py:
- #702 up-front title-count validation: kept (after num_panels). The
adjacent axes/panel-count check was dropped here because the
decomposition already relocated it into _plan_panels.
- #702 simplified title selection (dropped the per-panel try/except):
applied to the extracted _finalize_panel helper.
- #703 had no show()-level render-dispatch changes (as_points is
param-driven in render.py, handled inside _render_panel already); its
only show()-level change, get_extent -> _get_extent_fast, auto-merged
into the extent block and consumes _render_panel's `wants` dict.
Also sweeps up two pre-existing #703 lint/type nits in utils.py surfaced
by the merge: _fast_extent docstring (ruff D205) and _get_extent_fast
Any-return (mypy).
Verified: ruff + mypy clean; 109 non-visual show/shapes/labels tests pass.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 21 color helpers + 6 color-only format/uniqueness helpers verbatim from
utils.py into a new sibling module pl/_color.py. Imports flow one way:
_color -> utils (downward, for _get_list/to_hex/_build_alignment_dtype_hint/
_MPL_SINGLE_LETTER_COLORS). render.py, basic.py and _datashader.py are repointed
to import color symbols from _color.
The two validators still in utils (_type_check_params, _validate_graph_render_params)
use color symbols; they carry temporary function-local imports of _color (cycle-safe)
until they move to _validate.py in the next commit, where these become top-level imports.
No behavior change (verbatim move; all-private except set_zero_in_cmap_to_transparent,
which is not re-exported). Verified: no import cycle; 410 non-visual tests pass;
ruff + ruff-format clean. Pre-existing #703/#705 mypy/ruff debt in utils.py is
unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Move 17 validation/type-check functions verbatim from utils.py into a new sibling
module pl/_validate.py. Imports flow one way: _validate -> _color (_is_color_like,
_prepare_cmap_norm, _get_colors_for_categorical_obs, now top-level) and
_validate -> utils (downward). The temporary function-local _color imports added in
the previous commit are hoisted to top-level here and removed from utils.
render.py and basic.py repointed to _validate.
Completes the utils.py split (#696): utils 4918 -> 1311 lines, with
_geometry / _datashader / _color / _validate as single-concern siblings.
No import cycle anywhere. No behavior change (verbatim moves; all-private).
Verified: 410 non-visual tests pass; ruff + ruff-format clean. Pre-existing
#703/#705 mypy debt in utils.py is unrelated; --no-verify for that reason only.
timtreis added a commit that referenced this pull request Jun 14, 2026
Integrate #714 (show() decomposition) into the utils.py split. Only utils.py
conflicted:
- import block: dropped the now-unused `_locate_value` import (it moved to
_color.py with the color code that uses it); kept main's `_locate_value`
out of utils.
- `_fast_extent` docstring: took main's #714 version (D205 fix).
basic.py auto-merged: #714's decomposed show()/helpers now import color and
validation symbols from _color/_validate (the split's repoints), not utils.
Bonus: merging #714 brings its fixes for the pre-existing #703/#705 debt
(_resolve_measure_table str-return, _get_extent_fast Any-return, _fast_extent
D205), so the branch is now fully ruff + mypy clean (no --no-verify).
Verified: no import cycle; ruff + ruff-format + mypy all pass; 410 non-visual
tests pass.
@timtreis
timtreis deleted the feat/centroid-scatter-helper branch July 10, 2026 11:11
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@timtreis@codecov-commenter