sdata.table.obsm['spatial'] object populated with Nan's

[mkunst23]

Hi,

I'm trying to run the squidpy integration with spatialdata but can't do any spatial calculations (i.e. neighborhood connectivities) because the sdata.table.obsm['spatial'] object is populated with Nan's.

---

ValueError Traceback (most recent call last)
Cell In[13], line 1
----> 1 sq.gr.spatial_neighbors(sdata.table)

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/squidpy/gr/_build.py:160, in spatial_neighbors(adata, spatial_key, library_key, coord_type, n_neighs, radius, delaunay, n_rings, percentile, transform, set_diag, key_added, copy)
158 Dst = block_diag([m[1] for m in mats], format="csr")[ixs, :][:, ixs]
159 else:
--> 160 Adj, Dst = _build_fun(adata)
162 neighs_key = Key.uns.spatial_neighs(key_added)
163 conns_key = Key.obsp.spatial_conn(key_added)

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/squidpy/gr/_build.py:198, in _spatial_neighbor(adata, spatial_key, coord_type, n_neighs, radius, delaunay, n_rings, transform, set_diag, percentile)
196 Adj, Dst = _build_grid(coords, n_neighs=n_neighs, n_rings=n_rings, delaunay=delaunay, set_diag=set_diag)
197 elif coord_type == CoordType.GENERIC:
--> 198 Adj, Dst = _build_connectivity(
199 coords, n_neighs=n_neighs, radius=radius, delaunay=delaunay, return_distance=True, set_diag=set_diag
200 )
201 else:
202 raise NotImplementedError(f"Coordinate type {coord_type} is not yet implemented.")

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/squidpy/gr/_build.py:295, in _build_connectivity(coords, n_neighs, radius, delaunay, neigh_correct, set_diag, return_distance)
293 r = 1 if radius is None else radius if isinstance(radius, (int, float)) else max(radius)
294 tree = NearestNeighbors(n_neighbors=n_neighs, radius=r, metric="euclidean")
--> 295 tree.fit(coords)
297 if radius is None:
298 results = tree.kneighbors()

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/sklearn/base.py:1152, in _fit_context..decorator..wrapper(estimator, *args, **kwargs)
1145 estimator._validate_params()
1147 with config_context(
1148 skip_parameter_validation=(
1149 prefer_skip_nested_validation or global_skip_validation
1150 )
1151 ):
-> 1152 return fit_method(estimator, *args, **kwargs)

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/sklearn/neighbors/_unsupervised.py:175, in NearestNeighbors.fit(self, X, y)
154 @_fit_context(
155 # NearestNeighbors.metric is not validated yet
156 prefer_skip_nested_validation=False
157 )
158 def fit(self, X, y=None):
159 """Fit the nearest neighbors estimator from the training dataset.
160
161 Parameters
(...)
173 The fitted nearest neighbors estimator.
174 """
--> 175 return self._fit(X)

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/sklearn/neighbors/_base.py:498, in NeighborsBase._fit(self, X, y)
496 else:
497 if not isinstance(X, (KDTree, BallTree, NeighborsBase)):
--> 498 X = self._validate_data(X, accept_sparse="csr", order="C")
500 self._check_algorithm_metric()
501 if self.metric_params is None:

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/sklearn/base.py:605, in BaseEstimator._validate_data(self, X, y, reset, validate_separately, cast_to_ndarray, **check_params)
603 out = X, y
604 elif not no_val_X and no_val_y:
--> 605 out = check_array(X, input_name="X", **check_params)
606 elif no_val_X and not no_val_y:
607 out = _check_y(y, **check_params)

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/sklearn/utils/validation.py:957, in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator, input_name)
951 raise ValueError(
952 "Found array with dim %d. %s expected <= 2."
953 % (array.ndim, estimator_name)
954 )
956 if force_all_finite:
--> 957 _assert_all_finite(
958 array,
959 input_name=input_name,
960 estimator_name=estimator_name,
961 allow_nan=force_all_finite == "allow-nan",
962 )
964 if ensure_min_samples > 0:
965 n_samples = _num_samples(array)

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/sklearn/utils/validation.py:122, in _assert_all_finite(X, allow_nan, msg_dtype, estimator_name, input_name)
119 if first_pass_isfinite:
120 return
--> 122 _assert_all_finite_element_wise(
123 X,
124 xp=xp,
125 allow_nan=allow_nan,
126 msg_dtype=msg_dtype,
127 estimator_name=estimator_name,
128 input_name=input_name,
129 )

File /allen/programs/celltypes/workgroups/rnaseqanalysis/mFISH/michaelkunst/miniconda3/envs/SpatialData/lib/python3.10/site-packages/sklearn/utils/validation.py:171, in _assert_all_finite_element_wise(X, xp, allow_nan, msg_dtype, estimator_name, input_name)
154 if estimator_name and input_name == "X" and has_nan_error:
155 # Improve the error message on how to handle missing values in
156 # scikit-learn.
157 msg_err += (
158 f"\n{estimator_name} does not accept missing values"
159 " encoded as NaN natively. For supervised learning, you might want"
(...)
169 "#estimators-that-handle-nan-values"
170 )
--> 171 raise ValueError(msg_err)

ValueError: Input X contains NaN.
NearestNeighbors does not accept missing values encoded as NaN natively. For supervised learning, you might want to consider sklearn.ensemble.HistGradientBoostingClassifier and Regressor which accept missing values encoded as NaNs natively. Alternatively, it is possible to preprocess the data, for instance by using an imputer transformer in a pipeline or drop samples with missing values. See https://scikit-learn.org/stable/modules/impute.html You can find a list of all estimators that handle NaN values at the following page: https://scikit-learn.org/stable/modules/impute.html#estimators-that-handle-nan-values

sdata.table.obsm['spatial']

array([[nan, nan],
[nan, nan],
[nan, nan],
...,
[nan, nan],
[nan, nan],
[nan, nan]])

[LucaMarconato]

Hi, thanks for reporting.

The latest release of Squidpy added support for SpatialData and in theory passing sdata instead of sdata.table should work, but looking at the code I just realized that the automatic computation of the centroids independent of the elements being used is still not merged to main.

I will report to Squidpy/make a fix there. For the time please add the centroids manually to sdata.table.obsm['spatial'] using this code:

 if isinstance(regions, GeoDataFrame):
	dims = get_axes_names(regions)
	region = regions.iloc[region_index]
	shape = regions.geometry.iloc[0]
	if isinstance(shape, Polygon):
	 xy = region.geometry.centroid.coords.xy
	 centroid = np.array([[xy[0][0], xy[1][0]]])
	elif isinstance(shape, Point):
	 xy = region.geometry.coords.xy
	 centroid = np.array([[xy[0][0], xy[1][0]]])

If your data is aligned to a coordinate system using a transformation that is not an identity, please transform the centroids. You can do this by creating a points element from the centroids with PointsModel.parse() and by calling transform().

I will update the issue when Squidpy is updated.

[giovp]

I can confirm that using latest version you should be able to pass spatialdata as is

[LucaMarconato]

Hi, I have now updated the squidpy notebook to clarify the use cases and workflow. It's not in the docs yet but meanwhile you can find a preview here: https://github.com/scverse/spatialdata-notebooks/blob/improve/docs/notebooks/examples/squidpy_integration.ipynb. When the link will break it will mean that you will be able to see the notebook from the docs, here: https://spatialdata.scverse.org/en/latest/tutorials/notebooks/notebooks/examples/squidpy_integration.html.

The notebook uses a new conversion function (not released yet, but that can already be used): scverse/spatialdata-io#102.

I believe that using the functions that I linked would address the problem you described, so I will close the issue now. Anyway, feel free to reopen it if you think it's still not solved.