Local cloud

[sophiamaedler]

I was trying to initialize a spatialdata object directly from S3 as done in the tests here:

fromupathimportUPathimportspatialdataassdtest=UPath( "s3://spatialdata/spatialdata-sandbox/merfish.zarr", endpoint_url="https://s3.embl.de", anon=True )
sd.read_zarr(test)

Was failing with:

---------------------------------------------------------------------------TypeErrorTraceback (mostrecentcalllast)
CellIn[1], line42importspatialdataassd3test=UPath( "s3://spatialdata/spatialdata-sandbox/merfish.zarr", endpoint_url="https://s3.embl.de/", anon=True )
---->4sd.read_zarr(test)
File~/src/spatialdata/_io/io_zarr.py:282, inread_zarr(store, selection, on_bad_files)
272attrs=None274sdata=SpatialData(
275images=images,
276labels=labels,
(...) 281 )
-->282sdata.path=_create_upath(_store)
283returnsdataFile~/src/spatialdata/_core/spatialdata.py:590, inSpatialData.path(self, value)
588self._path=value589else:
-->590raiseTypeError("Path must be `None`, a `str` or a `Path` object.")
592ifnotself.is_self_contained():
593logger.info(
594"The SpatialData object is not self-contained "595"(i.e. it contains some elements that are Dask-backed "596"from locations outside {self.path})."597 )
TypeError: Pathmustbe`None`, a`str`ora`Path`object.

The implemented changes fix the issues and result in the sdata object being successfully read from S3.

The code now returns:

SpatialDataobject, withassociatedZarrstore: s3://spatialdata/spatialdata-sandbox/merfish.zarr
├── Images
│ └── 'rasterized': DataArray[cyx] (1, 522, 575)
├── Points
│ └── 'single_molecule': DataFramewithshape: (<Delayed>, 3) (2Dpoints)
├── Shapes
│ ├── 'anatomical': GeoDataFrameshape: (6, 1) (2Dshapes)
│ └── 'cells': GeoDataFrameshape: (2389, 2) (2Dshapes)
└── Tables
└── 'table': AnnData (2389, 268)
withcoordinatesystems:
▸ 'global', withelements:
rasterized (Images), single_molecule (Points), anatomical (Shapes), cells (Shapes)
withthefollowingDask-backedelementsnotbeingself-contained:
▸ rasterized: [path/spatialdata/spatialdata-sandbox/merfish.zarr/images/rasterized]
▸ single_molecule: [path/spatialdata/spatialdata-sandbox/merfish.zarr/points/single_molecule/points.parquet/part.0.parquet]

[sophiamaedler]

actions failing due to changes introduced in previous commit: c514a0b
I can take a look to see if I can figure out the problem.

[melonora]

There is PR #971 that fixes the remote storage completely; however, between that PR and zarrv3 being merged, zarrv3 got merged first. So #971 would require an update.

[melonora]

with the current dask unpinning this PR would not work anymore, neither does the other PR. I am implementing some fixes at the moment. Main problem is that now FSSspectstore requires an async file system.