updated read_visium() to read in spaceranger 2.0 files

[LLehner]

Can now read in spaceranger 2.0 files where the tissue position file name is different and a header is included. Code adapated from squidpy's read.visium() in response to scverse/squidpy#599 and scverse/scanpy#2296.

[codecov]

Codecov Report

Merging #2424 (e0a7409) into master (9969734) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@ Coverage Diff @@## master #2424 +/- ##
=======================================
Coverage 71.87% 71.87% =======================================
Files 98 98 Lines 11496 11496 =======================================
Hits 8263 8263 Misses 3233 3233 

Impacted Files	Coverage Δ
scanpy/readwrite.py	67.58% <100.00%> (ø)

[LLehner]

10x Visium data from both space ranger 1.0.0 and 2.0.0 can be read in using sc.read_visium() and plotted using squidpy with coordinates in correct order.

space ranger 1.0.0 data from 10x plotted with squidpy's spatial_scatter():

space ranger 2.0.0 data from 10x plotted with squidpy's spatial_scatter():

[LLehner]

@giovp

[ivirshup]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

[giovp]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

for sure, @LLehner could you? think he's is on holiday until next week

[LLehner]

for sure, @LLehner could you? think he's is on holiday until next week

sure, will do

[giovp]

LGTM thanks @LLehner !

[flying-sheep]

hi @LLehner, why is this header=1?. Doesn’t that mean “the second line of data”? Shouldn’t it be header=0?

[LLehner]

hi @flying-sheep, you are right. Header should be 0, otherwise the first row with values (besides the barcode) will be turned into headers and then overwritten. Didn't catch that before, since it didn't cause issues in the plotting examples.

[mortunco]

Folks. Forgive my ignorance if I am completely wrong but it seems read_visium still reads it wrong.

https://github.com/scverse/scanpy/blob/edd613026cd5991baf92c8308b5ee2375089adc8/scanpy/readwrite.py#L466C1-L473

another edit:

scanpy/scanpy/readwrite.py

Lines 466 to 482 in 89804c2

	positions.columns= [
	'in_tissue',
	'array_row',
	'array_col',
	'pxl_col_in_fullres',
	'pxl_row_in_fullres',
	]
	adata.obs=adata.obs.join(positions, how="left")
	adata.obsm['spatial'] =adata.obs[
	['pxl_row_in_fullres', 'pxl_col_in_fullres']
	].to_numpy()
	adata.obs.drop(
	columns=['pxl_row_in_fullres', 'pxl_col_in_fullres'],
	inplace=True,
	)

Basically line 476 is fixes the switch and makes the barcode coordinates in (x,y).

I apologise for the confusion.

[grst]

With scverse/spatialdata-io#102 we could consider replacing scanpy.read_visium with

defread_visium(*args, **kwargs): importspatialdata_ioreturnspatialdata_io.visium(*args, **kwargs).to_legacy_anndata()

and avoid maintaing the annoying spaceranger output parsing in multiple locations.

[LLehner]

Can now read in spaceranger 2.0 files where the tissue position file name is different and a header is included. Code adapated from squidpy's read.visium() in response to scverse/squidpy#599 and scverse/scanpy#2296.

[codecov]

Codecov Report

Merging #2424 (e0a7409) into master (9969734) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@ Coverage Diff @@## master #2424 +/- ##
=======================================
Coverage 71.87% 71.87% =======================================
Files 98 98 Lines 11496 11496 =======================================
Hits 8263 8263 Misses 3233 3233 

Impacted Files	Coverage Δ
scanpy/readwrite.py	67.58% <100.00%> (ø)

[LLehner]

10x Visium data from both space ranger 1.0.0 and 2.0.0 can be read in using sc.read_visium() and plotted using squidpy with coordinates in correct order.

space ranger 1.0.0 data from 10x plotted with squidpy's spatial_scatter():

space ranger 2.0.0 data from 10x plotted with squidpy's spatial_scatter():

[LLehner]

@giovp

[ivirshup]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

[giovp]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

for sure, @LLehner could you? think he's is on holiday until next week

[LLehner]

for sure, @LLehner could you? think he's is on holiday until next week

sure, will do

[giovp]

LGTM thanks @LLehner !

[flying-sheep]

hi @LLehner, why is this header=1?. Doesn’t that mean “the second line of data”? Shouldn’t it be header=0?

[LLehner]

hi @flying-sheep, you are right. Header should be 0, otherwise the first row with values (besides the barcode) will be turned into headers and then overwritten. Didn't catch that before, since it didn't cause issues in the plotting examples.

[mortunco]

Folks. Forgive my ignorance if I am completely wrong but it seems read_visium still reads it wrong.

https://github.com/scverse/scanpy/blob/edd613026cd5991baf92c8308b5ee2375089adc8/scanpy/readwrite.py#L466C1-L473

another edit:

scanpy/scanpy/readwrite.py

Lines 466 to 482 in 89804c2

	positions.columns= [
	'in_tissue',
	'array_row',
	'array_col',
	'pxl_col_in_fullres',
	'pxl_row_in_fullres',
	]
	adata.obs=adata.obs.join(positions, how="left")
	adata.obsm['spatial'] =adata.obs[
	['pxl_row_in_fullres', 'pxl_col_in_fullres']
	].to_numpy()
	adata.obs.drop(
	columns=['pxl_row_in_fullres', 'pxl_col_in_fullres'],
	inplace=True,
	)

Basically line 476 is fixes the switch and makes the barcode coordinates in (x,y).

I apologise for the confusion.

[grst]

With scverse/spatialdata-io#102 we could consider replacing scanpy.read_visium with

defread_visium(*args, **kwargs): importspatialdata_ioreturnspatialdata_io.visium(*args, **kwargs).to_legacy_anndata()

and avoid maintaing the annoying spaceranger output parsing in multiple locations.

[LLehner]

Can now read in spaceranger 2.0 files where the tissue position file name is different and a header is included. Code adapated from squidpy's read.visium() in response to scverse/squidpy#599 and scverse/scanpy#2296.

[codecov]

Codecov Report

Merging #2424 (e0a7409) into master (9969734) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@ Coverage Diff @@## master #2424 +/- ##
=======================================
Coverage 71.87% 71.87% =======================================
Files 98 98 Lines 11496 11496 =======================================
Hits 8263 8263 Misses 3233 3233 

Impacted Files	Coverage Δ
scanpy/readwrite.py	67.58% <100.00%> (ø)

[LLehner]

10x Visium data from both space ranger 1.0.0 and 2.0.0 can be read in using sc.read_visium() and plotted using squidpy with coordinates in correct order.

space ranger 1.0.0 data from 10x plotted with squidpy's spatial_scatter():

space ranger 2.0.0 data from 10x plotted with squidpy's spatial_scatter():

[LLehner]

@giovp

[ivirshup]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

[giovp]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

for sure, @LLehner could you? think he's is on holiday until next week

[LLehner]

for sure, @LLehner could you? think he's is on holiday until next week

sure, will do

[giovp]

LGTM thanks @LLehner !

[flying-sheep]

hi @LLehner, why is this header=1?. Doesn’t that mean “the second line of data”? Shouldn’t it be header=0?

[LLehner]

hi @flying-sheep, you are right. Header should be 0, otherwise the first row with values (besides the barcode) will be turned into headers and then overwritten. Didn't catch that before, since it didn't cause issues in the plotting examples.

[mortunco]

Folks. Forgive my ignorance if I am completely wrong but it seems read_visium still reads it wrong.

https://github.com/scverse/scanpy/blob/edd613026cd5991baf92c8308b5ee2375089adc8/scanpy/readwrite.py#L466C1-L473

another edit:

scanpy/scanpy/readwrite.py

Lines 466 to 482 in 89804c2

	positions.columns= [
	'in_tissue',
	'array_row',
	'array_col',
	'pxl_col_in_fullres',
	'pxl_row_in_fullres',
	]
	adata.obs=adata.obs.join(positions, how="left")
	adata.obsm['spatial'] =adata.obs[
	['pxl_row_in_fullres', 'pxl_col_in_fullres']
	].to_numpy()
	adata.obs.drop(
	columns=['pxl_row_in_fullres', 'pxl_col_in_fullres'],
	inplace=True,
	)

Basically line 476 is fixes the switch and makes the barcode coordinates in (x,y).

I apologise for the confusion.

[grst]

With scverse/spatialdata-io#102 we could consider replacing scanpy.read_visium with

defread_visium(*args, **kwargs): importspatialdata_ioreturnspatialdata_io.visium(*args, **kwargs).to_legacy_anndata()

and avoid maintaing the annoying spaceranger output parsing in multiple locations.

[LLehner]

Can now read in spaceranger 2.0 files where the tissue position file name is different and a header is included. Code adapated from squidpy's read.visium() in response to scverse/squidpy#599 and scverse/scanpy#2296.

[codecov]

Codecov Report

Merging #2424 (e0a7409) into master (9969734) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@ Coverage Diff @@## master #2424 +/- ##
=======================================
Coverage 71.87% 71.87% =======================================
Files 98 98 Lines 11496 11496 =======================================
Hits 8263 8263 Misses 3233 3233 

Impacted Files	Coverage Δ
scanpy/readwrite.py	67.58% <100.00%> (ø)

[LLehner]

10x Visium data from both space ranger 1.0.0 and 2.0.0 can be read in using sc.read_visium() and plotted using squidpy with coordinates in correct order.

space ranger 1.0.0 data from 10x plotted with squidpy's spatial_scatter():

space ranger 2.0.0 data from 10x plotted with squidpy's spatial_scatter():

[LLehner]

@giovp

[ivirshup]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

[giovp]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

for sure, @LLehner could you? think he's is on holiday until next week

[LLehner]

for sure, @LLehner could you? think he's is on holiday until next week

sure, will do

[giovp]

LGTM thanks @LLehner !

[flying-sheep]

hi @LLehner, why is this header=1?. Doesn’t that mean “the second line of data”? Shouldn’t it be header=0?

[LLehner]

hi @flying-sheep, you are right. Header should be 0, otherwise the first row with values (besides the barcode) will be turned into headers and then overwritten. Didn't catch that before, since it didn't cause issues in the plotting examples.

[mortunco]

Folks. Forgive my ignorance if I am completely wrong but it seems read_visium still reads it wrong.

https://github.com/scverse/scanpy/blob/edd613026cd5991baf92c8308b5ee2375089adc8/scanpy/readwrite.py#L466C1-L473

another edit:

scanpy/scanpy/readwrite.py

Lines 466 to 482 in 89804c2

	positions.columns= [
	'in_tissue',
	'array_row',
	'array_col',
	'pxl_col_in_fullres',
	'pxl_row_in_fullres',
	]
	adata.obs=adata.obs.join(positions, how="left")
	adata.obsm['spatial'] =adata.obs[
	['pxl_row_in_fullres', 'pxl_col_in_fullres']
	].to_numpy()
	adata.obs.drop(
	columns=['pxl_row_in_fullres', 'pxl_col_in_fullres'],
	inplace=True,
	)

Basically line 476 is fixes the switch and makes the barcode coordinates in (x,y).

I apologise for the confusion.

[grst]

With scverse/spatialdata-io#102 we could consider replacing scanpy.read_visium with

defread_visium(*args, **kwargs): importspatialdata_ioreturnspatialdata_io.visium(*args, **kwargs).to_legacy_anndata()

and avoid maintaing the annoying spaceranger output parsing in multiple locations.

[LLehner]

Can now read in spaceranger 2.0 files where the tissue position file name is different and a header is included. Code adapated from squidpy's read.visium() in response to scverse/squidpy#599 and scverse/scanpy#2296.

[codecov]

Codecov Report

Merging #2424 (e0a7409) into master (9969734) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@ Coverage Diff @@## master #2424 +/- ##
=======================================
Coverage 71.87% 71.87% =======================================
Files 98 98 Lines 11496 11496 =======================================
Hits 8263 8263 Misses 3233 3233 

Impacted Files	Coverage Δ
scanpy/readwrite.py	67.58% <100.00%> (ø)

[LLehner]

10x Visium data from both space ranger 1.0.0 and 2.0.0 can be read in using sc.read_visium() and plotted using squidpy with coordinates in correct order.

space ranger 1.0.0 data from 10x plotted with squidpy's spatial_scatter():

space ranger 2.0.0 data from 10x plotted with squidpy's spatial_scatter():

[LLehner]

@giovp

[ivirshup]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

[giovp]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

for sure, @LLehner could you? think he's is on holiday until next week

[LLehner]

for sure, @LLehner could you? think he's is on holiday until next week

sure, will do

[giovp]

LGTM thanks @LLehner !

[flying-sheep]

hi @LLehner, why is this header=1?. Doesn’t that mean “the second line of data”? Shouldn’t it be header=0?

[LLehner]

hi @flying-sheep, you are right. Header should be 0, otherwise the first row with values (besides the barcode) will be turned into headers and then overwritten. Didn't catch that before, since it didn't cause issues in the plotting examples.

[mortunco]

Folks. Forgive my ignorance if I am completely wrong but it seems read_visium still reads it wrong.

https://github.com/scverse/scanpy/blob/edd613026cd5991baf92c8308b5ee2375089adc8/scanpy/readwrite.py#L466C1-L473

another edit:

scanpy/scanpy/readwrite.py

Lines 466 to 482 in 89804c2

	positions.columns= [
	'in_tissue',
	'array_row',
	'array_col',
	'pxl_col_in_fullres',
	'pxl_row_in_fullres',
	]
	adata.obs=adata.obs.join(positions, how="left")
	adata.obsm['spatial'] =adata.obs[
	['pxl_row_in_fullres', 'pxl_col_in_fullres']
	].to_numpy()
	adata.obs.drop(
	columns=['pxl_row_in_fullres', 'pxl_col_in_fullres'],
	inplace=True,
	)

Basically line 476 is fixes the switch and makes the barcode coordinates in (x,y).

I apologise for the confusion.

[grst]

With scverse/spatialdata-io#102 we could consider replacing scanpy.read_visium with

defread_visium(*args, **kwargs): importspatialdata_ioreturnspatialdata_io.visium(*args, **kwargs).to_legacy_anndata()

and avoid maintaing the annoying spaceranger output parsing in multiple locations.

[LLehner]

Can now read in spaceranger 2.0 files where the tissue position file name is different and a header is included. Code adapated from squidpy's read.visium() in response to scverse/squidpy#599 and scverse/scanpy#2296.

[codecov]

Codecov Report

Merging #2424 (e0a7409) into master (9969734) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@ Coverage Diff @@## master #2424 +/- ##
=======================================
Coverage 71.87% 71.87% =======================================
Files 98 98 Lines 11496 11496 =======================================
Hits 8263 8263 Misses 3233 3233 

Impacted Files	Coverage Δ
scanpy/readwrite.py	67.58% <100.00%> (ø)

[LLehner]

10x Visium data from both space ranger 1.0.0 and 2.0.0 can be read in using sc.read_visium() and plotted using squidpy with coordinates in correct order.

space ranger 1.0.0 data from 10x plotted with squidpy's spatial_scatter():

space ranger 2.0.0 data from 10x plotted with squidpy's spatial_scatter():

[LLehner]

@giovp

[ivirshup]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

[giovp]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

for sure, @LLehner could you? think he's is on holiday until next week

[LLehner]

for sure, @LLehner could you? think he's is on holiday until next week

sure, will do

[giovp]

LGTM thanks @LLehner !

[flying-sheep]

hi @LLehner, why is this header=1?. Doesn’t that mean “the second line of data”? Shouldn’t it be header=0?

[LLehner]

hi @flying-sheep, you are right. Header should be 0, otherwise the first row with values (besides the barcode) will be turned into headers and then overwritten. Didn't catch that before, since it didn't cause issues in the plotting examples.

[mortunco]

Folks. Forgive my ignorance if I am completely wrong but it seems read_visium still reads it wrong.

https://github.com/scverse/scanpy/blob/edd613026cd5991baf92c8308b5ee2375089adc8/scanpy/readwrite.py#L466C1-L473

another edit:

scanpy/scanpy/readwrite.py

Lines 466 to 482 in 89804c2

	positions.columns= [
	'in_tissue',
	'array_row',
	'array_col',
	'pxl_col_in_fullres',
	'pxl_row_in_fullres',
	]
	adata.obs=adata.obs.join(positions, how="left")
	adata.obsm['spatial'] =adata.obs[
	['pxl_row_in_fullres', 'pxl_col_in_fullres']
	].to_numpy()
	adata.obs.drop(
	columns=['pxl_row_in_fullres', 'pxl_col_in_fullres'],
	inplace=True,
	)

Basically line 476 is fixes the switch and makes the barcode coordinates in (x,y).

I apologise for the confusion.

[grst]

With scverse/spatialdata-io#102 we could consider replacing scanpy.read_visium with

defread_visium(*args, **kwargs): importspatialdata_ioreturnspatialdata_io.visium(*args, **kwargs).to_legacy_anndata()

and avoid maintaing the annoying spaceranger output parsing in multiple locations.

[LLehner]

Can now read in spaceranger 2.0 files where the tissue position file name is different and a header is included. Code adapated from squidpy's read.visium() in response to scverse/squidpy#599 and scverse/scanpy#2296.

[codecov]

Codecov Report

Merging #2424 (e0a7409) into master (9969734) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@ Coverage Diff @@## master #2424 +/- ##
=======================================
Coverage 71.87% 71.87% =======================================
Files 98 98 Lines 11496 11496 =======================================
Hits 8263 8263 Misses 3233 3233 

Impacted Files	Coverage Δ
scanpy/readwrite.py	67.58% <100.00%> (ø)

[LLehner]

10x Visium data from both space ranger 1.0.0 and 2.0.0 can be read in using sc.read_visium() and plotted using squidpy with coordinates in correct order.

space ranger 1.0.0 data from 10x plotted with squidpy's spatial_scatter():

space ranger 2.0.0 data from 10x plotted with squidpy's spatial_scatter():

[LLehner]

@giovp

[ivirshup]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

[giovp]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

for sure, @LLehner could you? think he's is on holiday until next week

[LLehner]

for sure, @LLehner could you? think he's is on holiday until next week

sure, will do

[giovp]

LGTM thanks @LLehner !

[flying-sheep]

hi @LLehner, why is this header=1?. Doesn’t that mean “the second line of data”? Shouldn’t it be header=0?

[LLehner]

hi @flying-sheep, you are right. Header should be 0, otherwise the first row with values (besides the barcode) will be turned into headers and then overwritten. Didn't catch that before, since it didn't cause issues in the plotting examples.

[mortunco]

Folks. Forgive my ignorance if I am completely wrong but it seems read_visium still reads it wrong.

https://github.com/scverse/scanpy/blob/edd613026cd5991baf92c8308b5ee2375089adc8/scanpy/readwrite.py#L466C1-L473

another edit:

scanpy/scanpy/readwrite.py

Lines 466 to 482 in 89804c2

	positions.columns= [
	'in_tissue',
	'array_row',
	'array_col',
	'pxl_col_in_fullres',
	'pxl_row_in_fullres',
	]
	adata.obs=adata.obs.join(positions, how="left")
	adata.obsm['spatial'] =adata.obs[
	['pxl_row_in_fullres', 'pxl_col_in_fullres']
	].to_numpy()
	adata.obs.drop(
	columns=['pxl_row_in_fullres', 'pxl_col_in_fullres'],
	inplace=True,
	)

Basically line 476 is fixes the switch and makes the barcode coordinates in (x,y).

I apologise for the confusion.

[grst]

With scverse/spatialdata-io#102 we could consider replacing scanpy.read_visium with

defread_visium(*args, **kwargs): importspatialdata_ioreturnspatialdata_io.visium(*args, **kwargs).to_legacy_anndata()

and avoid maintaing the annoying spaceranger output parsing in multiple locations.

[LLehner]

Can now read in spaceranger 2.0 files where the tissue position file name is different and a header is included. Code adapated from squidpy's read.visium() in response to scverse/squidpy#599 and scverse/scanpy#2296.

[codecov]

Codecov Report

Merging #2424 (e0a7409) into master (9969734) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@ Coverage Diff @@## master #2424 +/- ##
=======================================
Coverage 71.87% 71.87% =======================================
Files 98 98 Lines 11496 11496 =======================================
Hits 8263 8263 Misses 3233 3233 

Impacted Files	Coverage Δ
scanpy/readwrite.py	67.58% <100.00%> (ø)

[LLehner]

10x Visium data from both space ranger 1.0.0 and 2.0.0 can be read in using sc.read_visium() and plotted using squidpy with coordinates in correct order.

space ranger 1.0.0 data from 10x plotted with squidpy's spatial_scatter():

space ranger 2.0.0 data from 10x plotted with squidpy's spatial_scatter():

[LLehner]

@giovp

[ivirshup]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

[giovp]

@giovp feel free to approve and merge. One request first:

Can this get a release note?

for sure, @LLehner could you? think he's is on holiday until next week

[LLehner]

for sure, @LLehner could you? think he's is on holiday until next week

sure, will do

[giovp]

LGTM thanks @LLehner !

[flying-sheep]

hi @LLehner, why is this header=1?. Doesn’t that mean “the second line of data”? Shouldn’t it be header=0?

[LLehner]

hi @flying-sheep, you are right. Header should be 0, otherwise the first row with values (besides the barcode) will be turned into headers and then overwritten. Didn't catch that before, since it didn't cause issues in the plotting examples.

[mortunco]

Folks. Forgive my ignorance if I am completely wrong but it seems read_visium still reads it wrong.

https://github.com/scverse/scanpy/blob/edd613026cd5991baf92c8308b5ee2375089adc8/scanpy/readwrite.py#L466C1-L473

another edit:

scanpy/scanpy/readwrite.py

Lines 466 to 482 in 89804c2

	positions.columns= [
	'in_tissue',
	'array_row',
	'array_col',
	'pxl_col_in_fullres',
	'pxl_row_in_fullres',
	]
	adata.obs=adata.obs.join(positions, how="left")
	adata.obsm['spatial'] =adata.obs[
	['pxl_row_in_fullres', 'pxl_col_in_fullres']
	].to_numpy()
	adata.obs.drop(
	columns=['pxl_row_in_fullres', 'pxl_col_in_fullres'],
	inplace=True,
	)

Basically line 476 is fixes the switch and makes the barcode coordinates in (x,y).

I apologise for the confusion.

[grst]

With scverse/spatialdata-io#102 we could consider replacing scanpy.read_visium with

defread_visium(*args, **kwargs): importspatialdata_ioreturnspatialdata_io.visium(*args, **kwargs).to_legacy_anndata()

and avoid maintaing the annoying spaceranger output parsing in multiple locations.