Skip to content

Support remote private storage by consistent use of substore - #442

Closed
berombau wants to merge 45 commits into
scverse:mainfrom
berombau:main
Closed

Support remote private storage by consistent use of substore#442
berombau wants to merge 45 commits into
scverse:mainfrom
berombau:main

Conversation

@berombau

@berombauberombau commented Jan 26, 2024

Copy link
Copy Markdown
Contributor

Closes#441, depends on (ome/ome-zarr-py#349)

@berombauberombau mentioned this pull request Jan 26, 2024
Open
@codecov

codecovBot commented Jan 26, 2024•
edited
Loading

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 71.42857% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 92.52%. Comparing base (b73f3a8) to head (7bb87ba).
Report is 2 commits behind head on main.

❗ Current head 7bb87ba differs from pull request most recent head c120a4d. Consider uploading reports for the commit c120a4d to get more accurate results

Additional details and impacted files
@@ Coverage Diff @@## main #442 +/- ##
==========================================
- Coverage 92.53% 92.52% -0.02% 
==========================================
Files 43 39 -4 Lines 6003 5270 -733 ==========================================
- Hits 5555 4876 -679 + Misses 448 394 -54
FilesCoverage Δ
src/spatialdata/_core/spatialdata.py94.43% <ø> (+2.21%)⬆️
src/spatialdata/_io/io_raster.py96.29% <100.00%> (ø)
src/spatialdata/_io/io_shapes.py100.00% <100.00%> (+2.04%)⬆️
src/spatialdata/_io/io_zarr.py86.08% <66.66%> (-2.29%)⬇️

... and 22 files with indirect coverage changes

berombauand others added 21 commits February 28, 2024 12:40
@berombau
Merge branch 'scverse:main' into main
4c150c5
@berombau
Merge branch 'scverse:main' into main
7bb87ba
@LucaMarconato
implemented incremental io; tests missing
5c73c84
@LucaMarconato
added draft for write_metadata(); need to write new tests
b534f6e
@LucaMarconato
wip better explanation
e7b9433
@LucaMarconato
fixed bug wrong order of points columns after spatial query
ebe9f4f
@LucaMarconato
testing the copying of metadata and their inclusion in assert_element…
f5f1098
…s_are_identical
@LucaMarconato
improved readwrite tests
695a1d8
@LucaMarconato
added tests for incremental io
a0974ab
@LucaMarconato
implemented write_metadata for transformations
b429cc9
@LucaMarconato
tests for incremental io of transformation, with separate validation for
d5be580
writing metadata incrementally
@LucaMarconato
tests for IO and incremental IO of consolidated metadata
a9cb077
@LucaMarconato
improved control over elements only on-disk/in-memory
e13075d
@LucaMarconato
added tests for delete_element_from_disk
fbf9e3c
@LucaMarconato
fix
ff85b2e
@LucaMarconato
added _check_element_not_on_disk_with_different_type()
239e0a7
@LucaMarconato
updated changelog
6b8069b
@LucaMarconato
fixed changelog
1bfe022
@LucaMarconato
attempt fix docs
87dd1a8
@LucaMarconato@kevinyamauchi
Update src/spatialdata/_io/_utils.py
7f2ec2d
Co-authored-by: Kevin Yamauchi <kevin.yamauchi@gmail.com>
@LucaMarconato
fixes from review
bee39fc
LucaMarconatoand others added 16 commits March 27, 2024 21:04
@LucaMarconato
Merge branch 'feature/incremental_io' ofhttps://github.com/scverse/s…
30a30ff
…patialdata into feature/incremental_io
@LucaMarconato
Merge branch 'main' into feature/incremental_io
d277eea
@ArneDefauw@pre-commit-ci@LucaMarconato
update test read write on disk (scverse#515)
57dd22e
* test read write on disk
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* improved tests for workarounds for incremental io
* fixed tests
* improved comment
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Luca Marconato <m.lucalmer@gmail.com>
@LucaMarconato@giovp
Update src/spatialdata/_core/spatialdata.py
582622f
Co-authored-by: Giovanni Palla <25887487+giovp@users.noreply.github.com>
@LucaMarconato
list of names for write_element() and delete_element_from_disk()
98037eb
@LucaMarconato
Merge branch 'main' into feature/incremental_io
5ee00c5
@LucaMarconato
improved docs
f2bea77
@berombau
Merge branch 'main' ofhttps://github.com/scverse/spatialdata
1145ec6
@berombau
Merge remote-tracking branch 'upstream/feature/incremental_io'
f1d4f27
@berombau
add upath support
7d9c70b
@berombau
move _open_zarr_store to core utils
1e1c251
@berombau
use _open_zarr_store instead of parse_url; use UPath instead of Path
439d0d2
@berombau
use zmetadata always instead of .zmetadata
74283b1
@berombau
Merge branch 'scverse:main' into main
e6b6ae9
@berombau
add remote tests with moto
6e04606
@berombau
update remote tests, work on consolidated http stores
e3e0c28
@giovp

giovp commented Apr 23, 2024

Copy link
Copy Markdown
Member

for some reason it didn't rebase to main...

@giovp

giovp commented Jul 1, 2024

Copy link
Copy Markdown
Member

hi @berombau seems that there are a lot of conflicts, and still didnt' rebase correctly, not sure what to do but maybe worth to port over the changes to separate PR?

@berombau

berombau commented Jul 3, 2024

Copy link
Copy Markdown
ContributorAuthor

Yeah, I think I switched from merging to rebasing somewhere halfway 😅 I'll try to cherry-pick the changes on a new branch and do a cleaner PR.

@berombauberombau closed this Jul 3, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Support for private remote object storage

4 participants

@berombau@giovp@LucaMarconato@ArneDefauw