Skip to content

**BREAKING** pygmt.grdcut: Refactor to store output in virtualfiles for grids - #3115

Merged
seisman merged 105 commits into
mainfrom
gmtimage
Mar 12, 2025
Merged

**BREAKING** pygmt.grdcut: Refactor to store output in virtualfiles for grids#3115
seisman merged 105 commits into
mainfrom
gmtimage

Conversation

@seisman

@seismanseisman commented Mar 17, 2024

Copy link
Copy Markdown
Member

This PR applies virtualfiles in pygmt.grdcut, so that this function doesn't have to write to temporary files (#2730).

grdcut is on of the complicated module that it works for grid, image and cube. We have to know the kind of the input before calling the Session.virtualfile_out method.

It turns out it's not trivial to determine the data kind of an input file. We have some options:

  1. Based on file extensions (e.g., .nc is a grid, .tif is a image)
  2. Based on magic bytes (xref: https://en.wikipedia.org/wiki/List_of_file_signatures)
  3. Try reading as a grid/image/cube. If fail, then try the 2nd one.
  4. Add a new kind parameter (default to "grid") and users should specify the kind parameter.

Option 1-2 may not always work; option 3 also dosn't always work (see #3115 (comment)).

This PR currently adopts option 4. Need to note option 4 meaning BREAKING:

  • previously, grdcut("@earth_day_01d") works; now must use grdcut("@earth_day_01d", kind="image")
  • previously, grdcut works 1/3/4-band images; now it only supports 3-band images. [We can support 1/4-band images later]

seismanand others added 2 commits March 20, 2024 11:00
Comment threadpygmt/src/grdcut.py Outdated
Comment threadpygmt/clib/session.py Outdated
Comment threadpygmt/src/grdcut.py Outdated
@seismanseisman changed the title WIP: Initial try to support raster images using temporary filesWIP: pygmt.grdcut: Support both grids and imagesMar 27, 2024
@seisman
seismanforce-pushed the datatypes/gmtgrid branch 3 times, most recently from d8f1160 to d0517e0CompareApril 1, 2024 07:11
@seisman
seisman changed the base branch from datatypes/gmtgrid to mainApril 1, 2024 11:32
@seismanseisman added the enhancement Improving an existing feature label Apr 19, 2024
weiji14 added a commit that referenced this pull request Jun 18, 2024
Special case `earth_day` to be loaded as GMT_IMAGE rather than GMT_GRID. Need to use `GMT_OUT|GMT_IS_REFERENCE` in virtualfile_out to avoid segfault, xref #3115.
Extra metadata from the _GMT_GRID_HEADER struct.
Reorder the dimensions to follow Channel, Height, Width (CHW) convention. Also added doctest checking output DataArray object and the image's x and y coordinates.
Get the registration and gtype info from the grid header and apply it to the GMT accessor attributes.
Trying to match some of the doctests in _GMT_GRID.
@seisman
seisman marked this pull request as ready for review December 9, 2024 11:02
@seismanseisman changed the title pygmt.grdcut: Refactor to store output in virtualfiles for grids**BREAKING** pygmt.grdcut: Refactor to store output in virtualfiles for gridsDec 10, 2024
@seismanseisman mentioned this pull request Dec 19, 2024
49 tasks
@seismanseisman mentioned this pull request Dec 19, 2024
3 tasks
@seisman

Copy link
Copy Markdown
MemberAuthor

I think it's better to not introduce last-minute breaking changes, so bump this PR to v0.15.0.

@seismanseisman removed this from the 0.14.0 milestone Dec 24, 2024
@seismanseisman removed the needs review This PR has higher priority and needs review. label Dec 27, 2024
@seismanseisman modified the milestone: 0.15.0Dec 27, 2024
@seismanseisman added this to the 0.15.0 milestone Jan 5, 2025
@seismanseisman added the needs review This PR has higher priority and needs review. label Jan 5, 2025
@seismanseisman removed the needs review This PR has higher priority and needs review. label Feb 11, 2025

@weiji14weiji14 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still think using the private gmt_raster_type func as mentioned at #3115 (comment) would be the best solution to auto-detect the raster kind, but failing that, the logic here to explicitly have the user pass in the type works too.

Just one typo on the match-case statement, and should be ok to merge once all tests pass without coverage misses.

Comment threadpygmt/src/grdcut.py Outdated
@seismanseisman added final review call This PR requires final review and approval from a second reviewer and removed final review call This PR requires final review and approval from a second reviewer labels Mar 12, 2025
@seisman
seisman merged commit 6a88408 into mainMar 12, 2025
@seisman
seisman deleted the gmtimage branch March 12, 2025 08:40
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementImproving an existing feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@seisman@weiji14