New grid object for structured grids by VeckoTheGecko · Pull Request #1966 · Parcels-code/Parcels

VeckoTheGecko · 2025-04-08T07:44:00Z

Currently making this PR for visibility and so that we can align our approaches between #1946 and this.

Note that this PR only adds the new Grid object (along with a GridAdapter object) but doesn't use them in the larger codebase. That will be for future development.

Just to clarify, I'm not 100% sure if GridAdapter will make it into the final v4 and can't even particularly comment on its usefulness due to the change of state/grid manipulation done by the rest of the codebase. It might be better to bring the rest of the codebase to work with the new Grid object.

Chose the correct base branch (v4-dev for v4 changes)
[] Fixes #
Added tests
Added documentation

cc @fluidnumerics-joe

fluidnumericsJoe · 2025-04-08T18:16:08Z

@VeckoTheGecko - Do we really want to maintain a duplicate of code already provided by xgcm ? ie, why wouldn't we want to just use xgcm as a dependency ?

VeckoTheGecko · 2025-04-09T09:17:49Z

Oh yes, I should have mentioned this in the issue body. I think we should vendor as opposed to adding it as a dependency for the following reasons:

xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable. Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

xgcm doesn't look to have active maintainers (source). Their last release was in 2022 (they have an intiative for a new release though Time to issue a new release xgcm/xgcm#670). The project is still stable though from what I can tell

fluidnumericsJoe · 2025-04-09T12:09:47Z

All good reasons.

VeckoTheGecko · 2025-04-09T13:03:01Z

For context as well xgcm/xgcm#658

VeckoTheGecko · 2025-04-10T12:46:05Z

I think that this is ready for review. Post-merge we can work out how to integrate this into Field, and decide if GridAdapter is needed after all

Note that the datasets.py file was also taken from xgcm

Remove tests that are no longer relevant

Assuming for now that left f-points are given for all Grids (which I think is an assumption that held for the grid class in Parcels v3)

jbusecke · 2025-08-21T00:01:43Z

Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

* xgcm doesn't look to have active maintainers ([source](https://discourse.pangeo.io/t/status-of-xgcm/4986/5)). Their last release was in 2022 (they have an intiative for a new release though [Time to issue a new release xgcm/xgcm#670](https://github.com/xgcm/xgcm/issues/670)). The project is still stable though from what I can tell

Hey @VeckoTheGecko sorry for the late comment (we just released xgcm 0.9.0 and I saw the link). I know we chatted about this personally, but I also wanted to state this publicly.

1. xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable.

I would be curious what parts of the API you are using here. I do not consider the Axis/Grid as private to be honest. We abandoned the idea of dropping axis, so I do not think that will be a risk for you folks here. I also think that the grid-awareness (and associated metadata parsing) is really the 'core product' of xgcm, whereas the calculus could at this point be entirely coupled out into one or many different packages that supply grid_ufuncs.

As you know my time is limited at the moment but xgcm is not dead. Might be worth checking out the new release too!

VeckoTheGecko · 2025-04-08T07:44:00Z

Currently making this PR for visibility and so that we can align our approaches between #1946 and this.

Note that this PR only adds the new Grid object (along with a GridAdapter object) but doesn't use them in the larger codebase. That will be for future development.

Just to clarify, I'm not 100% sure if GridAdapter will make it into the final v4 and can't even particularly comment on its usefulness due to the change of state/grid manipulation done by the rest of the codebase. It might be better to bring the rest of the codebase to work with the new Grid object.

Chose the correct base branch (v4-dev for v4 changes)
[] Fixes #
Added tests
Added documentation

cc @fluidnumerics-joe

fluidnumericsJoe · 2025-04-08T18:16:08Z

@VeckoTheGecko - Do we really want to maintain a duplicate of code already provided by xgcm ? ie, why wouldn't we want to just use xgcm as a dependency ?

VeckoTheGecko · 2025-04-09T09:17:49Z

Oh yes, I should have mentioned this in the issue body. I think we should vendor as opposed to adding it as a dependency for the following reasons:

xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable. Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

xgcm doesn't look to have active maintainers (source). Their last release was in 2022 (they have an intiative for a new release though Time to issue a new release xgcm/xgcm#670). The project is still stable though from what I can tell

fluidnumericsJoe · 2025-04-09T12:09:47Z

All good reasons.

VeckoTheGecko · 2025-04-09T13:03:01Z

For context as well xgcm/xgcm#658

VeckoTheGecko · 2025-04-10T12:46:05Z

I think that this is ready for review. Post-merge we can work out how to integrate this into Field, and decide if GridAdapter is needed after all

Note that the datasets.py file was also taken from xgcm

Remove tests that are no longer relevant

Assuming for now that left f-points are given for all Grids (which I think is an assumption that held for the grid class in Parcels v3)

jbusecke · 2025-08-21T00:01:43Z

Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

* xgcm doesn't look to have active maintainers ([source](https://discourse.pangeo.io/t/status-of-xgcm/4986/5)). Their last release was in 2022 (they have an intiative for a new release though [Time to issue a new release xgcm/xgcm#670](https://github.com/xgcm/xgcm/issues/670)). The project is still stable though from what I can tell

Hey @VeckoTheGecko sorry for the late comment (we just released xgcm 0.9.0 and I saw the link). I know we chatted about this personally, but I also wanted to state this publicly.

1. xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable.

I would be curious what parts of the API you are using here. I do not consider the Axis/Grid as private to be honest. We abandoned the idea of dropping axis, so I do not think that will be a risk for you folks here. I also think that the grid-awareness (and associated metadata parsing) is really the 'core product' of xgcm, whereas the calculus could at this point be entirely coupled out into one or many different packages that supply grid_ufuncs.

As you know my time is limited at the moment but xgcm is not dead. Might be worth checking out the new release too!

VeckoTheGecko · 2025-04-08T07:44:00Z

Currently making this PR for visibility and so that we can align our approaches between #1946 and this.

Note that this PR only adds the new Grid object (along with a GridAdapter object) but doesn't use them in the larger codebase. That will be for future development.

Just to clarify, I'm not 100% sure if GridAdapter will make it into the final v4 and can't even particularly comment on its usefulness due to the change of state/grid manipulation done by the rest of the codebase. It might be better to bring the rest of the codebase to work with the new Grid object.

Chose the correct base branch (v4-dev for v4 changes)
[] Fixes #
Added tests
Added documentation

cc @fluidnumerics-joe

fluidnumericsJoe · 2025-04-08T18:16:08Z

@VeckoTheGecko - Do we really want to maintain a duplicate of code already provided by xgcm ? ie, why wouldn't we want to just use xgcm as a dependency ?

VeckoTheGecko · 2025-04-09T09:17:49Z

Oh yes, I should have mentioned this in the issue body. I think we should vendor as opposed to adding it as a dependency for the following reasons:

xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable. Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

xgcm doesn't look to have active maintainers (source). Their last release was in 2022 (they have an intiative for a new release though Time to issue a new release xgcm/xgcm#670). The project is still stable though from what I can tell

fluidnumericsJoe · 2025-04-09T12:09:47Z

All good reasons.

VeckoTheGecko · 2025-04-09T13:03:01Z

For context as well xgcm/xgcm#658

VeckoTheGecko · 2025-04-10T12:46:05Z

I think that this is ready for review. Post-merge we can work out how to integrate this into Field, and decide if GridAdapter is needed after all

Note that the datasets.py file was also taken from xgcm

Remove tests that are no longer relevant

Assuming for now that left f-points are given for all Grids (which I think is an assumption that held for the grid class in Parcels v3)

jbusecke · 2025-08-21T00:01:43Z

Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

* xgcm doesn't look to have active maintainers ([source](https://discourse.pangeo.io/t/status-of-xgcm/4986/5)). Their last release was in 2022 (they have an intiative for a new release though [Time to issue a new release xgcm/xgcm#670](https://github.com/xgcm/xgcm/issues/670)). The project is still stable though from what I can tell

Hey @VeckoTheGecko sorry for the late comment (we just released xgcm 0.9.0 and I saw the link). I know we chatted about this personally, but I also wanted to state this publicly.

1. xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable.

I would be curious what parts of the API you are using here. I do not consider the Axis/Grid as private to be honest. We abandoned the idea of dropping axis, so I do not think that will be a risk for you folks here. I also think that the grid-awareness (and associated metadata parsing) is really the 'core product' of xgcm, whereas the calculus could at this point be entirely coupled out into one or many different packages that supply grid_ufuncs.

As you know my time is limited at the moment but xgcm is not dead. Might be worth checking out the new release too!

VeckoTheGecko · 2025-04-08T07:44:00Z

Currently making this PR for visibility and so that we can align our approaches between #1946 and this.

Note that this PR only adds the new Grid object (along with a GridAdapter object) but doesn't use them in the larger codebase. That will be for future development.

Just to clarify, I'm not 100% sure if GridAdapter will make it into the final v4 and can't even particularly comment on its usefulness due to the change of state/grid manipulation done by the rest of the codebase. It might be better to bring the rest of the codebase to work with the new Grid object.

Chose the correct base branch (v4-dev for v4 changes)
[] Fixes #
Added tests
Added documentation

cc @fluidnumerics-joe

fluidnumericsJoe · 2025-04-08T18:16:08Z

@VeckoTheGecko - Do we really want to maintain a duplicate of code already provided by xgcm ? ie, why wouldn't we want to just use xgcm as a dependency ?

VeckoTheGecko · 2025-04-09T09:17:49Z

Oh yes, I should have mentioned this in the issue body. I think we should vendor as opposed to adding it as a dependency for the following reasons:

xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable. Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

xgcm doesn't look to have active maintainers (source). Their last release was in 2022 (they have an intiative for a new release though Time to issue a new release xgcm/xgcm#670). The project is still stable though from what I can tell

fluidnumericsJoe · 2025-04-09T12:09:47Z

All good reasons.

VeckoTheGecko · 2025-04-09T13:03:01Z

For context as well xgcm/xgcm#658

VeckoTheGecko · 2025-04-10T12:46:05Z

I think that this is ready for review. Post-merge we can work out how to integrate this into Field, and decide if GridAdapter is needed after all

Note that the datasets.py file was also taken from xgcm

Remove tests that are no longer relevant

Assuming for now that left f-points are given for all Grids (which I think is an assumption that held for the grid class in Parcels v3)

jbusecke · 2025-08-21T00:01:43Z

Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

* xgcm doesn't look to have active maintainers ([source](https://discourse.pangeo.io/t/status-of-xgcm/4986/5)). Their last release was in 2022 (they have an intiative for a new release though [Time to issue a new release xgcm/xgcm#670](https://github.com/xgcm/xgcm/issues/670)). The project is still stable though from what I can tell

Hey @VeckoTheGecko sorry for the late comment (we just released xgcm 0.9.0 and I saw the link). I know we chatted about this personally, but I also wanted to state this publicly.

1. xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable.

I would be curious what parts of the API you are using here. I do not consider the Axis/Grid as private to be honest. We abandoned the idea of dropping axis, so I do not think that will be a risk for you folks here. I also think that the grid-awareness (and associated metadata parsing) is really the 'core product' of xgcm, whereas the calculus could at this point be entirely coupled out into one or many different packages that supply grid_ufuncs.

As you know my time is limited at the moment but xgcm is not dead. Might be worth checking out the new release too!

VeckoTheGecko · 2025-04-08T07:44:00Z

Currently making this PR for visibility and so that we can align our approaches between #1946 and this.

Note that this PR only adds the new Grid object (along with a GridAdapter object) but doesn't use them in the larger codebase. That will be for future development.

Just to clarify, I'm not 100% sure if GridAdapter will make it into the final v4 and can't even particularly comment on its usefulness due to the change of state/grid manipulation done by the rest of the codebase. It might be better to bring the rest of the codebase to work with the new Grid object.

Chose the correct base branch (v4-dev for v4 changes)
[] Fixes #
Added tests
Added documentation

cc @fluidnumerics-joe

fluidnumericsJoe · 2025-04-08T18:16:08Z

@VeckoTheGecko - Do we really want to maintain a duplicate of code already provided by xgcm ? ie, why wouldn't we want to just use xgcm as a dependency ?

VeckoTheGecko · 2025-04-09T09:17:49Z

Oh yes, I should have mentioned this in the issue body. I think we should vendor as opposed to adding it as a dependency for the following reasons:

xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable. Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

xgcm doesn't look to have active maintainers (source). Their last release was in 2022 (they have an intiative for a new release though Time to issue a new release xgcm/xgcm#670). The project is still stable though from what I can tell

fluidnumericsJoe · 2025-04-09T12:09:47Z

All good reasons.

VeckoTheGecko · 2025-04-09T13:03:01Z

For context as well xgcm/xgcm#658

VeckoTheGecko · 2025-04-10T12:46:05Z

I think that this is ready for review. Post-merge we can work out how to integrate this into Field, and decide if GridAdapter is needed after all

Note that the datasets.py file was also taken from xgcm

Remove tests that are no longer relevant

Assuming for now that left f-points are given for all Grids (which I think is an assumption that held for the grid class in Parcels v3)

jbusecke · 2025-08-21T00:01:43Z

Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

* xgcm doesn't look to have active maintainers ([source](https://discourse.pangeo.io/t/status-of-xgcm/4986/5)). Their last release was in 2022 (they have an intiative for a new release though [Time to issue a new release xgcm/xgcm#670](https://github.com/xgcm/xgcm/issues/670)). The project is still stable though from what I can tell

Hey @VeckoTheGecko sorry for the late comment (we just released xgcm 0.9.0 and I saw the link). I know we chatted about this personally, but I also wanted to state this publicly.

1. xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable.

I would be curious what parts of the API you are using here. I do not consider the Axis/Grid as private to be honest. We abandoned the idea of dropping axis, so I do not think that will be a risk for you folks here. I also think that the grid-awareness (and associated metadata parsing) is really the 'core product' of xgcm, whereas the calculus could at this point be entirely coupled out into one or many different packages that supply grid_ufuncs.

As you know my time is limited at the moment but xgcm is not dead. Might be worth checking out the new release too!

VeckoTheGecko · 2025-04-08T07:44:00Z

Currently making this PR for visibility and so that we can align our approaches between #1946 and this.

Note that this PR only adds the new Grid object (along with a GridAdapter object) but doesn't use them in the larger codebase. That will be for future development.

Just to clarify, I'm not 100% sure if GridAdapter will make it into the final v4 and can't even particularly comment on its usefulness due to the change of state/grid manipulation done by the rest of the codebase. It might be better to bring the rest of the codebase to work with the new Grid object.

Chose the correct base branch (v4-dev for v4 changes)
[] Fixes #
Added tests
Added documentation

cc @fluidnumerics-joe

fluidnumericsJoe · 2025-04-08T18:16:08Z

@VeckoTheGecko - Do we really want to maintain a duplicate of code already provided by xgcm ? ie, why wouldn't we want to just use xgcm as a dependency ?

VeckoTheGecko · 2025-04-09T09:17:49Z

Oh yes, I should have mentioned this in the issue body. I think we should vendor as opposed to adding it as a dependency for the following reasons:

xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable. Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

xgcm doesn't look to have active maintainers (source). Their last release was in 2022 (they have an intiative for a new release though Time to issue a new release xgcm/xgcm#670). The project is still stable though from what I can tell

fluidnumericsJoe · 2025-04-09T12:09:47Z

All good reasons.

VeckoTheGecko · 2025-04-09T13:03:01Z

For context as well xgcm/xgcm#658

VeckoTheGecko · 2025-04-10T12:46:05Z

I think that this is ready for review. Post-merge we can work out how to integrate this into Field, and decide if GridAdapter is needed after all

Note that the datasets.py file was also taken from xgcm

Remove tests that are no longer relevant

Assuming for now that left f-points are given for all Grids (which I think is an assumption that held for the grid class in Parcels v3)

jbusecke · 2025-08-21T00:01:43Z

Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

* xgcm doesn't look to have active maintainers ([source](https://discourse.pangeo.io/t/status-of-xgcm/4986/5)). Their last release was in 2022 (they have an intiative for a new release though [Time to issue a new release xgcm/xgcm#670](https://github.com/xgcm/xgcm/issues/670)). The project is still stable though from what I can tell

Hey @VeckoTheGecko sorry for the late comment (we just released xgcm 0.9.0 and I saw the link). I know we chatted about this personally, but I also wanted to state this publicly.

1. xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable.

I would be curious what parts of the API you are using here. I do not consider the Axis/Grid as private to be honest. We abandoned the idea of dropping axis, so I do not think that will be a risk for you folks here. I also think that the grid-awareness (and associated metadata parsing) is really the 'core product' of xgcm, whereas the calculus could at this point be entirely coupled out into one or many different packages that supply grid_ufuncs.

As you know my time is limited at the moment but xgcm is not dead. Might be worth checking out the new release too!

VeckoTheGecko · 2025-04-08T07:44:00Z

Currently making this PR for visibility and so that we can align our approaches between #1946 and this.

Note that this PR only adds the new Grid object (along with a GridAdapter object) but doesn't use them in the larger codebase. That will be for future development.

Just to clarify, I'm not 100% sure if GridAdapter will make it into the final v4 and can't even particularly comment on its usefulness due to the change of state/grid manipulation done by the rest of the codebase. It might be better to bring the rest of the codebase to work with the new Grid object.

Chose the correct base branch (v4-dev for v4 changes)
[] Fixes #
Added tests
Added documentation

cc @fluidnumerics-joe

fluidnumericsJoe · 2025-04-08T18:16:08Z

@VeckoTheGecko - Do we really want to maintain a duplicate of code already provided by xgcm ? ie, why wouldn't we want to just use xgcm as a dependency ?

VeckoTheGecko · 2025-04-09T09:17:49Z

Oh yes, I should have mentioned this in the issue body. I think we should vendor as opposed to adding it as a dependency for the following reasons:

xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable. Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

xgcm doesn't look to have active maintainers (source). Their last release was in 2022 (they have an intiative for a new release though Time to issue a new release xgcm/xgcm#670). The project is still stable though from what I can tell

fluidnumericsJoe · 2025-04-09T12:09:47Z

All good reasons.

VeckoTheGecko · 2025-04-09T13:03:01Z

For context as well xgcm/xgcm#658

VeckoTheGecko · 2025-04-10T12:46:05Z

I think that this is ready for review. Post-merge we can work out how to integrate this into Field, and decide if GridAdapter is needed after all

Note that the datasets.py file was also taken from xgcm

Remove tests that are no longer relevant

Assuming for now that left f-points are given for all Grids (which I think is an assumption that held for the grid class in Parcels v3)

jbusecke · 2025-08-21T00:01:43Z

Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

* xgcm doesn't look to have active maintainers ([source](https://discourse.pangeo.io/t/status-of-xgcm/4986/5)). Their last release was in 2022 (they have an intiative for a new release though [Time to issue a new release xgcm/xgcm#670](https://github.com/xgcm/xgcm/issues/670)). The project is still stable though from what I can tell

Hey @VeckoTheGecko sorry for the late comment (we just released xgcm 0.9.0 and I saw the link). I know we chatted about this personally, but I also wanted to state this publicly.

1. xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable.

I would be curious what parts of the API you are using here. I do not consider the Axis/Grid as private to be honest. We abandoned the idea of dropping axis, so I do not think that will be a risk for you folks here. I also think that the grid-awareness (and associated metadata parsing) is really the 'core product' of xgcm, whereas the calculus could at this point be entirely coupled out into one or many different packages that supply grid_ufuncs.

As you know my time is limited at the moment but xgcm is not dead. Might be worth checking out the new release too!

VeckoTheGecko · 2025-04-08T07:44:00Z

Currently making this PR for visibility and so that we can align our approaches between #1946 and this.

Note that this PR only adds the new Grid object (along with a GridAdapter object) but doesn't use them in the larger codebase. That will be for future development.

Just to clarify, I'm not 100% sure if GridAdapter will make it into the final v4 and can't even particularly comment on its usefulness due to the change of state/grid manipulation done by the rest of the codebase. It might be better to bring the rest of the codebase to work with the new Grid object.

Chose the correct base branch (v4-dev for v4 changes)
[] Fixes #
Added tests
Added documentation

cc @fluidnumerics-joe

fluidnumericsJoe · 2025-04-08T18:16:08Z

@VeckoTheGecko - Do we really want to maintain a duplicate of code already provided by xgcm ? ie, why wouldn't we want to just use xgcm as a dependency ?

VeckoTheGecko · 2025-04-09T09:17:49Z

Oh yes, I should have mentioned this in the issue body. I think we should vendor as opposed to adding it as a dependency for the following reasons:

xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable. Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

xgcm doesn't look to have active maintainers (source). Their last release was in 2022 (they have an intiative for a new release though Time to issue a new release xgcm/xgcm#670). The project is still stable though from what I can tell

fluidnumericsJoe · 2025-04-09T12:09:47Z

All good reasons.

VeckoTheGecko · 2025-04-09T13:03:01Z

For context as well xgcm/xgcm#658

VeckoTheGecko · 2025-04-10T12:46:05Z

I think that this is ready for review. Post-merge we can work out how to integrate this into Field, and decide if GridAdapter is needed after all

Note that the datasets.py file was also taken from xgcm

Remove tests that are no longer relevant

Assuming for now that left f-points are given for all Grids (which I think is an assumption that held for the grid class in Parcels v3)

jbusecke · 2025-08-21T00:01:43Z

Also they're looking to get rid of the Axis object from the public API Lets get rid of Axis xgcm/xgcm#405 , which I think further would make things difficult for us.

* xgcm doesn't look to have active maintainers ([source](https://discourse.pangeo.io/t/status-of-xgcm/4986/5)). Their last release was in 2022 (they have an intiative for a new release though [Time to issue a new release xgcm/xgcm#670](https://github.com/xgcm/xgcm/issues/670)). The project is still stable though from what I can tell

Hey @VeckoTheGecko sorry for the late comment (we just released xgcm 0.9.0 and I saw the link). I know we chatted about this personally, but I also wanted to state this publicly.

1. xgcm is quite a specific tool (i.e., enabling grid aware calculus on ocean grids). We only want the "grid aware" part, which (a) is not particularly part of the public API, and (b) is a relatively small part of their codebase . Depending on xgcm would couple us to their development, and relying on their private API would be potentially unstable.

I would be curious what parts of the API you are using here. I do not consider the Axis/Grid as private to be honest. We abandoned the idea of dropping axis, so I do not think that will be a risk for you folks here. I also think that the grid-awareness (and associated metadata parsing) is really the 'core product' of xgcm, whereas the calculus could at this point be entirely coupled out into one or many different packages that supply grid_ufuncs.

As you know my time is limited at the moment but xgcm is not dead. Might be worth checking out the new release too!

github-project-automationBot added this to Parcels development Apr 8, 2025

github-project-automationBot moved this to Backlog in Parcels development Apr 8, 2025

VeckoTheGecko force-pushed the v/new-grid branch from d7abe76 to 4f89ed5Compare April 8, 2025 07:56

VeckoTheGecko marked this pull request as ready for review April 10, 2025 12:46

VeckoTheGecko requested review from erikvansebille and fluidnumericsJoe April 10, 2025 12:46

erikvansebille reviewed Apr 10, 2025
View reviewed changes

Comment threadparcels/v4/comodo.py
Comment threadtests/v4/grid_datasets.py
Comment threadtests/vendor/xgcm_datasets.py Outdated

erikvansebille approved these changes Apr 10, 2025
View reviewed changes

fluidnumericsJoe approved these changes Apr 10, 2025
View reviewed changes

VeckoTheGecko added 16 commits April 11, 2025 16:05

Take xgcm.Grid and adapt for our uses
5659a2b
Note that the datasets.py file was also taken from xgcm

Adapt xgcm tests
f26a308
Remove tests that are no longer relevant

Remove unused functions and TODOs
b181643

Rename file to xgcm_datasets.py
72fadde

run pre-commit
b2ffce7

Move v4 grid files
5e9a145

Run pre-commit and manual fixes
899f4e3

Update reprs and remove warning
b552b03

Remove unused imports
0bffe9e

remove Grid._ti from old grid
ca7dcfc

Add grid adapter test cases and mock implementation
bdc7041

Define GridAdapter tdim, xdim, ydim and zdim
ff62de7

Add time to test_gridadapter dataset
f175eea

Add Z dim to test_gridadapter dataset
f7fa370

Define GridAdapter lon, lat, depth, and time
df93007
Assuming for now that left f-points are given for all Grids (which I think is an assumption that held for the grid class in Parcels v3)

Limit scope of keyerror except blocks
82b8271

VeckoTheGecko added 18 commits April 11, 2025 16:05

Add _z4d to gridadapter
ec49bc0

Move vendored file to vendor folder
8627730

Update test suite repr
86f4d38

Add curvilinear test grid datasets
c6de1cd

Update grid datasets with depth and time
78889e1

Move dataset and start with new test
4c56c06

Update test
29568b9

Use lon, lat, depth arrays on underlying dataset
70588bc

Add grid_type property to grid adapter
faf7a6c

Rename test file
b434eaa

Add mesh kwarg to adapter
9fdd993

Remove mesh property from adapter
1fd9b14

Remove kwargs from Grid.create_grid
42d3715

Update isinstance to GridType check
98436c6

Update grid_type on adapter to use enum
d34a45c

Remove GridCode alias
bc0f7b0

Add lonlat_minmax to GridAdapter
182a22b

review feedback
f52ef63

VeckoTheGecko force-pushed the v/new-grid branch from 1701a1a to f52ef63Compare April 11, 2025 14:05

Move dataset functions
f3475cc

VeckoTheGecko force-pushed the v/new-grid branch from 5bb7475 to f3475ccCompare April 11, 2025 14:13

VeckoTheGecko enabled auto-merge April 11, 2025 14:14

VeckoTheGecko added 2 commits April 15, 2025 10:47

Patch grid._gtype call and pixi run tests-notebooks
324047f

remove doubled up TimeConverter.__eq__
ca24d17

VeckoTheGecko disabled auto-merge April 15, 2025 09:34

VeckoTheGecko merged commit 1316f7a into v4-devApr 15, 2025

VeckoTheGecko deleted the v/new-grid branch April 15, 2025 09:34

github-project-automationBot moved this from Backlog to Done in Parcels development Apr 15, 2025

VeckoTheGecko mentioned this pull request Sep 9, 2025
Make xgcm a dependency and remove vendored code #2193
Closed