Testing and fixing NEMO spatialhash index_search by erikvansebille · Pull Request #2153 · Parcels-code/Parcels

erikvansebille · 2025-08-19T07:23:16Z

This PR adds a unit test for spatial hash on 1/4 degree NEMO curvilinear grid (from https://github.com/OceanParcels/parcels-data/tree/main/data/NemoCurvilinear_data).

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875; which technically also surrounds the location) instead of the cell that snugly surrounds the location

Chose the correct base branch (main for v3 changes, v4-dev for v4 changes)
Fixes Adding script for benchmarking the curvilinear search function parcels-benchmarks#5
Added tests

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875)

erikvansebille · 2025-08-19T08:30:18Z

A simple change to the lines below seems to fix the failing unit test
https://github.com/OceanParcels/Parcels/blob/85f025be11cd122f8ac4bba463db2a97860c2549/parcels/spatialhash.py#L141-L150

- if xi1[eid] <= xi2[eid]:+ if abs(xi1[eid] - xi2[eid]) > 225:+ pass+ elif xi1[eid] <= xi2[eid]:

This way, the grid cells that span the antemeridian (where the left corner is 179.875 and the right corner is -179.875) are not included in the hash table. However, we would need to make this work only when the field _mesh_type="spherical" (because on flat grids, it's to be expected that grid cells have distances of more than 225 meters between them)

@fluidnumerics-joe, does this give you some pointers/ideas how to fix the spatial hash.py function to make the unit test in this PR pass?

PS: Note that the 'special' value of 225 used above is also present in the v3-version of _index_search:
https://github.com/OceanParcels/Parcels/blob/fb88390d2bc9e2d45cd3056c601c7acaa9afcdac/parcels/_index_search.py#L253-L254

erikvansebille · 2025-08-20T12:18:30Z

Here's a map of the longitude grid

as suggested in #2153 (comment)

erikvansebille · 2025-09-01T10:26:17Z

With the new Morton having, this test passes. Good to pull it into the test-suite?

erikvansebille · 2025-08-19T07:23:16Z

This PR adds a unit test for spatial hash on 1/4 degree NEMO curvilinear grid (from https://github.com/OceanParcels/parcels-data/tree/main/data/NemoCurvilinear_data).

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875; which technically also surrounds the location) instead of the cell that snugly surrounds the location

Chose the correct base branch (main for v3 changes, v4-dev for v4 changes)
Fixes Adding script for benchmarking the curvilinear search function parcels-benchmarks#5
Added tests

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875)

erikvansebille · 2025-08-19T08:30:18Z

A simple change to the lines below seems to fix the failing unit test
https://github.com/OceanParcels/Parcels/blob/85f025be11cd122f8ac4bba463db2a97860c2549/parcels/spatialhash.py#L141-L150

- if xi1[eid] <= xi2[eid]:+ if abs(xi1[eid] - xi2[eid]) > 225:+ pass+ elif xi1[eid] <= xi2[eid]:

This way, the grid cells that span the antemeridian (where the left corner is 179.875 and the right corner is -179.875) are not included in the hash table. However, we would need to make this work only when the field _mesh_type="spherical" (because on flat grids, it's to be expected that grid cells have distances of more than 225 meters between them)

@fluidnumerics-joe, does this give you some pointers/ideas how to fix the spatial hash.py function to make the unit test in this PR pass?

PS: Note that the 'special' value of 225 used above is also present in the v3-version of _index_search:
https://github.com/OceanParcels/Parcels/blob/fb88390d2bc9e2d45cd3056c601c7acaa9afcdac/parcels/_index_search.py#L253-L254

erikvansebille · 2025-08-20T12:18:30Z

Here's a map of the longitude grid

as suggested in #2153 (comment)

erikvansebille · 2025-09-01T10:26:17Z

With the new Morton having, this test passes. Good to pull it into the test-suite?

erikvansebille · 2025-08-19T07:23:16Z

This PR adds a unit test for spatial hash on 1/4 degree NEMO curvilinear grid (from https://github.com/OceanParcels/parcels-data/tree/main/data/NemoCurvilinear_data).

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875; which technically also surrounds the location) instead of the cell that snugly surrounds the location

Chose the correct base branch (main for v3 changes, v4-dev for v4 changes)
Fixes Adding script for benchmarking the curvilinear search function parcels-benchmarks#5
Added tests

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875)

erikvansebille · 2025-08-19T08:30:18Z

A simple change to the lines below seems to fix the failing unit test
https://github.com/OceanParcels/Parcels/blob/85f025be11cd122f8ac4bba463db2a97860c2549/parcels/spatialhash.py#L141-L150

- if xi1[eid] <= xi2[eid]:+ if abs(xi1[eid] - xi2[eid]) > 225:+ pass+ elif xi1[eid] <= xi2[eid]:

This way, the grid cells that span the antemeridian (where the left corner is 179.875 and the right corner is -179.875) are not included in the hash table. However, we would need to make this work only when the field _mesh_type="spherical" (because on flat grids, it's to be expected that grid cells have distances of more than 225 meters between them)

@fluidnumerics-joe, does this give you some pointers/ideas how to fix the spatial hash.py function to make the unit test in this PR pass?

PS: Note that the 'special' value of 225 used above is also present in the v3-version of _index_search:
https://github.com/OceanParcels/Parcels/blob/fb88390d2bc9e2d45cd3056c601c7acaa9afcdac/parcels/_index_search.py#L253-L254

erikvansebille · 2025-08-20T12:18:30Z

Here's a map of the longitude grid

as suggested in #2153 (comment)

erikvansebille · 2025-09-01T10:26:17Z

With the new Morton having, this test passes. Good to pull it into the test-suite?

erikvansebille · 2025-08-19T07:23:16Z

This PR adds a unit test for spatial hash on 1/4 degree NEMO curvilinear grid (from https://github.com/OceanParcels/parcels-data/tree/main/data/NemoCurvilinear_data).

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875; which technically also surrounds the location) instead of the cell that snugly surrounds the location

Chose the correct base branch (main for v3 changes, v4-dev for v4 changes)
Fixes Adding script for benchmarking the curvilinear search function parcels-benchmarks#5
Added tests

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875)

erikvansebille · 2025-08-19T08:30:18Z

A simple change to the lines below seems to fix the failing unit test
https://github.com/OceanParcels/Parcels/blob/85f025be11cd122f8ac4bba463db2a97860c2549/parcels/spatialhash.py#L141-L150

- if xi1[eid] <= xi2[eid]:+ if abs(xi1[eid] - xi2[eid]) > 225:+ pass+ elif xi1[eid] <= xi2[eid]:

This way, the grid cells that span the antemeridian (where the left corner is 179.875 and the right corner is -179.875) are not included in the hash table. However, we would need to make this work only when the field _mesh_type="spherical" (because on flat grids, it's to be expected that grid cells have distances of more than 225 meters between them)

@fluidnumerics-joe, does this give you some pointers/ideas how to fix the spatial hash.py function to make the unit test in this PR pass?

PS: Note that the 'special' value of 225 used above is also present in the v3-version of _index_search:
https://github.com/OceanParcels/Parcels/blob/fb88390d2bc9e2d45cd3056c601c7acaa9afcdac/parcels/_index_search.py#L253-L254

erikvansebille · 2025-08-20T12:18:30Z

Here's a map of the longitude grid

as suggested in #2153 (comment)

erikvansebille · 2025-09-01T10:26:17Z

With the new Morton having, this test passes. Good to pull it into the test-suite?

erikvansebille · 2025-08-19T07:23:16Z

This PR adds a unit test for spatial hash on 1/4 degree NEMO curvilinear grid (from https://github.com/OceanParcels/parcels-data/tree/main/data/NemoCurvilinear_data).

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875; which technically also surrounds the location) instead of the cell that snugly surrounds the location

Chose the correct base branch (main for v3 changes, v4-dev for v4 changes)
Fixes Adding script for benchmarking the curvilinear search function parcels-benchmarks#5
Added tests

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875)

erikvansebille · 2025-08-19T08:30:18Z

A simple change to the lines below seems to fix the failing unit test
https://github.com/OceanParcels/Parcels/blob/85f025be11cd122f8ac4bba463db2a97860c2549/parcels/spatialhash.py#L141-L150

- if xi1[eid] <= xi2[eid]:+ if abs(xi1[eid] - xi2[eid]) > 225:+ pass+ elif xi1[eid] <= xi2[eid]:

This way, the grid cells that span the antemeridian (where the left corner is 179.875 and the right corner is -179.875) are not included in the hash table. However, we would need to make this work only when the field _mesh_type="spherical" (because on flat grids, it's to be expected that grid cells have distances of more than 225 meters between them)

@fluidnumerics-joe, does this give you some pointers/ideas how to fix the spatial hash.py function to make the unit test in this PR pass?

PS: Note that the 'special' value of 225 used above is also present in the v3-version of _index_search:
https://github.com/OceanParcels/Parcels/blob/fb88390d2bc9e2d45cd3056c601c7acaa9afcdac/parcels/_index_search.py#L253-L254

erikvansebille · 2025-08-20T12:18:30Z

Here's a map of the longitude grid

as suggested in #2153 (comment)

erikvansebille · 2025-09-01T10:26:17Z

With the new Morton having, this test passes. Good to pull it into the test-suite?

erikvansebille · 2025-08-19T07:23:16Z

This PR adds a unit test for spatial hash on 1/4 degree NEMO curvilinear grid (from https://github.com/OceanParcels/parcels-data/tree/main/data/NemoCurvilinear_data).

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875; which technically also surrounds the location) instead of the cell that snugly surrounds the location

Chose the correct base branch (main for v3 changes, v4-dev for v4 changes)
Fixes Adding script for benchmarking the curvilinear search function parcels-benchmarks#5
Added tests

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875)

erikvansebille · 2025-08-19T08:30:18Z

A simple change to the lines below seems to fix the failing unit test
https://github.com/OceanParcels/Parcels/blob/85f025be11cd122f8ac4bba463db2a97860c2549/parcels/spatialhash.py#L141-L150

- if xi1[eid] <= xi2[eid]:+ if abs(xi1[eid] - xi2[eid]) > 225:+ pass+ elif xi1[eid] <= xi2[eid]:

This way, the grid cells that span the antemeridian (where the left corner is 179.875 and the right corner is -179.875) are not included in the hash table. However, we would need to make this work only when the field _mesh_type="spherical" (because on flat grids, it's to be expected that grid cells have distances of more than 225 meters between them)

@fluidnumerics-joe, does this give you some pointers/ideas how to fix the spatial hash.py function to make the unit test in this PR pass?

PS: Note that the 'special' value of 225 used above is also present in the v3-version of _index_search:
https://github.com/OceanParcels/Parcels/blob/fb88390d2bc9e2d45cd3056c601c7acaa9afcdac/parcels/_index_search.py#L253-L254

erikvansebille · 2025-08-20T12:18:30Z

Here's a map of the longitude grid

as suggested in #2153 (comment)

erikvansebille · 2025-09-01T10:26:17Z

With the new Morton having, this test passes. Good to pull it into the test-suite?

erikvansebille · 2025-08-19T07:23:16Z

This PR adds a unit test for spatial hash on 1/4 degree NEMO curvilinear grid (from https://github.com/OceanParcels/parcels-data/tree/main/data/NemoCurvilinear_data).

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875; which technically also surrounds the location) instead of the cell that snugly surrounds the location

Chose the correct base branch (main for v3 changes, v4-dev for v4 changes)
Fixes Adding script for benchmarking the curvilinear search function parcels-benchmarks#5
Added tests

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875)

erikvansebille · 2025-08-19T08:30:18Z

A simple change to the lines below seems to fix the failing unit test
https://github.com/OceanParcels/Parcels/blob/85f025be11cd122f8ac4bba463db2a97860c2549/parcels/spatialhash.py#L141-L150

- if xi1[eid] <= xi2[eid]:+ if abs(xi1[eid] - xi2[eid]) > 225:+ pass+ elif xi1[eid] <= xi2[eid]:

This way, the grid cells that span the antemeridian (where the left corner is 179.875 and the right corner is -179.875) are not included in the hash table. However, we would need to make this work only when the field _mesh_type="spherical" (because on flat grids, it's to be expected that grid cells have distances of more than 225 meters between them)

@fluidnumerics-joe, does this give you some pointers/ideas how to fix the spatial hash.py function to make the unit test in this PR pass?

PS: Note that the 'special' value of 225 used above is also present in the v3-version of _index_search:
https://github.com/OceanParcels/Parcels/blob/fb88390d2bc9e2d45cd3056c601c7acaa9afcdac/parcels/_index_search.py#L253-L254

erikvansebille · 2025-08-20T12:18:30Z

Here's a map of the longitude grid

as suggested in #2153 (comment)

erikvansebille · 2025-09-01T10:26:17Z

With the new Morton having, this test passes. Good to pull it into the test-suite?

erikvansebille · 2025-08-19T07:23:16Z

This PR adds a unit test for spatial hash on 1/4 degree NEMO curvilinear grid (from https://github.com/OceanParcels/parcels-data/tree/main/data/NemoCurvilinear_data).

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875; which technically also surrounds the location) instead of the cell that snugly surrounds the location

Chose the correct base branch (main for v3 changes, v4-dev for v4 changes)
Fixes Adding script for benchmarking the curvilinear search function parcels-benchmarks#5
Added tests

The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875)

erikvansebille · 2025-08-19T08:30:18Z

A simple change to the lines below seems to fix the failing unit test
https://github.com/OceanParcels/Parcels/blob/85f025be11cd122f8ac4bba463db2a97860c2549/parcels/spatialhash.py#L141-L150

- if xi1[eid] <= xi2[eid]:+ if abs(xi1[eid] - xi2[eid]) > 225:+ pass+ elif xi1[eid] <= xi2[eid]:

This way, the grid cells that span the antemeridian (where the left corner is 179.875 and the right corner is -179.875) are not included in the hash table. However, we would need to make this work only when the field _mesh_type="spherical" (because on flat grids, it's to be expected that grid cells have distances of more than 225 meters between them)

@fluidnumerics-joe, does this give you some pointers/ideas how to fix the spatial hash.py function to make the unit test in this PR pass?

PS: Note that the 'special' value of 225 used above is also present in the v3-version of _index_search:
https://github.com/OceanParcels/Parcels/blob/fb88390d2bc9e2d45cd3056c601c7acaa9afcdac/parcels/_index_search.py#L253-L254

erikvansebille · 2025-08-20T12:18:30Z

Here's a map of the longitude grid

as suggested in #2153 (comment)

erikvansebille · 2025-09-01T10:26:17Z

With the new Morton having, this test passes. Good to pull it into the test-suite?

Adding unit test for spatial hash on 1/4 degree NEMO curvilinear grid
85f025b
The unit test currently fails, because the spatial hash index returned is the cell on the anti-meridian (between 179.875 and 179.875)

github-project-automationBot added this to Parcels development Aug 19, 2025

github-project-automationBot moved this to Backlog in Parcels development Aug 19, 2025

erikvansebille mentioned this pull request Aug 19, 2025
Adding script for benchmarking the curvilinear search function Parcels-code/parcels-benchmarks#5
Draft

Base automatically changed from vectorized-kernel to v4-devAugust 19, 2025 18:26

Merge branch 'v4-dev' into testing_nemo_spatialhash
fadca90

erikvansebille added a commit that referenced this pull request Aug 22, 2025
Temporary fix to spatialhash
1a64033
as suggested in #2153 (comment)

erikvansebille added 2 commits September 1, 2025 12:23

Merge branch 'v4-dev' into testing_nemo_spatialhash
e476ebe

Vectorizing _reconnect_bnd_indices
3eb2803

erikvansebille marked this pull request as ready for review September 1, 2025 10:26

erikvansebille requested a review from VeckoTheGecko September 1, 2025 10:26

VeckoTheGecko approved these changes Sep 2, 2025
View reviewed changes

Comment threadtests/v4/test_index_search.py

Merge branch 'v4-dev' into testing_nemo_spatialhash
9e91ee0

erikvansebille enabled auto-merge September 3, 2025 06:09

erikvansebille merged commit b185396 into v4-devSep 3, 2025
8 of 9 checks passed

erikvansebille deleted the testing_nemo_spatialhash branch September 3, 2025 06:13

github-project-automationBot moved this from Backlog to Done in Parcels development Sep 3, 2025