Apply function to points within circular neighborhood by ahijevyc · Pull Request #941 · UXARRAY/uxarray

ahijevyc · 2024-09-09T16:44:16Z

Apply a neighborhood filter within a circular radius r to a UxDataset or UxDataArray.

Closes#930

Overview

This is kind of like uxarray.UxDataArray.inverse_distance_weighted_remap , but the neighborhood is defined by distance, not a number of nearest neighbors. This is ideally suited for a variable resolution mesh, in which a constant of neighbors doesn't have a constant sized neighborhood. Another difference is that this neighborhood filter does not weight data by inverse distance.

Just like uxarray.UxDataArray.subset.bounding_circle this function uses ball_tree.query_radius to select grid elements in a circular neighborhood, but this function finds the neighborhood for all elements in grid, not just one center_coordinate.

The filter function func may be a user-defined function, but uses np.mean by default. It could be min, max, np.median. It can even use functions that require additional arguments, like np.percentile if you supply the argument(s) with functools.partial (see below)

Expected Usage

fromfunctoolsimportpartialimportnumpyasnpimportuxarraygrid_path="/glade/campaign/mmm/wmr/weiwang/cps/irma3/2020/tk707_conus/init.nc"data_path="/glade/campaign/mmm/wmr/weiwang/cps/irma3/mp6/tk707/diag.2017-09-07_09.00.00.nc"uxds=uxarray.open_mfdataset(
grid_path,
data_path
)
# Trim domainlon_bounds= (-74, -64)
lat_bounds= (18, 24)
uxda=uxds["refl10cm_max"].isel(Time=0).subset.bounding_box(lon_bounds, lat_bounds)
# this is how you use this function to smooth with 0.25-deg filter.uxda_mean=uxda.neighborhood_filter(func=np.mean, r=0.25)
# this is another way to use this function with np.percentileuxda_max=uxda.neighborhood_filter(func=partial(np.percentile, q=90), r=0.25)
(uxda.plot.rasterize() +uxda_mean.plot.rasterize() +uxda_max.plot.rasterize()).cols(1)

PR Checklist

General

An issue is linked created and linked
Add appropriate labels
Filled out Overview and Expected Usage (if applicable) sections

Testing

Adequate tests are created if there is new functionality
Tests cover all possible logical paths in your function
Tests are not too basic (such as simply calling a function and nothing else)

Documentation

Docstrings have been added to all new functions
Docstrings have updated with any function changes
Internal functions have a preceding underscore (_); _neighborhood_filter is internal to uxarray/grid/neighbors.py
User functions added to docs/api.rst (the split user/internal api files no longer exist)

Examples

Any new notebook examples added to docs/examples/ folder
Clear the output of all cells before committing
New notebook files added to docs/examples.rst toctree
New notebook files added to new entry in docs/gallery.yml with appropriate thumbnail photo in docs/_static/thumbnails/

philipc2

A few initial comments:

Kept neighborhood and dual additions

philipc2 · 2025-03-07T22:23:17Z

HI @ahijevyc

Apologies for not getting to this PR earlier.

Looking at the implementation here, it looks great. It does however bring to light a possible need for us to consider a better, more streamlined, approach to handling these types of groupings and then applying some function on the result.

I mention this because of our Topological Aggregations. For this family of functions, we have distinct methods (i.e. topological_mean()), which looking back at doesn't seem like the preferred approach, especially if we plan to implement groupings like the neighborhood one and perhaps other spatial ones.

Very generally speaking, these functions essentially:

Group unstructured grid elements based on some condition/algorithm. Here we use the KD/BallTree to determine the candidate elements, while in the topological aggregations we use the connectivity information
Apply some function to the grouping (i.e. mean())
Store the results back on the unstructured grid element (node, edge, or face)

I wonder if this would be a good opportunity to extend the inherited .groupby() method from Xarray to support these spatial groupings.

I'm not sure of calling these approaches "kernels" is appropriate, but for the sake of this example, we could provide spatial kernels the user could pass into groupby() and then perform aggregations directly on the result. This feels much more in line with Xarray's design philosophy.

# radial neighborhood of r=0.25uxds['t2m'].groupby(kernel=ux.BoundingCircle(r=0.25)).mean()
# 2 deg by 2 deg bounding box uxds['t2m'].groupby(kernel=ux.BoundingBox(dlon=2, dlat=2))
# group the nodes that surround each face and find the maximumuxds['node_centered_var'].groupby(kernel=ux.FaceNode()).max()
# this is equivalent to the following in the current releaseuxds['node_centered_var'].topological_max(destination='face')

I'll ping @aaronzedwick and @erogluorhan for their thoughts on this. I personally really like the design above and think that it aligns well with the overall design.

aaronzedwick · 2025-03-10T16:34:00Z

HI @ahijevyc
Apologies for not getting to this PR earlier.
Looking at the implementation here, it looks great. It does however bring to light a possible need for us to consider a better, more streamlined, approach to handling these types of groupings and then applying some function on the result.
I mention this because of our Topological Aggregations. For this family of functions, we have distinct methods (i.e. topological_mean()), which looking back at doesn't seem like the preferred approach, especially if we plan to implement groupings like the neighborhood one and perhaps other spatial ones.
Very generally speaking, these functions essentially:
Group unstructured grid elements based on some condition/algorithm. Here we use the KD/BallTree to determine the candidate elements, while in the topological aggregations we use the connectivity information
Apply some function to the grouping (i.e. mean())
Store the results back on the unstructured grid element (node, edge, or face)
I wonder if this would be a good opportunity to extend the inherited .groupby() method from Xarray to support these spatial groupings.
I'm not sure of calling these approaches "kernels" is appropriate, but for the sake of this example, we could provide spatial kernels the user could pass into groupby() and then perform aggregations directly on the result. This feels much more in line with Xarray's design philosophy.
# radial neighborhood of r=0.25uxds['t2m'].groupby(kernel=ux.BoundingCircle(r=0.25)).mean()
# 2 deg by 2 deg bounding box uxds['t2m'].groupby(kernel=ux.BoundingBox(dlon=2, dlat=2))
# group the nodes that surround each face and find the maximumuxds['node_centered_var'].groupby(kernel=ux.FaceNode()).max()
# this is equivalent to the following in the current releaseuxds['node_centered_var'].topological_max(destination='face')
I'll ping @aaronzedwick and @erogluorhan for their thoughts on this. I personally really like the design above and think that it aligns well with the overall design.

That is interesting. You suggesting changing the way we do aggregations entirely? Then this would affect the reduction PR I am working on then. Perhaps this PR could implement that change if you wish. I am fine with this, if you want to, it sounds like it would be intuitive.

philipc2 · 2025-03-10T17:53:25Z

You suggesting changing the way we do aggregations entirely?

We should be changing the API for it, since the current one is not sustainable if we plan to implement many times of spatial grouping methods.

The underlying functionality would remain mostly unchanged.

Perhaps this PR could implement that change if you wish.

If we do decide to do this, I'll open up a separate PR starting with the topological aggregations.

aaronzedwick · 2025-03-10T20:03:00Z

You suggesting changing the way we do aggregations entirely?
We should be changing the API for it, since the current one is not sustainable if we plan to implement many times of spatial grouping methods.
The underlying functionality would remain mostly unchanged.
Perhaps this PR could implement that change if you wish.
If we do decide to do this, I'll open up a separate PR starting with the topological aggregations.

So would the reductions PR be obsolete?

philipc2 · 2025-03-10T20:05:53Z

You suggesting changing the way we do aggregations entirely?
We should be changing the API for it, since the current one is not sustainable if we plan to implement many times of spatial grouping methods.
The underlying functionality would remain mostly unchanged.
Perhaps this PR could implement that change if you wish.
If we do decide to do this, I'll open up a separate PR starting with the topological aggregations.
So would the reductions PR be obsolete?

No. The underlying implementation would remain the same, since we would still need those implemented.

This would just provide a different interface for it, with a more "Xarray-like" interface.

aaronzedwick · 2025-03-10T20:15:35Z

You suggesting changing the way we do aggregations entirely?
We should be changing the API for it, since the current one is not sustainable if we plan to implement many times of spatial grouping methods.
The underlying functionality would remain mostly unchanged.
Perhaps this PR could implement that change if you wish.
If we do decide to do this, I'll open up a separate PR starting with the topological aggregations.
So would the reductions PR be obsolete?
No. The underlying implementation would remain the same, since we would still need those implemented.
This would just provide a different interface for it, with a more "Xarray-like" interface.

Ah, okay, I see. That makes sense, thanks for the clarification!

erogluorhan

Overall this looks great, but I have a couple concerns in the low-level implementation for which @cmdupuis3 could be helpful as well (added as a reviewer). See below

erogluorhan · 2026-08-04T19:54:46Z

+ func: Callable = np.mean,
+ r: float = 1.0,
+ ) -> UxDataArray:
+ """Apply a neighborhood filter, replacing the value at each grid


Might use a little rephrasing: "replace" might be misleading here since function returns a new data array and actually doesn't change the original data?

erogluorhan · 2026-08-04T19:58:56Z

+
+ # Apply func along the last (grid) axis only, so any extra leading
+ # dimensions (e.g. time) are preserved rather than being collapsed.
+ for i, idx in enumerate(neighbor_indices):


This will iterate through every single geometric element, e.g. faces, right? It looks very costly for km-scale data. Was there any consideration of vectorization for this?
cc: @cmdupuis3 for your thoughts

Something you could do is make a boolean mask out of neighbor_indices as keys, then mask it to neighbor_masked, then you could delete the whole conditional/try/except block and iterate over only the valid entries. If you really need the full-scale destination_data, you can use the mask to reshape the output.

erogluorhan · 2026-08-04T20:08:03Z

+ for i, idx in enumerate(neighbor_indices):
+ if len(idx):
+ try:
+ destination_data[..., i] = func(data[..., idx], axis=-1)


From what I can tell on this line, every single iteration will trigger rebuilding and executing the dask task graph from scratch because this line returns a lazy dask scalar when data is dask-backed and assigning that into a numpy array index.
I believe this could be avoided if uxdataarray.py/neighborhood_filter did an upfront .compute() for data (and actually that function already claims "lazy (dask-backed) data is computed eagerly and the result is always NumPy-backed").
cc: @cmdupuis3 you dealt with bundled .compute()s through xarray API in #1560, it might not directly related to here, but you may still have some thoughts.

Looks like it. Just at a glance, this block seems like it's basically a ufunc implementation.

cmdupuis3 · 2026-08-05T17:19:16Z

I'm working on a chunk-friendly optimized version based on gufuncs, I'll add my own PR off this one and y'all can merge it. The one caveat is that we'd have to hand-enumerate the specific reductions we want to be able to run optimally, I think otherwise we can offer a fallback option but the performance would suffer. I don't think that would happen that often in practice though.

rajeeja · 2026-08-05T18:15:36Z

I'm working on a chunk-friendly optimized version based on gufuncs, I'll add my own PR off this one and y'all can merge it. The one caveat is that we'd have to hand-enumerate the specific reductions we want to be able to run optimally, I think otherwise we can offer a fallback option but the performance would suffer. I don't think that would happen that often in practice though.

Makes sense, thanks for finding this out, that code would lose out-of-core/parallel benefits; most of the times this would be one of the steps in the bigger pipeline and if others are nice dask ops that would be a good win. I'll keep an eye on the followup to this.

Follows the gufunc work with the API changes it made possible. Kernel factory. The buffered kernels (median and friends) share one gather loop, with the reduction supplied as a numba-compilable callable. numba supports the NumPy reductions in nopython mode, so np.median is used directly rather than reimplemented -- it partitions instead of fully sorting, making it 1.6x faster than the hand-written sort it replaces, as well as shorter. Adds ptp, std, var, quantile and percentile; the parameterized ones carry their argument as a scalar core dimension so q and ddof vary per call without recompiling. Named reductions. `func` now takes a name ("mean", "quantile", ...) with parameters as ordinary keyword arguments. The previous `axis=-1` contract could not be jitted -- numba rejects the axis kwarg on mean/median/percentile -- and dispatching on a function object cannot see through functools.partial, so `partial(np.percentile, q=90)`, the example in our own docstring, could never reach a kernel. A name always can. Callables remain accepted on the old contract as an escape hatch, and np.mean and friends still map to their kernels so existing code keeps the fast path. Neighborhoods. Finding the neighbors costs more than reducing over them -- after the kernel work it is ~95% of a call -- and it was repeated on every call, including once per variable in the dataset path. Grid.neighborhoods() does the search once and returns an object to reduce over repeatedly: 3.5x for four reductions at one radius on a 196k-face grid, and a dataset filter now costs one query per grid location rather than one per variable (8 variables: 1.74s -> 0.23s). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

cmdupuis3 · 2026-08-05T23:57:47Z

@ahijevyc @rajeeja I have a draft PR sitting in the fork, I'll keep working on it there. I'm not totally satisfied with the level of complexity. Using a functional approach inside vectorized gufuncs makes it hard to get performance and good readability at once.

- numba guvectorize kernels for compiled parallel reductions - Neighborhoods object: one BallTree query reused across variables/reductions - Named reduction API: func='mean', func='percentile', q=90, etc.

cmdupuis3 · 2026-08-06T15:18:05Z

pre-commit.ci autofix

for more information, see https://pre-commit.ci

cmdupuis3 · 2026-08-13T22:54:26Z

I have some more changes to consider on my cmd/941 branch. The gist is that I wanted to get away from passing numpy functions to the neighborhood filter, because in order to get the vectorized kernels working, you'd basically have to have a dictionary of numpy functions to vectorized kernels, and the API would be sort of a lie.

Instead, my API proposal is that we have all the named kernels be methods. So, we can call the vectorized kernels by name without mystifying what's actually running, and have nb.reduce(func) be the catch-all for external kernels.

This has the added advantage that the kernels are now separable from the neighborhood construction, so you can store a neighborhood and call multiple kernels on it rather than constructing a new neighborhood each time.

On the other hand, it raises the possibility of having multiple neighborhoodsesesssses, so I renamed them to be singular as objects.

rajeeja · 2026-08-13T23:40:58Z

I have some more changes to consider on my cmd/941 branch. The gist is that I wanted to get away from passing numpy functions to the neighborhood filter, because in order to get the vectorized kernels working, you'd basically have to have a dictionary of numpy functions to vectorized kernels, and the API would be sort of a lie.
Instead, my API proposal is that we have all the named kernels be methods. So, we can call the vectorized kernels by name without mystifying what's actually running, and have nb.reduce(func) be the catch-all for external kernels.
This has the added advantage that the kernels are now separable from the neighborhood construction, so you can store a neighborhood and call multiple kernels on it rather than constructing a new neighborhood each time.
On the other hand, it raises the possibility of having multiple neighborhoodsesesssses, so I renamed them to be singular as objects.

I like this design, it is simpler , less duplication and more pythonic. The whole _filter wasn't really needed. One question - Do we really need _BoundNeighborhoodReductions as an ABC, or can the common reduction logic be expressed through a simpler composition/delegation pattern?

cmdupuis3 · 2026-08-13T23:57:34Z

I kind of think there should be a way to unify all three classes somehow, but I haven't found it yet. I can try some more things and let you know.

cmdupuis3 · 2026-08-14T00:44:13Z

@rajeeja Alright, I refactored it a bit and got rid of the ABC (although spiritually it still basically is one). I attempted taking a compositional approach, but there's no nice solution that doesn't clutter up the API or duplicate all the reduction methods, or have some other drawbacks.

github-actions · 2026-08-18T17:57:00Z

ASV Benchmarking

Benchmark Comparison Results

Benchmarks that have improved:

Change	Before [`fda71bb`]	After [`4ca774e`]	Ratio	Benchmark (Parameter)
-	163±10ms	138±0.3ms	0.85	bench_connectivity.Connectivity.time_node_edge('120km')
-	2.00±0.03ms	1.77±0.01ms	0.89	geometry_samebody.SameBodyConstLat.time_accux_dispatch

Benchmarks that have stayed the same:

Before [`fda71bb`]	After [`4ca774e`]	Ratio	Benchmark (Parameter)
149±4ms	145±5ms	0.97	bench_connectivity.Connectivity.time_edge_face('120km')
8.15±0.04ms	8.25±0.03ms	1.01	bench_connectivity.Connectivity.time_edge_face('480km')
167±30ms	138±1ms	~0.83	bench_connectivity.Connectivity.time_edge_node('120km')
7.57±0.02ms	7.49±0.06ms	0.99	bench_connectivity.Connectivity.time_edge_node('480km')
138±0.7ms	145±7ms	1.05	bench_connectivity.Connectivity.time_face_edge('120km')
8.11±0.5ms	8.95±0.7ms	~1.10	bench_connectivity.Connectivity.time_face_edge('480km')
574±10ms	560±2ms	0.98	bench_connectivity.Connectivity.time_face_face('120km')
34.8±0.08ms	37.7±2ms	1.08	bench_connectivity.Connectivity.time_face_face('480km')
44.8±1μs	44.1±0.9μs	0.98	bench_connectivity.Connectivity.time_face_node('120km')
47.4±4μs	44.0±1μs	0.93	bench_connectivity.Connectivity.time_face_node('480km')
310±6μs	310±7μs	1.00	bench_connectivity.Connectivity.time_n_nodes_per_face('120km')
228±10μs	228±3μs	1.00	bench_connectivity.Connectivity.time_n_nodes_per_face('480km')
7.74±0.07ms	7.76±0.02ms	1.00	bench_connectivity.Connectivity.time_node_edge('480km')
43.4±0.2ms	44.6±0.5ms	1.03	bench_connectivity.Connectivity.time_node_face('120km')
3.20±0.1ms	2.91±0.02ms	0.91	bench_connectivity.Connectivity.time_node_face('480km')
5.88±0.02ms	5.88±0.05ms	1.00	face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
1.75±0.01ms	1.75±0.02ms	1.00	face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
6.28±6s	8.35±9ms	~0.00	face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
1.35±0.05ms	1.26±0.02ms	0.94	face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
57.3k	57.3k	1.00	face_bounds.FaceBounds.track_nbytes_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
12.3k	12.3k	1.00	face_bounds.FaceBounds.track_nbytes_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
123k	123k	1.00	face_bounds.FaceBounds.track_nbytes_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
128	128	1.00	face_bounds.FaceBounds.track_nbytes_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
1.27M	1.27M	1.00	face_bounds.FaceBounds.track_nbytes_grid_with_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
50.1k	50.1k	1.00	face_bounds.FaceBounds.track_nbytes_grid_with_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
1.48M	1.48M	1.00	face_bounds.FaceBounds.track_nbytes_grid_with_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
712	712	1.00	face_bounds.FaceBounds.track_nbytes_grid_with_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
1.95M	1.93M	0.99	face_bounds.FaceBounds.track_peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
1.95M	1.94M	0.99	face_bounds.FaceBounds.track_peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
2.12M	2.1M	0.99	face_bounds.FaceBounds.track_peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
38.3k	38.3k	1.00	face_bounds.FaceBounds.track_peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
777±30ns	785±30ns	1.01	geometry_kernels.AccucrossKernels.time_accucross
1.72±0.03μs	1.80±0.06μs	1.05	geometry_kernels.AccucrossKernels.time_accucross_pair
296±30ns	285±20ns	0.96	geometry_kernels.EFTPrimitives.time_acc_sqrt_re
253±9ns	264±20ns	1.05	geometry_kernels.EFTPrimitives.time_diff_of_products
242±10ns	296±20ns	~1.23	geometry_kernels.EFTPrimitives.time_two_prod
256±30ns	248±6ns	0.97	geometry_kernels.EFTPrimitives.time_two_sum
1.08±0.02μs	1.01±0.04μs	0.94	geometry_kernels.GCAConstLatIntersection.time_accux_constlat_kernel
750±30ns	747±20ns	1.00	geometry_kernels.GCAConstLatIntersection.time_gca_const_lat_intersection
1.34±0.03μs	1.41±0.1μs	1.05	geometry_kernels.GCAConstLatIntersection.time_try_gca_const_lat_intersection
1.15±0.07μs	1.07±0.04μs	0.93	geometry_kernels.GCAGCAIntersection.time_accux_gca_kernel
998±10ns	1.07±0.03μs	1.07	geometry_kernels.GCAGCAIntersection.time_gca_gca_intersection
1.53±0.05μs	1.53±0.07μs	1.00	geometry_kernels.GCAGCAIntersection.time_try_gca_gca_intersection
36.5±0.9μs	36.5±0.5μs	1.00	geometry_kernels.OrientPredicates.time_on_minor_arc
777±40ns	839±60ns	1.08	geometry_kernels.OrientPredicates.time_orient3d_on_sphere
578±9μs	563±7μs	0.97	geometry_samebody.SameBodyConstLat.time_accux_kernel
1.19±0.01ms	1.20±0.02ms	1.01	geometry_samebody.SameBodyConstLat.time_fp64_dispatch
164±40μs	118±20μs	~0.72	geometry_samebody.SameBodyConstLat.time_fp64_kernel
29.9±1ms	27.4±0.2ms	0.92	geometry_samebody_gcagca.SameBodyGcaGca.time_accux_dispatch
6.37±0.02ms	6.52±0.09ms	1.02	geometry_samebody_gcagca.SameBodyGcaGca.time_accux_kernel
23.9±0.1ms	25.4±0.7ms	1.06	geometry_samebody_gcagca.SameBodyGcaGca.time_fp64_dispatch
4.05±0.2ms	3.81±0.04ms	0.94	geometry_samebody_gcagca.SameBodyGcaGca.time_fp64_kernel
594±40ms	7.11±0.2s	~11.96	import.Imports.timeraw_import_uxarray
2.12±0.02ms	2.18±0.01ms	1.03	mpas_ocean.CheckNorm.time_check_norm('120km')
1.45±0.01ms	1.49±0.1ms	1.02	mpas_ocean.CheckNorm.time_check_norm('480km')
507±0.8ms	507±2ms	1.00	mpas_ocean.ConnectivityConstruction.time_face_face_connectivity('120km')
32.4±0.3ms	35.0±3ms	1.08	mpas_ocean.ConnectivityConstruction.time_face_face_connectivity('480km')
469±20μs	468±10μs	1.00	mpas_ocean.ConnectivityConstruction.time_n_nodes_per_face('120km')
378±1μs	399±10μs	1.06	mpas_ocean.ConnectivityConstruction.time_n_nodes_per_face('480km')
3.25±0.04ms	3.26±0.02ms	1.00	mpas_ocean.ConstructFaceLatLon.time_cartesian_averaging('120km')
2.55±0.01ms	2.54±0.01ms	1.00	mpas_ocean.ConstructFaceLatLon.time_cartesian_averaging('480km')
2.35±0.1s	2.36±0s	1.00	mpas_ocean.ConstructFaceLatLon.time_welzl('120km')
149±0.9ms	148±0.5ms	0.99	mpas_ocean.ConstructFaceLatLon.time_welzl('480km')
9.09±0.02ms	9.11±0.01ms	1.00	mpas_ocean.ConstructTreeStructures.time_ball_tree('120km')
642±30μs	613±10μs	0.95	mpas_ocean.ConstructTreeStructures.time_ball_tree('480km')
6.81±0.02ms	6.80±0.01ms	1.00	mpas_ocean.ConstructTreeStructures.time_kd_tree('120km')
480±5μs	474±10μs	0.99	mpas_ocean.ConstructTreeStructures.time_kd_tree('480km')
378±1ms	385±2ms	1.02	mpas_ocean.CrossSections.time_const_lat('120km', 1)
192±2ms	193±2ms	1.01	mpas_ocean.CrossSections.time_const_lat('120km', 2)
98.2±0.2ms	98.1±0.7ms	1.00	mpas_ocean.CrossSections.time_const_lat('120km', 4)
335±3ms	339±3ms	1.01	mpas_ocean.CrossSections.time_const_lat('480km', 1)
168±0.4ms	170±1ms	1.01	mpas_ocean.CrossSections.time_const_lat('480km', 2)
91.3±5ms	86.9±0.4ms	0.95	mpas_ocean.CrossSections.time_const_lat('480km', 4)
18.3±0.02ms	18.3±0.08ms	1.00	mpas_ocean.DualMesh.time_dual_mesh_construction('120km')
2.02±0.07ms	1.99±0.09ms	0.99	mpas_ocean.DualMesh.time_dual_mesh_construction('480km')
59.4±1ms	58.7±0.4ms	0.99	mpas_ocean.FaceAreas.time_face_areas('120km')
5.88±0.1ms	5.82±0.07ms	0.99	mpas_ocean.FaceAreas.time_face_areas('480km')
229k	229k	1.00	mpas_ocean.FaceAreas.track_nbytes_face_areas('120km')
14.3k	14.3k	1.00	mpas_ocean.FaceAreas.track_nbytes_face_areas('480km')
2.12M	2.12M	1.00	mpas_ocean.FaceAreas.track_peakmem_face_areas('120km')
804k	798k	0.99	mpas_ocean.FaceAreas.track_peakmem_face_areas('480km')
573±10ms	570±3ms	1.00	mpas_ocean.GeoDataFrame.time_to_geodataframe('120km', False)
33.8±0.3ms	34.3±0.5ms	1.02	mpas_ocean.GeoDataFrame.time_to_geodataframe('120km', True)
49.0±0.3ms	51.3±0.2ms	1.05	mpas_ocean.GeoDataFrame.time_to_geodataframe('480km', False)
3.72±0.09ms	3.69±0.07ms	0.99	mpas_ocean.GeoDataFrame.time_to_geodataframe('480km', True)
176±10ms	166±0.6ms	0.94	mpas_ocean.Gradient.time_gradient('120km')
11.2±0.07ms	11.1±0.03ms	1.00	mpas_ocean.Gradient.time_gradient('480km')
457k	457k	1.00	mpas_ocean.Gradient.track_nbytes_gradient('120km')
28.7k	28.7k	1.00	mpas_ocean.Gradient.track_nbytes_gradient('480km')
5.08M	5.08M	1.00	mpas_ocean.Gradient.track_peakmem_gradient('120km')
328k	328k	1.00	mpas_ocean.Gradient.track_peakmem_gradient('480km')
303±20μs	300±7μs	0.99	mpas_ocean.HoleEdgeIndices.time_construct_hole_edge_indices('120km')
152±7μs	140±5μs	0.92	mpas_ocean.HoleEdgeIndices.time_construct_hole_edge_indices('480km')
409±10μs	415±9μs	1.02	mpas_ocean.Integrate.time_integrate('120km')
370±9μs	380±10μs	1.03	mpas_ocean.Integrate.time_integrate('480km')
18.4M	18.4M	1.00	mpas_ocean.Integrate.track_nbytes_integrate('120km')
1.2M	1.2M	1.00	mpas_ocean.Integrate.track_nbytes_integrate('480km')
115±7ms	108±2ms	0.94	mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'exclude')
108±2ms	108±1ms	1.00	mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'include')
114±7ms	107±1ms	0.94	mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'split')
8.47±0.07ms	8.48±0.07ms	1.00	mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'exclude')
8.55±0.1ms	8.42±0.1ms	0.98	mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'include')
8.45±0.06ms	8.27±0.08ms	0.98	mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'split')
266±8μs	277±9μs	1.04	mpas_ocean.PointInPolygon.time_face_search_lonlat('120km')
243±10μs	227±4μs	0.94	mpas_ocean.PointInPolygon.time_face_search_lonlat('480km')
227±2μs	229±1μs	1.01	mpas_ocean.PointInPolygon.time_face_search_xyz('120km')
202±8μs	212±6μs	1.05	mpas_ocean.PointInPolygon.time_face_search_xyz('480km')
139±0.06ms	138±0.3ms	1.00	mpas_ocean.RemapDownsample.time_bilinear_remapping
140±0.3ms	141±1ms	1.01	mpas_ocean.RemapDownsample.time_inverse_distance_weighted_remapping
10.4±0.06ms	10.5±0.1ms	1.01	mpas_ocean.RemapDownsample.time_nearest_neighbor_remapping
745±3ms	741±3ms	1.00	mpas_ocean.RemapUpsample.time_bilinear_remapping
25.4±1ms	23.9±0.6ms	0.94	mpas_ocean.RemapUpsample.time_inverse_distance_weighted_remapping
8.95±0.2ms	8.89±0.2ms	0.99	mpas_ocean.RemapUpsample.time_nearest_neighbor_remapping
5.37±0.02ms	5.47±0.09ms	1.02	mpas_ocean.ZonalAverage.time_zonal_average('120km')
2.75±0.02ms	2.72±0.02ms	0.99	mpas_ocean.ZonalAverage.time_zonal_average('480km')
4.69±0.02ms	4.89±0.3ms	1.04	quad_hexagon.QuadHexagon.time_open_dataset
3.93±0.02ms	3.90±0.02ms	0.99	quad_hexagon.QuadHexagon.time_open_grid
408	408	1.00	quad_hexagon.QuadHexagon.track_nbytes_open_dataset
392	392	1.00	quad_hexagon.QuadHexagon.track_nbytes_open_grid
73.5k	73.1k	1.00	quad_hexagon.QuadHexagon.track_peakmem_open_dataset
72.8k	72.2k	0.99	quad_hexagon.QuadHexagon.track_peakmem_open_grid

Benchmarks that have got worse:

Change	Before [`fda71bb`]	After [`4ca774e`]	Ratio	Benchmark (Parameter)
+	336M	404M	1.2	face_bounds.FaceBoundsColdStartRss.peakmem_open_and_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
+	365M	434M	1.19	face_bounds.FaceBoundsColdStartRss.peakmem_open_and_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
+	337M	405M	1.2	face_bounds.FaceBoundsColdStartRss.peakmem_open_and_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
+	337M	405M	1.2	face_bounds.FaceBoundsColdStartRss.peakmem_open_and_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
+	293M	362M	1.23	import.Imports.track_peakmem_import_uxarray
+	356M	424M	1.19	mpas_ocean.CrossSectionsPeakMem.peakmem_const_lat('120km', 1)
+	356M	426M	1.2	mpas_ocean.CrossSectionsPeakMem.peakmem_const_lat('120km', 2)
+	356M	424M	1.19	mpas_ocean.CrossSectionsPeakMem.peakmem_const_lat('120km', 4)
+	339M	408M	1.2	mpas_ocean.CrossSectionsPeakMem.peakmem_const_lat('480km', 1)
+	339M	407M	1.2	mpas_ocean.CrossSectionsPeakMem.peakmem_const_lat('480km', 2)
+	338M	407M	1.2	mpas_ocean.CrossSectionsPeakMem.peakmem_const_lat('480km', 4)
+	352M	418M	1.19	mpas_ocean.GradientColdStartRss.peakmem_gradient('120km')
+	331M	398M	1.2	mpas_ocean.GradientColdStartRss.peakmem_gradient('480km')
+	358M	425M	1.19	mpas_ocean.ZonalAveragePeakMem.peakmem_zonal_average('120km')
+	341M	407M	1.2	mpas_ocean.ZonalAveragePeakMem.peakmem_zonal_average('480km')

ahijevyc added 2 commits September 7, 2024 11:09

Neighborhood filter
75d832e

Merge remote-tracking branch 'upstream/main' into neighborhood_filter
b850bc4

ahijevyc added the new feature New feature or request label Sep 9, 2024

ahijevyc self-assigned this Sep 9, 2024

ahijevyc mentioned this pull request Sep 9, 2024
Add neighborhood filter to remap #923
Closed
14 tasks

ahijevyc added 2 commits September 9, 2024 10:48

ruff recommendations
8ec0193

added Callable to Type checking
5605949

philipc2 linked an issue Sep 9, 2024 that may be closed by this pull request
Apply a neighborhood filter with radius r to all elements of UxDataArray #930
Open

philipc2 reviewed Sep 9, 2024
View reviewed changes

Comment threaduxarray/core/dataarray.py Outdated

Merge branch 'main' into ahijevyc/neighborhood_filter
70ba961

philipc2 previously requested changes Sep 9, 2024
View reviewed changes

Comment threaduxarray/core/dataarray.py Outdated
Comment threaduxarray/core/dataarray.py
Comment threaduxarray/core/dataset.py Outdated

ahijevycand others added 4 commits September 9, 2024 15:10

np.vstack().T faster than np.c
0c7bc1e

Fix some comments
d6d8a33

Merge branch 'main' into ahijevyc/neighborhood_filter
47b9cda

Merge branch 'main' into ahijevyc/neighborhood_filter
f75db0d

ahijevyc commented Oct 2, 2024
View reviewed changes

Comment threaduxarray/core/dataarray.py Outdated

Merge branch 'main' into ahijevyc/neighborhood_filter
bddd2fa
Kept neighborhood and dual additions

ahijevycand others added 4 commits March 17, 2025 14:12

Merge branch 'main' into ahijevyc/neighborhood_filter
a0b6361

missing imports
6c59af7

Merge branch 'main' into ahijevyc/neighborhood_filter
67d0c11

Merge branch 'main' into ahijevyc/neighborhood_filter
8979745

philipc2 mentioned this pull request May 14, 2025
Add Azimuthal Averaging #1255
Merged
9 tasks

ahijevyc added 2 commits May 28, 2025 11:47

Merge branch 'main' into ahijevyc/neighborhood_filter
a8875cd

Merge branch 'UXARRAY:main' into ahijevyc/neighborhood_filter
f4af498

erogluorhan removed the request for review from philipc2 August 4, 2026 19:37

erogluorhan reviewed Aug 4, 2026
View reviewed changes

erogluorhan requested a review from cmdupuis3 August 4, 2026 20:19

cmdupuis3 reviewed Aug 5, 2026
View reviewed changes

Comment threaduxarray/core/dataarray.py Outdated

cmdupuis3and others added 3 commits August 5, 2026 13:40

941: gufunc for neighborhood filter
b964260

941: some cleanup
c7d2711

rajeeja added 2 commits August 6, 2026 02:32

Merge main into ahijevyc/neighborhood_filter to resolve conflicts
c258e63

Merge Chris Dupuis gufunc/Neighborhoods work into neighborhood_filter
39479a1
- numba guvectorize kernels for compiled parallel reductions - Neighborhoods object: one BallTree query reused across variables/reductions - Named reduction API: func='mean', func='percentile', q=90, etc.

rajeeja assigned cmdupuis3 Aug 6, 2026

pre-commit-ciBotand others added 3 commits August 6, 2026 15:18

[pre-commit.ci] auto fixes from pre-commit.com hooks
c240cd3
for more information, see https://pre-commit.ci

Merge branch 'main' into ahijevyc/neighborhood_filter
728bf87

Merge branch 'main' into ahijevyc/neighborhood_filter
4e4fb76

rajeeja requested review from cmdupuis3, erogluorhan and philipc2 August 8, 2026 17:11

cmdupuis3 added 3 commits August 17, 2026 12:23

Merge branch 'main' into ahijevyc/neighborhood_filter
c72e863

Merge branch 'main' into ahijevyc/neighborhood_filter
dcf6ea4

Merge branch 'main' into ahijevyc/neighborhood_filter
c9b6435

erogluorhan added the run-benchmark Run ASV benchmark workflow label Aug 18, 2026

Merge branch 'main' into ahijevyc/neighborhood_filter
a2b0962

Uh oh!

Conversation

ahijevyc commented Sep 9, 2024• edited by rajeeja Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Expected Usage

PR Checklist

Uh oh!

Uh oh!

philipc2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

philipc2 commented Mar 7, 2025

Uh oh!

aaronzedwick commented Mar 10, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

philipc2 commented Mar 10, 2025

Uh oh!

aaronzedwick commented Mar 10, 2025

Uh oh!

philipc2 commented Mar 10, 2025

Uh oh!

aaronzedwick commented Mar 10, 2025

Uh oh!

erogluorhan left a comment

Choose a reason for hiding this comment

Uh oh!

erogluorhanAug 4, 2026

Choose a reason for hiding this comment

Uh oh!

erogluorhanAug 4, 2026

Choose a reason for hiding this comment

Uh oh!

cmdupuis3Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

erogluorhanAug 4, 2026

Choose a reason for hiding this comment

Uh oh!

cmdupuis3Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cmdupuis3 commented Aug 5, 2026

Uh oh!

rajeeja commented Aug 5, 2026

Uh oh!

cmdupuis3 commented Aug 5, 2026

Uh oh!

cmdupuis3 commented Aug 6, 2026

Uh oh!

cmdupuis3 commented Aug 13, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rajeeja commented Aug 13, 2026

Uh oh!

cmdupuis3 commented Aug 13, 2026

Uh oh!

cmdupuis3 commented Aug 14, 2026

Uh oh!

github-actionsBot commented Aug 18, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ASV Benchmarking

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ahijevyc commented Sep 9, 2024•
edited by rajeeja
Loading

aaronzedwick commented Mar 10, 2025•
edited
Loading

cmdupuis3 commented Aug 13, 2026•
edited
Loading

github-actionsBot commented Aug 18, 2026•
edited
Loading