…implementations (#45459) ### Rationale for this change The GEOMETRY and GEOGRAPHY logical types are being proposed as an addition to the Parquet format. ### What changes are included in this PR? This is a continuation of @ Kontinuation 's initial PR (#43977) implementing apache/parquet-format#240 , which included: - Added geometry logical types (printing, serialization, deserialization) - Added geometry column statistics (serialization, deserialization, writing) - Support reading/writing parquet files containing geometry columns Changes after this were: - Rebasing on the latest apache/arrow - Split geography/geometry types - Synchronize the final parameter names (e.g., no more "encoding", "edges" -> "algorithm") - Simplify geometry_util_internal.h and use Status instead of exceptions according to suggestions from the previous PR In order to write test files, I also: - Implemented conversion to/from the GeoArrow extension type - Wired the requisite options to pyarrow so that the files could be written from Python Those last two are probably a bit much for this particular PR, and I'm happy to move them. Some things that aren't in this PR (but should be in this one or a future PR): - Update the bounding box logic to implement the "wraparound" bounding boxes where `max > min` (and generally make sure the stats for geography are written for trivial cases) - Test more invalid WKB cases ### Are these changes tested? Yes! ### Are there any user-facing changes? Yes! Example from the included Python bindings: ```python import geopandas import geopandas.testing import geoarrow.pyarrow as _ # for extension type registration import pyarrow as pa from pyarrow import parquet # More example files at # https://github.com/geoarrow/geoarrow-data gdf = geopandas.read_file( "https://raw.githubusercontent.com/geoarrow/geoarrow-data/v0.2.0-rc6/example-crs/files/example-crs_vermont-utm.fgb" ) gdf.total_bounds #> array([ 625858.19400305, 4733644.25036889, 775539.58040423, #> 4989817.92403143]) gdf.crs.to_authority() #> ('EPSG', '32618') tab = pa.table(gdf.to_arrow()) # Use store_schema=False to explicitly check conversion to Parquet LogicalType # This example also works with store_schema=True (the default) and without # an explicit arrow_extensions_enabled=True on read. parquet.write_table(tab, "vermont.parquet", store_schema=False) f = parquet.ParquetFile("vermont.parquet", arrow_extensions_enabled=True) f.schema #> <pyarrow._parquet.ParquetSchema object at 0x1402e5940> #> required group field_id=-1 schema { #> optional binary field_id=-1 geometry (Geometry(crs={"type":"ProjectedCRS", ...})); #> } f.metadata.row_group(0).column(0).geo_statistics #> <pyarrow._parquet.GeoStatistics object at 0x127df3eb0> #> geospatial_types: [3] #> xmin: 625858.1940030524, xmax: 775539.5804042327 #> ymin: 4733644.250368893, ymax: 4989817.92403143 #> zmin: None, zmax: None #> mmin: None, mmax: None gdf2 = geopandas.GeoDataFrame.from_arrow(f.read()) gdf2.crs.to_authority() #> ('EPSG', '32618') geopandas.testing.assert_geodataframe_equal(gdf2, gdf) ``` * GitHub Issue: #45522 Lead-authored-by: Dewey Dunnington <dewey@wherobots.com> Co-authored-by: Dewey Dunnington <dewey@fishandwhistle.net> Co-authored-by: Kristin Cowalcijk <bo@wherobots.com> Co-authored-by: Gang Wu <ustcwg@gmail.com> Co-authored-by: Antoine Pitrou <pitrou@free.fr> Signed-off-by: Dewey Dunnington <dewey@wherobots.com>

Uh oh!

[Parquet][C++] Implement Geography and Geometry types in the C++ Parquet implementation #45522

Description

Describe the enhancement requested

Component(s)

Activity

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions