diff --git a/docs/source/python/dataset.rst b/docs/source/python/dataset.rst index ae14e396e16c..3d9983415cef 100644 --- a/docs/source/python/dataset.rst +++ b/docs/source/python/dataset.rst @@ -325,6 +325,24 @@ The currently available classes are :class:`~pyarrow.fs.S3FileSystem` and details. +Reading from Minio +------------------ + +In addition to cloud storage, pyarrow also supports reading from a +`MinIO https://github.com/minio/minio`_ object storage instance emulating S3 +APIs. Paired with `toxiproxy https://github.com/shopify/toxiproxy`_, this is +useful for testing or benchmarking. + +.. code-block:: python + + from pyarrow import fs + + # By default, MinIO will listen for unencrypted HTTP traffic. + minio = fs.S3FileSystem(scheme="http", endpoint="localhost:9000") + dataset = ds.dataset("ursa-labs-taxi-data/", filesystem=minio, + partitioning=["year", "month"]) + + Manual specification of the Dataset -----------------------------------