Describe the enhancement requested
This is a follow up issue from:
Which added the key-value options to FileSystemFromUri. This issue makes S3FileSystem actually consume the options.
This is required in order to split S3Filesystem out of libarrow and into its own dedicated module (see: #49399) because once s3 is on its own module users can no longer create their own S3Options and must use FileSystemFromUri.
We should expose, through the options channel, the S3Options settings that are not already expressible in the URI. The exception is access_key/secret_key, which can already be set in the URI (s3://access_key:secret_key@bucket); exposing them as options as well provides a path to set credentials without putting them in the URI (which is a security smell).
Component(s)
C++
Describe the enhancement requested
This is a follow up issue from:
Which added the key-value options to
FileSystemFromUri. This issue makesS3FileSystemactually consume the options.This is required in order to split S3Filesystem out of libarrow and into its own dedicated module (see: #49399) because once s3 is on its own module users can no longer create their own S3Options and must use
FileSystemFromUri.We should expose, through the options channel, the
S3Optionssettings that are not already expressible in the URI. The exception isaccess_key/secret_key, which can already be set in the URI (s3://access_key:secret_key@bucket); exposing them as options as well provides a path to set credentials without putting them in the URI (which is a security smell).Component(s)
C++