[CI/Python] Have automatized tests for S3

[asfimport]

Currently We don't run S3 integration tests regularly.

Possible solutions:

• mock it within python/pytest

• simply run the s3 tests with an S3 credential provided

• create a hdfs-integration like docker-compose setup and run an S3 mock server (e.g.: https://github.com/adobe/S3Mock, https://github.com/jubos/fake-s3, https://github.com/gaul/s3proxy, https://github.com/jserver/mock-s3)

  For more see discussion ARROW-2038: [Python] Strip s3:// scheme in S3FSWrapper isdir() and isfile() #3286

Reporter: Krisztian Szucs / @kszucs
Assignee: Rok Mihevc / @rok
Watchers: Rok Mihevc / @rok

Related issues:

• [Python/C++] Implement NativeFile interfaces for Amazon S3 (Is contained by)
• [C++] Add filesystem implementation for Amazon S3 (relates to)

PRs and other links:

• GitHub Pull Request #5200

_{Note: This issue was originally created as ARROW-4208. Please see the migration documentation for further details.}

[asfimport]

Daniel Rodriguez:
Another option here would be to use minio (an S3 API compatible server). I have use is successfully to test s3contents.

[asfimport]

Rok Mihevc / @rok:
With #5200 we get minio server in pytest via fixture. It will work run in docker images, travis and appveyor.
So far we have one test in python.

Do we want to add other tests now? Do we have regression examples?

[asfimport]

Krisztian Szucs / @kszucs:
Resolved by #5423