Skip to content

ARROW-1456: [Python] Run s3fs unit tests in Travis CI - #5200

Closed
rok wants to merge 1 commit into
apache:masterfrom
rok:ARROW-1456-s3-tests
Closed

ARROW-1456: [Python] Run s3fs unit tests in Travis CI#5200
rok wants to merge 1 commit into
apache:masterfrom
rok:ARROW-1456-s3-tests

Conversation

@rok

@rok rok commented Aug 26, 2019

Copy link
Copy Markdown
Member

This is to enable Python S3 tests in CI with minio as requested in ARROW-1456 and ARROW-4208.

@rok
rok force-pushed the ARROW-1456-s3-tests branch from d7cf0a9 to 5a1b9be Compare August 26, 2019 22:12
@rok rok changed the title ARROW-1456: [Python] Run s3fs unit tests in Travis CI [WIP] ARROW-1456: [Python] Run s3fs unit tests in Travis CI Aug 26, 2019
@rok
rok force-pushed the ARROW-1456-s3-tests branch from 5a1b9be to 0e1ee34 Compare August 26, 2019 22:28
@kszucs

kszucs commented Aug 29, 2019

Copy link
Copy Markdown
Member

@ursabot build

@rok
rok force-pushed the ARROW-1456-s3-tests branch 5 times, most recently from 67858d0 to f7f2805 Compare September 3, 2019 16:50
@rok

rok commented Sep 3, 2019

Copy link
Copy Markdown
Member Author

@ursabot build

@rok

rok commented Sep 3, 2019

Copy link
Copy Markdown
Member Author

@kszucs, I'd like to add minio binary and python s3fs package to the docker image. This would be to enable S3 CI tests. Do I need to do something besides this PR?

@rok
rok force-pushed the ARROW-1456-s3-tests branch from f7f2805 to 6a70df1 Compare September 3, 2019 18:23
@rok
rok force-pushed the ARROW-1456-s3-tests branch 16 times, most recently from 172c2b0 to b7fbbeb Compare September 14, 2019 12:29
@rok rok changed the title [WIP] ARROW-1456: [Python] Run s3fs unit tests in Travis CI ARROW-1456: [Python] Run s3fs unit tests in Travis CI Sep 14, 2019
@rok
rok force-pushed the ARROW-1456-s3-tests branch from b7fbbeb to a0c441e Compare September 14, 2019 12:42
@rok
rok force-pushed the ARROW-1456-s3-tests branch from a0c441e to 59f1e14 Compare September 14, 2019 16:29
@codecov-io

codecov-io commented Sep 14, 2019

Copy link
Copy Markdown

Codecov Report

Merging #5200 into master will increase coverage by 4.32%.
The diff coverage is 94.44%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master    #5200       +/-   ##
===========================================
+ Coverage   84.33%   88.65%    +4.32%     
===========================================
  Files         160      958      +798     
  Lines       10495   127472   +116977     
  Branches     1498     1498               
===========================================
+ Hits         8851   113016   +104165     
- Misses       1633    14091    +12458     
- Partials       11      365      +354
Impacted Files Coverage Δ
python/pyarrow/tests/test_parquet.py 97.13% <100%> (ø)
python/pyarrow/tests/conftest.py 70.83% <66.66%> (ø)
python/pyarrow/ipc.pxi 80.87% <0%> (ø)
cpp/src/parquet/column_page.h 100% <0%> (ø)
cpp/src/plasma/test/external_store_tests.cc 100% <0%> (ø)
cpp/src/arrow/array/builder_decimal.cc 86.95% <0%> (ø)
cpp/src/plasma/client.cc 96.5% <0%> (ø)
cpp/src/arrow/compute/compute_test.cc 100% <0%> (ø)
cpp/src/arrow/python/io.cc 91.3% <0%> (ø)
python/pyarrow/hdfs.py 34.24% <0%> (ø)
... and 790 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fec7143...f0c1e33. Read the comment docs.

Enabling Python S3 tests in CI with minio.
@rok
rok force-pushed the ARROW-1456-s3-tests branch from 59f1e14 to f0c1e33 Compare September 22, 2019 18:40
@rok

rok commented Sep 24, 2019

Copy link
Copy Markdown
Member Author

ARROW-6655 seems to do this and more. @kszucs shall I close this PR once ARROW-6655 is merged?

@kszucs

kszucs commented Sep 24, 2019

Copy link
Copy Markdown
Member

@rok yes, we should close it in favor of ARROW-6655.

It introduces Minio for the filesystem bindings, and I've also updated the s3fs accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants