ARROW-5633: [Python] Enable bz2 in Linux wheels

[pitrou]

No description provided.

[pitrou]

@ursabot help

[ursabot]

No such command "help".

[pitrou]

@ursabot crossbow build wheel

[ursabot]

No such command "build".

[pitrou]

@ursabot crossbow wheel

[pitrou]

@ursabot build wheel

[ursabot]

Got unexpected extra argument (wheel)

[pitrou]

EEk!

[kszucs]

@ursabot crossbow --help

[ursabot]

Usage: @ursabot crossbow [OPTIONS] COMMAND [ARGS]...
Trigger crossbow builds for this pull request
Options:
--help Show this message and exit.
Commands:
package Submit crossbow packaging tasks.
test Submit crossbow testing tasks.

[kszucs]

@ursabot crossbow package --help

[pitrou]

@ursabot crossbow package wheel

[ursabot]

Usage: @ursabot crossbow package [OPTIONS] [TASK]...
Submit crossbow packaging tasks.
See groups defined in arrow/dev/tasks/tasks.yml
Options:
-g, --group [conda|wheel|linux|gandiva]
Submit task groups as defined in task.yml
--help Show this message and exit.

[ursabot]

AMD64 Conda Crossbow (#22369) builder failed.

Revision: 3c0e62eaa7232a8760193687b41eec901552292e

Crossbow: 'python crossbow.py ...' step's stderr:

Error: Invalid task(s) {'wheel'}. Must be one of {'ubuntu-xenial', 'conda-linux-gcc-py27', 'wheel-win-cp37m', 'ubuntu-bionic', 'debian-stretch-arm64', 'wheel-osx-cp35m', 'wheel-win-cp35m', 'centos-7', 'gandiva-jar-osx', 'conda-win-vs2015-py37', 'debian-stretch', 'conda-linux-gcc-py37', 'wheel-linux-cp37m', 'wheel-osx-cp36m', 'wheel-linux-cp36m', 'ubuntu-cosmic', 'conda-osx-clang-py36', 'conda-osx-clang-py37', 'wheel-linux-cp27m', 'conda-linux-gcc-py36', 'centos-6', 'wheel-win-cp36m', 'wheel-linux-cp27mu', 'conda-win-vs2015-py36', 'wheel-linux-cp35m', 'wheel-osx-cp27m', 'wheel-osx-cp37m', 'gandiva-jar-trusty', 'conda-osx-clang-py27'}

[pitrou]

@ursabot crossbow package wheel-win-cp36m

[kszucs]

@pitrou You want to run @ursabot crossbow package --group wheel
or a standalone task, just like You did :)

[ursabot]

AMD64 Conda Crossbow (#22370) builder has been succeeded.

Revision: 3c0e62eaa7232a8760193687b41eec901552292e

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-29

Task	Status
wheel-win-cp36m

[kszucs]

@pitrou For testing the linux wheels you need to rebuild the image, add docker-compose build python-manylinux1 after https://github.com/apache/arrow/blob/master/dev/tasks/python-wheels/travis.linux.yml#L49 then submit the linux task.

[pitrou]

@kszucs I've already rebuilt and uploaded the image from my machine.

[pitrou]

@kszucs What does this task do? How is it different from the manylinux task on travis-CI?

[kszucs]

Basically the same, but uploads the artifact and the wheel tasks build wheels for multiple python versions.

[pitrou]

@ursabot crossbow package wheel-win-cp36m

[ursabot]

AMD64 Conda Crossbow (#22382) builder has been succeeded.

Revision: 519ae92d908270fb9629693b2bf947f73cce574d

Submitted crossbow builds: ursa-labs/crossbow @ ursabot-30

Task	Status
wheel-win-cp36m

[pitrou]

I give up on Windows. @kou@xhochy feel free to take a look.

[codecov-io]

Codecov Report

Merging #4628 into master will decrease coverage by 23.77%.
The diff coverage is n/a.

@@ Coverage Diff @@## master #4628 +/- ##
===========================================
- Coverage 88.49% 64.71% -23.78% 
===========================================
Files 890 480 -410 Lines 110406 62556 -47850 Branches 1418 0 -1418 ===========================================
- Hits 97701 40482 -57219 - Misses 12424 22074 +9650 + Partials 281 0 -281

Impacted Files	Coverage Δ
cpp/src/arrow/util/memory.h	0% <0%> (-100%)	⬇️
cpp/src/gandiva/date_utils.h	0% <0%> (-100%)	⬇️
cpp/src/arrow/util/memory.cc	0% <0%> (-100%)	⬇️
cpp/src/arrow/filesystem/util-internal.cc	0% <0%> (-100%)	⬇️
cpp/src/arrow/util/sse-util.h	0% <0%> (-100%)	⬇️
cpp/src/gandiva/decimal_type_util.h	0% <0%> (-100%)	⬇️
cpp/src/arrow/compute/logical_type.h	0% <0%> (-100%)	⬇️
cpp/src/parquet/hasher.h	0% <0%> (-100%)	⬇️
cpp/src/gandiva/basic_decimal_scalar.h	0% <0%> (-100%)	⬇️
cpp/src/arrow/compute/kernels/boolean.cc	0% <0%> (-100%)	⬇️
... and 641 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 961927a...5bc9e83. Read the comment docs.

[wesm]

The Windows wheel build is currently broken? Is there a JIRA for this?

[pitrou]

@wesm Why broken?

[wesm]

You said "I give up on Windows." I wondered what you meant

[wesm]

+1

[pitrou]

Ah, no, it's simply that I gave up on enabling bz2 in Windows wheels.

[wesm]

OK, I opened https://issues.apache.org/jira/browse/ARROW-5667