GH-18036: [Packaging][Python] Add support for musllinux_1_2

[kvanzuijlen]

Rationale for this change

This change adds pre-building of musllinux wheels, which can be installed on Alpine Linux

What changes are included in this PR?

I'm only using musllinux_1_2 since I couldn't get musllinux_1_1 to work (and can't say for sure if it's even needed)

Are these changes tested?

TBA

Are there any user-facing changes?

Wheels will be added for musllinux

• GitHub Issue: [Packaging] Add wheels for Alpine Linux #18036

[github-actions]

⚠️ GitHub issue #18036has been automatically assigned in GitHub to PR creator.

[kvanzuijlen]

@kou I'm running archery docker run python-wheel-musllinux-1-2 locally right now and most of it seems to work. Right now it fails locally on the cmake --build . --target install step, which seems to be running within the container itself. That step seems to run out of memory locally though, so I'm not sure if I can test everything right now. I'm now trying to run it again with a little more memory allocated to Docker, but if that doesn't help I'll need to run it on my other laptop/I'd be unable to run it locally.

[kou]

How much memory did you use?

[kvanzuijlen]

I increased the max amount to 10gb, so I'm starting to think it's not actually a memory issue, but it's the only thing I can find when googling the error. I'll try some more troubleshooting steps (like running the build interactively to inspect the environment after the error).

[kou]

Could you share your logs?

[kvanzuijlen]

Of course!

+ cmake --build . --target install
[58/128] Building CXX object src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/Unity/unity_1_cxx.cxx.o
FAILED: src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/Unity/unity_1_cxx.cxx.o /usr/local/bin/ccache /usr/bin/c++ -DARROW_DS_EXPORTING -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_AVX512 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_HDFS -DARROW_MIMALLOC -DARROW_WITH_BROTLI -DARROW_WITH_BZ2 -DARROW_WITH_LZ4 -DARROW_WITH_RE2 -DARROW_WITH_SNAPPY -DARROW_WITH_TIMING_TESTS -DARROW_WITH_UTF8PROC -DARROW_WITH_ZLIB -DARROW_WITH_ZSTD -DAWS_SDK_VERSION_MAJOR=1 -DAWS_SDK_VERSION_MINOR=11 -DAWS_SDK_VERSION_PATCH=201 -DAWS_USE_EPOLL -DAZ_RTTI -DCURL_STATICLIB -DURI_STATIC_BUILD -DUTF8PROC_STATIC -I/tmp/arrow-build/src -I/arrow/cpp/src -I/arrow/cpp/src/generated -I/arrow/cpp/src/parquet -isystem /arrow/cpp/thirdparty/flatbuffers/include -isystem /arrow/cpp/thirdparty/hadoop/include -isystem /opt/vcpkg/installed/amd64-linux-static-release/include -isystem /tmp/arrow-build/orc_ep-install/include -isystem /opt/vcpkg/installed/amd64-linux-static-release/share/rapidjson/../../include -isystem /tmp/arrow-build/jemalloc_ep-prefix/src -isystem /tmp/arrow-build/mimalloc_ep/src/mimalloc_ep/include/mimalloc-2.0 -Wno-noexcept-type -Wno-subobject-linkage -fdiagnostics-color=always -Wall -fno-semantic-interposition -msse4.2 -O3 -DNDEBUG -O2 -ftree-vectorize -fPIC -DS2N_KYBER512R3_AVX2_BMI2 -DS2N_CPUID_AVAILABLE -DS2N_FEATURES_AVAILABLE -fPIC -DS2N_FALL_THROUGH_SUPPORTED -DS2N___RESTRICT__SUPPORTED -DS2N_MADVISE_SUPPORTED -DS2N_CLONE_SUPPORTED -std=c++17 -MD -MT src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/Unity/unity_1_cxx.cxx.o -MF src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/Unity/unity_1_cxx.cxx.o.d -o src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/Unity/unity_1_cxx.cxx.o -c /tmp/arrow-build/src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/Unity/unity_1_cxx.cxx
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
[71/128] Building CXX object src/parquet/CMakeFiles/parquet_objlib.dir/Unity/unity_1_cxx.cxx.o
ninja: build stopped: subcommand failed.

[kou]

Could you disable CMAKE_UNITY_BUILD for now?
https://github.com/apache/arrow/pull/40177/files#diff-323a5673627df4e855450c3b25e09df91f0b5e1beb654194f2b2bbb6f33f6e90R73

[kvanzuijlen]

Could you disable CMAKE_UNITY_BUILD for now? https://github.com/apache/arrow/pull/40177/files#diff-323a5673627df4e855450c3b25e09df91f0b5e1beb654194f2b2bbb6f33f6e90R73

Running now! Will let you know the result when it's finished

[kvanzuijlen]

It got a little further and now times out git status:

+ python setup.py bdist_wheel
setup.py:34: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
dist.fetch_build_eggs(dist.setup_requires)
WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
Traceback (most recent call last):
File "setup.py", line 475, in <module>
setup(
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools/__init__.py", line 103, in setup
return distutils.core.setup(**attrs)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 147, in setup
_setup_distribution = dist = klass(attrs)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools/dist.py", line 303, in __init__
_Distribution.__init__(self, dist_attrs)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 283, in __init__
self.finalize_options()
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools/dist.py", line 654, in finalize_options
ep(self)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools/dist.py", line 674, in _finalize_setup_keywords
ep.load()(self, ep.name, value)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/_integration/setuptools.py", line 101, in version_keyword
_assign_version(dist, config)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/_integration/setuptools.py", line 53, in _assign_version
maybe_version = _get_version(config, force_write_version_files=True)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/_get_version_impl.py", line 93, in _get_version
parsed_version = parse_version(config)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/_get_version_impl.py", line 56, in parse_version
or parse_scm_version(config)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/_get_version_impl.py", line 27, in parse_scm_version
parse_result = config.parse(config.absolute_root, config=config)
File "setup.py", line 430, in parse_git
return parse(root, **kwargs)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/git.py", line 211, in parse
return _git_parse_inner(
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/git.py", line 267, in _git_parse_inner
dirty = wd.is_dirty()
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/git.py", line 94, in is_dirty
return run_git(
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/git.py", line 58, in run_git
return _run(
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/setuptools_scm/_run_cmd.py", line 144, in run
res = subprocess.run(
File "/opt/python/cp38-cp38/lib/python3.8/subprocess.py", line 495, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "/opt/python/cp38-cp38/lib/python3.8/subprocess.py", line 1028, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/opt/python/cp38-cp38/lib/python3.8/subprocess.py", line 1885, in _communicate
self._check_timeout(endtime, orig_timeout, stdout, stderr)
File "/opt/python/cp38-cp38/lib/python3.8/subprocess.py", line 1072, in _check_timeout
raise TimeoutExpired(
subprocess.TimeoutExpired: Command '['git', '--git-dir', '/arrow/.git', 'status', '--porcelain', '--untracked-files=no']' timed out after 20 seconds

[kou]

If you run the git status ... command manually, does it work?

[kvanzuijlen]

Yes. Not sure if it's because not everything has ran yet (I just started the Docker container using docker run -v .:/arrow:delegated -v ${DOCKER_VOLUME_PREFIX}python-wheel-musllinux-1-2-ccache:/ccache:delegated -it apache/arrow-dev:amd64-python-3.8-wheel-musllinux-1-2-vcpkg-a42af01b72c28a8e1d7b48107b33e4f286a55ef6 /bin/ash) or something else?

[kou]

You can run the same configured container by archery docker run python-wheel-musllinux-1-12 bash.

[kvanzuijlen]

It runs fine when I run it manually.

[github-actions]

Thank you for your contribution. Unfortunately, this pull request has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this PR will be closed in 14 days. Feel free to re-open this if it has been closed in error. If you do not have repository permissions to reopen the PR, please tag a maintainer.

[kou]

Done by #45470.