Skip to content

ARROW-7735: [Release][Python] Use pip to install dependencies for wheel verification - #6339

Closed
kszucs wants to merge 2 commits into
apache:masterfrom
kszucs:wheel-verification
Closed

ARROW-7735: [Release][Python] Use pip to install dependencies for wheel verification#6339
kszucs wants to merge 2 commits into
apache:masterfrom
kszucs:wheel-verification

Conversation

@kszucs

@kszucskszucs commented Feb 2, 2020

Copy link
Copy Markdown
Member

The wheel verification script fails for python 3.5.
At the same time the wheel properly works for python 3.5 docker
images without conda environment.

Conda forge doesn't maintain packages for python 3.5 anymore
and something must have mixed with the numpy versions.

This change fixed the wheel verification locally for me.

@github-actions

Copy link
Copy Markdown

Thanks for opening a pull request!

Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW

Then could you also rename pull request title in the following format?

ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}

See also:

@koukou changed the title [Release][Python] Use pip to install dependencies for wheel verificationARROW-7735: [Release][Python] Use pip to install dependencies for wheel verificationFeb 2, 2020
@github-actions

Copy link
Copy Markdown

@kou

kou commented Feb 2, 2020

Copy link
Copy Markdown
Member

Python 3.5 + manylinux1 passes with this change.

But Python 3.5 + manylinux2000 shows the following error:

+ for ml_spec in ${manylinuxes}
+ [[ 3.5m = \2\.\7\m\u ]]
+ pip install python-rc/0.16.0-rc2/pyarrow-0.16.0-cp35-cp35m-manylinux2010_x86_64.whl
pyarrow-0.16.0-cp35-cp35m-manylinux2010_x86_64.whl is not a supported wheel on this platform.
$ python -c "from pip._internal import pep425tags; print('\n'.join(['-'.join(t) for t in pep425tags.get_supported()]))"cp35-cp35m-manylinux1_x86_64cp35-cp35m-linux_x86_64cp35-abi3-manylinux1_x86_64cp35-abi3-linux_x86_64cp35-none-manylinux1_x86_64cp35-none-linux_x86_64cp34-abi3-manylinux1_x86_64cp34-abi3-linux_x86_64cp33-abi3-manylinux1_x86_64cp33-abi3-linux_x86_64cp32-abi3-manylinux1_x86_64cp32-abi3-linux_x86_64py3-none-manylinux1_x86_64py3-none-linux_x86_64cp35-none-anycp3-none-anypy35-none-anypy3-none-anypy34-none-anypy33-none-anypy32-none-anypy31-none-anypy30-none-any

@kou

kou commented Feb 2, 2020

Copy link
Copy Markdown
Member

It may be better that we use Docker (python:2.7, python:3.5, python:3.6 and python:3.7) instead of Conda.

$ % docker run -it --rm python:3.5Python 3.5.9 (default, Feb 2 2020, 10:14:19) [GCC 8.3.0] on linuxType "help", "copyright", "credits" or "license" for more information.>>> from pip._internal import pep425tags; print('\n'.join([str(t) for t in pep425tags.get_supported()]))cp35-cp35m-manylinux2014_x86_64cp35-cp35m-manylinux2010_x86_64cp35-cp35m-manylinux1_x86_64cp35-cp35m-linux_x86_64cp35-abi3-manylinux2014_x86_64cp35-abi3-manylinux2010_x86_64cp35-abi3-manylinux1_x86_64cp35-abi3-linux_x86_64cp35-none-manylinux2014_x86_64cp35-none-manylinux2010_x86_64cp35-none-manylinux1_x86_64cp35-none-linux_x86_64cp34-abi3-manylinux2014_x86_64cp34-abi3-manylinux2010_x86_64cp34-abi3-manylinux1_x86_64cp34-abi3-linux_x86_64cp33-abi3-manylinux2014_x86_64cp33-abi3-manylinux2010_x86_64cp33-abi3-manylinux1_x86_64cp33-abi3-linux_x86_64cp32-abi3-manylinux2014_x86_64cp32-abi3-manylinux2010_x86_64cp32-abi3-manylinux1_x86_64cp32-abi3-linux_x86_64py35-none-manylinux2014_x86_64py35-none-manylinux2010_x86_64py35-none-manylinux1_x86_64py35-none-linux_x86_64py3-none-manylinux2014_x86_64py3-none-manylinux2010_x86_64py3-none-manylinux1_x86_64py3-none-linux_x86_64py34-none-manylinux2014_x86_64py34-none-manylinux2010_x86_64py34-none-manylinux1_x86_64py34-none-linux_x86_64py33-none-manylinux2014_x86_64py33-none-manylinux2010_x86_64py33-none-manylinux1_x86_64py33-none-linux_x86_64py32-none-manylinux2014_x86_64py32-none-manylinux2010_x86_64py32-none-manylinux1_x86_64py32-none-linux_x86_64py31-none-manylinux2014_x86_64py31-none-manylinux2010_x86_64py31-none-manylinux1_x86_64py31-none-linux_x86_64py30-none-manylinux2014_x86_64py30-none-manylinux2010_x86_64py30-none-manylinux1_x86_64py30-none-linux_x86_64cp35-none-anypy35-none-anypy3-none-anypy34-none-anypy33-none-anypy32-none-anypy31-none-anypy30-none-any

@kszucs

Copy link
Copy Markdown
MemberAuthor

@kou for testing the produced wheels we already use these docker images - which doesn't mean that we cannot rerun these checks during the release verification process.

Although docker would only be suitable for testing the linux wheels.

@pitrou

pitrou commented Feb 3, 2020

Copy link
Copy Markdown
Member

But Python 3.5 + manylinux2000 shows the following error:

It's quite possible that manylinux2010 doesn't have tooling for Python 3.5.
We should stop caring about Python 3.5. It's obsolete.

@pitroupitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

pip install -r ${ARROW_DIR}/python/requirements-test.txt

# execute the tests
pytest -sv --pyargs pyarrow

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The -s and -v flags are spammy. Better to remove them IMHO.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a sec

@kszucskszucs left a comment

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@kszucskszucs closed this in cb81f7dFeb 3, 2020
kszucs added a commit that referenced this pull request Feb 7, 2020
…el verification
The wheel verification script fails for python 3.5.
At the same time the wheel properly works for python 3.5 docker
images without conda environment.
Conda forge doesn't maintain packages for python 3.5 anymore
and something must have mixed with the numpy versions.
This change fixed the wheel verification locally for me.
Closes#6339 from kszucs/wheel-verification and squashes the following commits:
3e96949 <Krisztián Szűcs> remove pytest verbose flags
026e5fb <Krisztián Szűcs> use pip to install dependencies for wheel verification
Authored-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@kszucs@kou@pitrou