Limits GitHub3.py in order to avoid backtracking by potiuk · Pull Request #21824 · apache/airflow

potiuk · 2022-02-25T18:05:14Z

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App
Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced
PyJWT>=2.3.0 but so far pip was able to resolve it without getting
into a long backtracking loop and figure out that github3 3.0.0 version
is the right version similarly limiting it to 3.1.2 causes pip not to
enter the backtracking loop. Apparently when there Are 3 versions with
PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and
fails to resolve that github3 3.0.0 is the right version to use.
This limitation could be removed if PyJWT limitation < 2.0.0 is dropped
from FAB or when pip resolution is improved to handle the case, The
issue which describes this PIP behaviour and hopefully allowing to
improve it is tracked in pypa/pip#10924.

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced PyJWT>=2.3.0 but so far `pip` was able to resolve it without getting into a long backtracking loop and figure out that github3 3.0.0 version is the right version similarly limiting it to 3.1.2 causes pip not to enter the backtracking loop. Apparently when there Are 3 versions with PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and fails to resolve that github3 3.0.0 is the right version to use. This limitation could be removed if PyJWT limitation < 2.0.0 is dropped from FAB or when pip resolution is improved to handle the case, The issue which describes this PIP behaviour and hopefully allowing to improve it is tracked in pypa/pip#10924.

jedcunningham

Just a coupe nits.

github-actions · 2022-02-25T18:11:15Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

github-actions · 2022-02-25T18:11:16Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

potiuk · 2022-02-25T19:34:31Z

That was an accidental fail with image 3.9. Merging.

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924)

dstandish · 2022-02-25T22:57:07Z

thank you @potiuk 🙏

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message.

…21825) * Add CI jobs and tooling to aid with tracking backtracking pip issues This is a follow-up after investigation done with failing main builds (resulting in #21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message. * Update dev/breeze/src/airflow_ci/find_newer_dependencies.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>

potiuk · 2022-02-25T18:05:14Z

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App
Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced
PyJWT>=2.3.0 but so far pip was able to resolve it without getting
into a long backtracking loop and figure out that github3 3.0.0 version
is the right version similarly limiting it to 3.1.2 causes pip not to
enter the backtracking loop. Apparently when there Are 3 versions with
PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and
fails to resolve that github3 3.0.0 is the right version to use.
This limitation could be removed if PyJWT limitation < 2.0.0 is dropped
from FAB or when pip resolution is improved to handle the case, The
issue which describes this PIP behaviour and hopefully allowing to
improve it is tracked in pypa/pip#10924.

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced PyJWT>=2.3.0 but so far `pip` was able to resolve it without getting into a long backtracking loop and figure out that github3 3.0.0 version is the right version similarly limiting it to 3.1.2 causes pip not to enter the backtracking loop. Apparently when there Are 3 versions with PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and fails to resolve that github3 3.0.0 is the right version to use. This limitation could be removed if PyJWT limitation < 2.0.0 is dropped from FAB or when pip resolution is improved to handle the case, The issue which describes this PIP behaviour and hopefully allowing to improve it is tracked in pypa/pip#10924.

jedcunningham

Just a coupe nits.

github-actions · 2022-02-25T18:11:15Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

github-actions · 2022-02-25T18:11:16Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

potiuk · 2022-02-25T19:34:31Z

That was an accidental fail with image 3.9. Merging.

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924)

dstandish · 2022-02-25T22:57:07Z

thank you @potiuk 🙏

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message.

…21825) * Add CI jobs and tooling to aid with tracking backtracking pip issues This is a follow-up after investigation done with failing main builds (resulting in #21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message. * Update dev/breeze/src/airflow_ci/find_newer_dependencies.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>

potiuk · 2022-02-25T18:05:14Z

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App
Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced
PyJWT>=2.3.0 but so far pip was able to resolve it without getting
into a long backtracking loop and figure out that github3 3.0.0 version
is the right version similarly limiting it to 3.1.2 causes pip not to
enter the backtracking loop. Apparently when there Are 3 versions with
PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and
fails to resolve that github3 3.0.0 is the right version to use.
This limitation could be removed if PyJWT limitation < 2.0.0 is dropped
from FAB or when pip resolution is improved to handle the case, The
issue which describes this PIP behaviour and hopefully allowing to
improve it is tracked in pypa/pip#10924.

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced PyJWT>=2.3.0 but so far `pip` was able to resolve it without getting into a long backtracking loop and figure out that github3 3.0.0 version is the right version similarly limiting it to 3.1.2 causes pip not to enter the backtracking loop. Apparently when there Are 3 versions with PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and fails to resolve that github3 3.0.0 is the right version to use. This limitation could be removed if PyJWT limitation < 2.0.0 is dropped from FAB or when pip resolution is improved to handle the case, The issue which describes this PIP behaviour and hopefully allowing to improve it is tracked in pypa/pip#10924.

jedcunningham

Just a coupe nits.

github-actions · 2022-02-25T18:11:15Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

github-actions · 2022-02-25T18:11:16Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

potiuk · 2022-02-25T19:34:31Z

That was an accidental fail with image 3.9. Merging.

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924)

dstandish · 2022-02-25T22:57:07Z

thank you @potiuk 🙏

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message.

…21825) * Add CI jobs and tooling to aid with tracking backtracking pip issues This is a follow-up after investigation done with failing main builds (resulting in #21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message. * Update dev/breeze/src/airflow_ci/find_newer_dependencies.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>

potiuk · 2022-02-25T18:05:14Z

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App
Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced
PyJWT>=2.3.0 but so far pip was able to resolve it without getting
into a long backtracking loop and figure out that github3 3.0.0 version
is the right version similarly limiting it to 3.1.2 causes pip not to
enter the backtracking loop. Apparently when there Are 3 versions with
PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and
fails to resolve that github3 3.0.0 is the right version to use.
This limitation could be removed if PyJWT limitation < 2.0.0 is dropped
from FAB or when pip resolution is improved to handle the case, The
issue which describes this PIP behaviour and hopefully allowing to
improve it is tracked in pypa/pip#10924.

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced PyJWT>=2.3.0 but so far `pip` was able to resolve it without getting into a long backtracking loop and figure out that github3 3.0.0 version is the right version similarly limiting it to 3.1.2 causes pip not to enter the backtracking loop. Apparently when there Are 3 versions with PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and fails to resolve that github3 3.0.0 is the right version to use. This limitation could be removed if PyJWT limitation < 2.0.0 is dropped from FAB or when pip resolution is improved to handle the case, The issue which describes this PIP behaviour and hopefully allowing to improve it is tracked in pypa/pip#10924.

jedcunningham

Just a coupe nits.

github-actions · 2022-02-25T18:11:15Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

github-actions · 2022-02-25T18:11:16Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

potiuk · 2022-02-25T19:34:31Z

That was an accidental fail with image 3.9. Merging.

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924)

dstandish · 2022-02-25T22:57:07Z

thank you @potiuk 🙏

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message.

…21825) * Add CI jobs and tooling to aid with tracking backtracking pip issues This is a follow-up after investigation done with failing main builds (resulting in #21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message. * Update dev/breeze/src/airflow_ci/find_newer_dependencies.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>

potiuk · 2022-02-25T18:05:14Z

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App
Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced
PyJWT>=2.3.0 but so far pip was able to resolve it without getting
into a long backtracking loop and figure out that github3 3.0.0 version
is the right version similarly limiting it to 3.1.2 causes pip not to
enter the backtracking loop. Apparently when there Are 3 versions with
PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and
fails to resolve that github3 3.0.0 is the right version to use.
This limitation could be removed if PyJWT limitation < 2.0.0 is dropped
from FAB or when pip resolution is improved to handle the case, The
issue which describes this PIP behaviour and hopefully allowing to
improve it is tracked in pypa/pip#10924.

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced PyJWT>=2.3.0 but so far `pip` was able to resolve it without getting into a long backtracking loop and figure out that github3 3.0.0 version is the right version similarly limiting it to 3.1.2 causes pip not to enter the backtracking loop. Apparently when there Are 3 versions with PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and fails to resolve that github3 3.0.0 is the right version to use. This limitation could be removed if PyJWT limitation < 2.0.0 is dropped from FAB or when pip resolution is improved to handle the case, The issue which describes this PIP behaviour and hopefully allowing to improve it is tracked in pypa/pip#10924.

jedcunningham

Just a coupe nits.

github-actions · 2022-02-25T18:11:15Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

github-actions · 2022-02-25T18:11:16Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

potiuk · 2022-02-25T19:34:31Z

That was an accidental fail with image 3.9. Merging.

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924)

dstandish · 2022-02-25T22:57:07Z

thank you @potiuk 🙏

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message.

…21825) * Add CI jobs and tooling to aid with tracking backtracking pip issues This is a follow-up after investigation done with failing main builds (resulting in #21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message. * Update dev/breeze/src/airflow_ci/find_newer_dependencies.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>

potiuk · 2022-02-25T18:05:14Z

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App
Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced
PyJWT>=2.3.0 but so far pip was able to resolve it without getting
into a long backtracking loop and figure out that github3 3.0.0 version
is the right version similarly limiting it to 3.1.2 causes pip not to
enter the backtracking loop. Apparently when there Are 3 versions with
PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and
fails to resolve that github3 3.0.0 is the right version to use.
This limitation could be removed if PyJWT limitation < 2.0.0 is dropped
from FAB or when pip resolution is improved to handle the case, The
issue which describes this PIP behaviour and hopefully allowing to
improve it is tracked in pypa/pip#10924.

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced PyJWT>=2.3.0 but so far `pip` was able to resolve it without getting into a long backtracking loop and figure out that github3 3.0.0 version is the right version similarly limiting it to 3.1.2 causes pip not to enter the backtracking loop. Apparently when there Are 3 versions with PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and fails to resolve that github3 3.0.0 is the right version to use. This limitation could be removed if PyJWT limitation < 2.0.0 is dropped from FAB or when pip resolution is improved to handle the case, The issue which describes this PIP behaviour and hopefully allowing to improve it is tracked in pypa/pip#10924.

jedcunningham

Just a coupe nits.

github-actions · 2022-02-25T18:11:15Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

github-actions · 2022-02-25T18:11:16Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

potiuk · 2022-02-25T19:34:31Z

That was an accidental fail with image 3.9. Merging.

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924)

dstandish · 2022-02-25T22:57:07Z

thank you @potiuk 🙏

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message.

…21825) * Add CI jobs and tooling to aid with tracking backtracking pip issues This is a follow-up after investigation done with failing main builds (resulting in #21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message. * Update dev/breeze/src/airflow_ci/find_newer_dependencies.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>

potiuk · 2022-02-25T18:05:14Z

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App
Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced
PyJWT>=2.3.0 but so far pip was able to resolve it without getting
into a long backtracking loop and figure out that github3 3.0.0 version
is the right version similarly limiting it to 3.1.2 causes pip not to
enter the backtracking loop. Apparently when there Are 3 versions with
PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and
fails to resolve that github3 3.0.0 is the right version to use.
This limitation could be removed if PyJWT limitation < 2.0.0 is dropped
from FAB or when pip resolution is improved to handle the case, The
issue which describes this PIP behaviour and hopefully allowing to
improve it is tracked in pypa/pip#10924.

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced PyJWT>=2.3.0 but so far `pip` was able to resolve it without getting into a long backtracking loop and figure out that github3 3.0.0 version is the right version similarly limiting it to 3.1.2 causes pip not to enter the backtracking loop. Apparently when there Are 3 versions with PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and fails to resolve that github3 3.0.0 is the right version to use. This limitation could be removed if PyJWT limitation < 2.0.0 is dropped from FAB or when pip resolution is improved to handle the case, The issue which describes this PIP behaviour and hopefully allowing to improve it is tracked in pypa/pip#10924.

jedcunningham

Just a coupe nits.

github-actions · 2022-02-25T18:11:15Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

github-actions · 2022-02-25T18:11:16Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

potiuk · 2022-02-25T19:34:31Z

That was an accidental fail with image 3.9. Merging.

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924)

dstandish · 2022-02-25T22:57:07Z

thank you @potiuk 🙏

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message.

…21825) * Add CI jobs and tooling to aid with tracking backtracking pip issues This is a follow-up after investigation done with failing main builds (resulting in #21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message. * Update dev/breeze/src/airflow_ci/find_newer_dependencies.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>

potiuk · 2022-02-25T18:05:14Z

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App
Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced
PyJWT>=2.3.0 but so far pip was able to resolve it without getting
into a long backtracking loop and figure out that github3 3.0.0 version
is the right version similarly limiting it to 3.1.2 causes pip not to
enter the backtracking loop. Apparently when there Are 3 versions with
PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and
fails to resolve that github3 3.0.0 is the right version to use.
This limitation could be removed if PyJWT limitation < 2.0.0 is dropped
from FAB or when pip resolution is improved to handle the case, The
issue which describes this PIP behaviour and hopefully allowing to
improve it is tracked in pypa/pip#10924.

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

Github3 version 3.1.2 requires PyJWT>=2.3.0 which clashes with Flask App Builder where PyJWT is <2.0.0 Actually GitHub3.1.0 already introduced PyJWT>=2.3.0 but so far `pip` was able to resolve it without getting into a long backtracking loop and figure out that github3 3.0.0 version is the right version similarly limiting it to 3.1.2 causes pip not to enter the backtracking loop. Apparently when there Are 3 versions with PyJWT>=2.3.0 (3.1.0, 3.1.1 an 3.1.2) pip enters into backtrack loop and fails to resolve that github3 3.0.0 is the right version to use. This limitation could be removed if PyJWT limitation < 2.0.0 is dropped from FAB or when pip resolution is improved to handle the case, The issue which describes this PIP behaviour and hopefully allowing to improve it is tracked in pypa/pip#10924.

jedcunningham

Just a coupe nits.

github-actions · 2022-02-25T18:11:15Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

github-actions · 2022-02-25T18:11:16Z

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

potiuk · 2022-02-25T19:34:31Z

That was an accidental fail with image 3.9. Merging.

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924)

dstandish · 2022-02-25T22:57:07Z

thank you @potiuk 🙏

This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message.

…21825) * Add CI jobs and tooling to aid with tracking backtracking pip issues This is a follow-up after investigation done with failing main builds (resulting in #21824 and tracked in pypa/pip#10924) Handling failure of image building has been also improved as part of the PR. Instead of separate "cancel" job (which did not really work anyway) we build and push empty images instead and the empty images are handled in the "wait for images" job with appropriate message. * Update dev/breeze/src/airflow_ci/find_newer_dependencies.py Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com> Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>

potiuk requested review from ashb, dstandish, eladkal, ephraimbuddy and kaxil February 25, 2022 18:05

jedcunningham approved these changes Feb 25, 2022
View reviewed changes

Comment threadsetup.py Outdated
Comment threadsetup.py Outdated

github-actionsBot added the full tests needed We need to run full set of tests for this PR to merge label Feb 25, 2022

potiukand others added 2 commits February 25, 2022 19:11

Update setup.py
7e83b71
Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

Update setup.py
0f1544b
Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>

potiuk merged commit 16f5185 into apache:mainFeb 25, 2022

potiuk deleted the fix-main-pip-backtracking branch February 25, 2022 19:34

potiuk added a commit to potiuk/airflow that referenced this pull request Feb 25, 2022
Add CI jobs and tooling to aid with tracking backtracking pip issues
63e10cd
This is a follow-up after investigation done with failing main builds (resulting in apache#21824 and tracked in pypa/pip#10924)

jedcunningham added the type:misc/internal Changelog: Misc changes that should appear in change log label Feb 28, 2022

ephraimbuddy modified the milestone: Airflow 2.2.5 Mar 24, 2022

ephraimbuddy added changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) and removed type:misc/internal Changelog: Misc changes that should appear in change log labels Apr 26, 2022