Fix tox installs by jmahlik · Pull Request #4603 · aws/sagemaker-python-sdk

jmahlik · 2024-04-22T16:26:04Z

Issue #, if available:

Description of changes:
Greatly speed up the tox runtime.

Testing done:
tox -e flake8,black-format,black-check,spelling,twine,doc8,typing,docstyle -r -p auto now completes in a few minutes vs. over an hour locally.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting.

… needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591

Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

* change: adjust tox black stages to install from requirements file Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting. * change: improve tox install times by not installing all deps when not needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591 * change: bump twine version to 5.0.0 Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

jmahlik · 2024-04-22T16:26:04Z

Issue #, if available:

Closes#4591

Description of changes:
Greatly speed up the tox runtime.

Testing done:
tox -e flake8,black-format,black-check,spelling,twine,doc8,typing,docstyle -r -p auto now completes in a few minutes vs. over an hour locally.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting.

… needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591

Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

* change: adjust tox black stages to install from requirements file Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting. * change: improve tox install times by not installing all deps when not needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591 * change: bump twine version to 5.0.0 Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

jmahlik · 2024-04-22T16:26:04Z

Issue #, if available:

Closes#4591

Description of changes:
Greatly speed up the tox runtime.

Testing done:
tox -e flake8,black-format,black-check,spelling,twine,doc8,typing,docstyle -r -p auto now completes in a few minutes vs. over an hour locally.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting.

… needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591

Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

* change: adjust tox black stages to install from requirements file Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting. * change: improve tox install times by not installing all deps when not needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591 * change: bump twine version to 5.0.0 Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

jmahlik · 2024-04-22T16:26:04Z

Issue #, if available:

Closes#4591

Description of changes:
Greatly speed up the tox runtime.

Testing done:
tox -e flake8,black-format,black-check,spelling,twine,doc8,typing,docstyle -r -p auto now completes in a few minutes vs. over an hour locally.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting.

… needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591

Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

* change: adjust tox black stages to install from requirements file Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting. * change: improve tox install times by not installing all deps when not needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591 * change: bump twine version to 5.0.0 Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

jmahlik · 2024-04-22T16:26:04Z

Issue #, if available:

Closes#4591

Description of changes:
Greatly speed up the tox runtime.

Testing done:
tox -e flake8,black-format,black-check,spelling,twine,doc8,typing,docstyle -r -p auto now completes in a few minutes vs. over an hour locally.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting.

… needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591

Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

* change: adjust tox black stages to install from requirements file Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting. * change: improve tox install times by not installing all deps when not needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591 * change: bump twine version to 5.0.0 Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

jmahlik · 2024-04-22T16:26:04Z

Issue #, if available:

Closes#4591

Description of changes:
Greatly speed up the tox runtime.

Testing done:
tox -e flake8,black-format,black-check,spelling,twine,doc8,typing,docstyle -r -p auto now completes in a few minutes vs. over an hour locally.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting.

… needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591

Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

* change: adjust tox black stages to install from requirements file Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting. * change: improve tox install times by not installing all deps when not needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591 * change: bump twine version to 5.0.0 Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

jmahlik · 2024-04-22T16:26:04Z

Issue #, if available:

Closes#4591

Description of changes:
Greatly speed up the tox runtime.

Testing done:
tox -e flake8,black-format,black-check,spelling,twine,doc8,typing,docstyle -r -p auto now completes in a few minutes vs. over an hour locally.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting.

… needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591

Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

* change: adjust tox black stages to install from requirements file Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting. * change: improve tox install times by not installing all deps when not needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591 * change: bump twine version to 5.0.0 Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

jmahlik · 2024-04-22T16:26:04Z

Issue #, if available:

Closes#4591

Description of changes:
Greatly speed up the tox runtime.

Testing done:
tox -e flake8,black-format,black-check,spelling,twine,doc8,typing,docstyle -r -p auto now completes in a few minutes vs. over an hour locally.

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting.

… needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591

Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

* change: adjust tox black stages to install from requirements file Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting. * change: improve tox install times by not installing all deps when not needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591 * change: bump twine version to 5.0.0 Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

jmahlik requested a review from a team as a code owner April 22, 2024 16:26

jmahlik requested review from knikure and removed request for a teamApril 22, 2024 16:26

jmahlik had a problem deploying to manual-approval April 22, 2024 16:26 — with GitHub Actions Error

jmahlik mentioned this pull request Apr 22, 2024
fix: properly close files in lineage queries and tests #4587
Merged
9 tasks

jmahlik added 3 commits April 22, 2024 13:44

change: adjust tox black stages to install from requirements file
a93cb63
Also remove uneeded config in the command, it is now set in the pyproject.toml so IDE's and other tooling outside tox picks up the setting.

change: improve tox install times by not installing all deps when not…
51cc1db
… needed Moved to using deps to override the test extra being installed in skip_install tox stages. Now only the needed deps are installed. Improves install time by around 7x. Closesaws#4591

change: bump twine version to 5.0.0
9b5f592
Twine 3.8.0 didn't include its dependencies. The tox stage happened to work from packaging being installed as a transitive. Ref: pypa/twine#894

jmahlik force-pushed the fix-tox-installs branch from 8fcfad0 to 9b5f592Compare April 22, 2024 18:44

jmahlik temporarily deployed to manual-approval April 22, 2024 18:44 — with GitHub Actions Inactive

liujiaorr approved these changes Apr 22, 2024
View reviewed changes

liujiaorr merged commit f026f02 into aws:masterApr 22, 2024