Uh oh!
There was an error while loading. Please reload this page.
Fixed replacement of - to . in sdist package name - #37410
Conversation
Taragolis
left a comment
There was a problem hiding this comment.
Looks like it is another part which could be simplified by packaging.utils ... probably one day.
potiuk
commented
Feb 14, 2024
A bit more complex - because we are using |
I can't see a problem for generate valid canonical name from provider_id and build mapping between them (in globals?) importjsonfrompathlibimportPathfrompackaging.utilsimportcanonicalize_name, parse_wheel_filename, parse_sdist_filenamePROJECT_DIR=Path("").resolve(). # Some path hereprovider_dependencies=PROJECT_DIR/"generated"/"provider_dependencies.json"assertprovider_dependencies.exists()
provider_dependencies_json=json.loads(provider_dependencies.read_text())
mapping= {
pi: canonicalize_name(f"apache-airflow-providers-{pi}") forpiinprovider_dependencies_json
}
inverted_mapping= {v: kfork, vinmapping.items()}
print(json.dumps(mapping, indent=2))
# Samplessdist_files= [
"apache-airflow-providers-airbyte-3.2.1.tar.gz",
"apache_airflow_providers_airbyte-3.6.0.tar.gz"
]
wheel_files= [
"apache_airflow_providers_airbyte-3.6.0-py3-none-any.whl",
"apache-airflow-providers-airbyte-42.0.0-py3-none-any.whl",
# Invalid name"apache_airflow_providers_airbyte-42.0.0.whl",
]
forsdistinsdist_files:
package, _=parse_sdist_filename(sdist)
provider_id=inverted_mapping[package]
print(f"Provider ID: {provider_id}, sdist: {sdist}")
forwheelinwheel_files:
package, *_=parse_wheel_filename(wheel)
provider_id=inverted_mapping[package]
print(f"Provider ID: {provider_id}, sdist: {wheel}") |
potiuk
commented
Feb 14, 2024
Ah ok yes, could be :) . Feel free :) |
Follow up after #37406 - we could not test this change as there was no way to test canary builds but as the capability has been added in #37408 we can fix the failure and actually test it in a "canary" PR.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.