ci: publish each distribution from its own PyPI environment - #63
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Registering the PyPI pending trusted publishers failed for real.
nl2sql-engineregistered fine;
nl2sql-apiwas rejected with:PyPI keys a pending publisher on the tuple (owner, repository, workflow,
environment) and requires it to be unique. All three of our distributions
publish from the same owner, the same repository and the same
publish_pypi.yamlwith no environment — so all three tuples were identical and only the first
registration could succeed.
The environment is the only field left that can distinguish them.
What changed
publish_pypi.yaml's singlepypijob becomes a three-leg matrix, one leg perpackage, each in its own GitHub Environment:
nl2sql-enginepypi-nl2sql-enginenl2sql-apipypi-nl2sql-apinl2sql-adapter-sdkpypi-nl2sql-adapter-sdkA matrix rather than three copy-pasted jobs: GitHub evaluates expressions in
environment:, soenvironment: pypi-${{ matrix.package }}expresses the wholedifference between the legs in one line, and the publish steps stay identical.
pypa/gh-action-pypi-publishuploads everything inpackages-dir, so each legdownloads the shared
distartifact and copies only its own sdist + wheelinto
upload/before pointingpackages-dirat it. The trailing-in theglob is load-bearing —
nl2sql_a*would match bothnl2sql_apiandnl2sql_adapter_sdk. Verified against a locally builtdist/holding all sixfiles; each glob selects exactly two:
id-token: writestays on the publishing job, since OIDC is the entirecredential.
buildand its smoke gate are untouched: if the wheels do notinstall and import, nothing publishes.
The job name stays
pypi, soghcranddocskeepneeds: pypiunchanged —a dependency on a matrix job waits for every leg, so both still run only
after all three uploads succeed.
Action required before the first release
The pending publishers must be registered with these environment names. At
pypi.org/manage/account/publishing,
for each of the three project names, use owner
nadeem4, repositorynl2sql,workflow
publish_pypi.yaml, and the environment from the table above.Leaving Environment blank is what caused the rejection —
nl2sql-engine'sexisting entry was registered with no environment and needs to be recreated with
pypi-nl2sql-engine.The GitHub Environments themselves need no setup: referencing them in the
workflow is enough for GitHub to create them on the first run, after which they
appear under Settings → Environments (where reviewers or branch rules can be
added if the release should be gated further). None were created by this PR.
docs/development/releasing.mdis updated to match: the pending-publisher tablenow carries the Environment column, states the uniqueness constraint that forces
it, and no longer says to leave Environment empty.
Verification
yaml.safe_loadparsespublish_pypi.yaml; job graph as above.dist/(above).mkdocs build --strictclean (throwaway venv fromrequirements-docs.txt).CHANGELOG.mdtouched; release-please owns both.