Uh oh!
There was an error while loading. Please reload this page.
Add Windows x86_64 Wheels to Release Workflow - #109
Conversation
Following what is done by componentize-py more closely, build and store the artifact for our source-built CPython targeting WASI once on linux. Then place this CPython in a place it will be picked up for platform-specific wheel builds. This should reduce the total overall CI work, reduce complexity, and enable us to support Windows where there are issues with tooling to build CPythong from source for WASI (that we would rather not have to try to solve). It _might_ be possible to reduce CI times further in the future by caching this CPython build, but for now keeping it simple.
`cargo check` will run build.rs (which builds CPython) but hopefully will reduce other compilation work that we subsequently discard.
We don't need these and they cause problems on Windows; just exclude them when creating the archive.
We had diverging ways of doing this path lookup, converting the path to posix form, etc. There was also a bug with the path lookup in the manylinux container. Make this consistent by having extracting the functionality to a small helper script based around cargo metadata's json output.
erikrose
left a comment
There was a problem hiding this comment.
Hooray! Want to push a 0.1.2 pretty soon? I hear some people use Windows.
| .github/scripts/setup-nightly.sh ${{ env.RUST_NIGHTLY }} | ||
| rustup target add wasm32-unknown-unknown | ||
| cargo fetch | ||
| MANIFEST_DIR=$(cargo metadata --format-version=1 | jq -r '.packages[] | select(.name == "componentize-py") | .manifest_path' | xargs dirname) |
There was a problem hiding this comment.
Took me a minute to guess what's going on here. Are you sticking the previously built CPython-for-wasi into the build dir so the Linux build doesn't rebuild it? (Is it smart/dumb enough not to rebuild it?)
There was a problem hiding this comment.
Yep, that's what's going on. Primarily motivated by Windows not being able to build CPython at all presently (but should be reused for all of the second-pass builds). This is the model used by componentize-py to get windows working and what I ended up adopting here.
Uh oh!
There was an error while loading. Please reload this page.
Building CPython for WASI presents a bunch of issues; I ended up following what is done by componentize-py by building CPython target WASI once on Linux and doing the delicate dance to reuse that artifact when rounding out the platform-specific wheel.
I was hoping this might reduce CI times as we avoid having to build CPython more than once; while that is the case, we end up having to rebuild chunks of the dependency graph in both stages, so CI time is increased. Eventually, I think upstream componentize-py might look to reuse prebuilt CPython releases for WASI, but that will require some more work and belongs in upstream.
https://github.com/fastly/compute-sdk-python/actions/runs/26839132628 shows the successful run. I tweaked the triggers on the workflow a bit to make it easier to do manual dispatches. CI will also run the release workflow on PRs that modify the release.yml file as well (but only on tag will we create a pre-release).