feat: Build improvements - #142
Conversation
e2d040c to
5fad96b
Compare
…ting ~/.cache/zig. Add an option to write a slimmer manifest.
5fad96b to
00a15a5
Compare
There was a problem hiding this comment.
Pull request overview
Improves the Selene build pipeline by (1) scoping Zig’s cache to a configurable directory to avoid polluting users’ global Zig cache, and (2) reducing the size/time cost of writing Selene build manifests by optionally emitting a “complete” manifest.
Changes:
- Add
complete_manifesttoselene_sim.build.build()to optionally omit intermediate artifacts from the written manifest. - Extend
invoke_zig()to accept acache_dirand propagate it viaZIG_LOCAL_CACHE_DIR; wire this through Helios/Selene builtin build steps. - Adjust wheel build CI configuration (cibuildwheel rust setup steps; GitHub Actions cache restore keys).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| selene-sim/python/selene_sim/build.py | Adds complete_manifest and conditionally trims manifest artifact entries. |
| selene-core/python/selene_core/build_utils/utils.py | Adds cache_dir support to Zig invocation via environment. |
| selene-core/python/selene_core/build_utils/builtins/selene.py | Passes Zig cache dir during final link step. |
| selene-core/python/selene_core/build_utils/builtins/helios.py | Passes Zig cache dir during compile/link steps across platforms. |
| pyproject.toml | Updates cibuildwheel rust setup steps (currently with an invalid rustup toolchain install invocation). |
| .github/workflows/build_wheels.yml | Enables cache fallback via restore-keys for build environment caching. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| handle = subprocess.Popen( | ||
| argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env | ||
| ) | ||
| stdout, stderr = handle.communicate() |
There was a problem hiding this comment.
invoke_zig now captures stdout/stderr via Popen(..., stdout=PIPE, stderr=PIPE) but does not stream output to the console even when verbose=True, and all stdout is silently swallowed by callers (nobody uses the returned string). Consider keeping the previous behavior (inherit stdout; capture stderr for errors) or adding an explicit capture_output/stream_output option so builds don’t become opaque and long-running commands still show progress when desired.
There was a problem hiding this comment.
@copilot apply changes based on this feedback
There was a problem hiding this comment.
Fixed in commit f6edf63. invoke_zig now inherits stdout (streams to the console) when verbose=True, and suppresses it via DEVNULL when verbose=False. stderr is always captured so it can be included in any RuntimeError. The return type is now None since no callers used the returned string.
| input_artifact.resource, | ||
| selene_lib, | ||
| *link_flags, | ||
| cache_dir=build_ctx.artifact_dir, |
There was a problem hiding this comment.
cache_dir is set to build_ctx.artifact_dir, which will mix Zig’s cache contents with the user-visible build artifacts (e.g., program.helios.o / program.selene.x) and can bloat that directory unexpectedly. Consider using a dedicated subdirectory (e.g., artifact_dir / "zig-cache") and ensure it exists before invoking Zig.
There was a problem hiding this comment.
@copilot apply changes based on this feedback
There was a problem hiding this comment.
Fixed in commit f6edf63. All five invoke_zig call sites in helios.py and selene.py now use artifact_dir / "zig-cache" as the dedicated cache subdirectory, with mkdir(exist_ok=True) called before each invocation to ensure it exists.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Quantinuum/selene/sessions/579881d4-5abf-492a-8309-39432766ce16 Co-authored-by: jake-arkinstall <65358059+jake-arkinstall@users.noreply.github.com>
…tory Agent-Logs-Url: https://github.com/Quantinuum/selene/sessions/2021ef0b-74b1-41f0-93ab-a39ce2004503 Co-authored-by: jake-arkinstall <65358059+jake-arkinstall@users.noreply.github.com>
🤖 I have created a release *beep* *boop* --- ## [0.2.7](selene-core-v0.2.6...selene-core-v0.2.7) (2026-04-10) ### Features * Build improvements ([#142](#142)) ([12f399b](12f399b)) * test on QIS instead of relying on the upper stack ([#150](#150)) ([b80a9c4](b80a9c4)) * Use mingw instead of msvc for windows wheels ([#143](#143)) ([3d91514](3d91514)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [0.2.13](selene-sim-v0.2.12...selene-sim-v0.2.13) (2026-04-10) ### Features * Build improvements ([#142](#142)) ([12f399b](12f399b)) * test on QIS instead of relying on the upper stack ([#150](#150)) ([b80a9c4](b80a9c4)) * Use mingw instead of msvc for windows wheels ([#143](#143)) ([3d91514](3d91514)) ### Documentation * add repository citation guidance and metadata ([#146](#146)) ([880a58b](880a58b)), closes [#145](#145) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [0.3.0-alpha.1](selene-core-v0.3.0-alpha.0...selene-core-v0.3.0-alpha.1) (2026-06-12) ### ⚠ BREAKING CHANGES * Use struct exports for all plugin types, separate error model and simulator ([#169](#169)) * Add handling for an additional gateset ([#119](#119)) ### Features * Add __version__ attributes ([#137](#137)) ([379ae01](379ae01)) * Add handling for an additional gateset ([#119](#119)) ([5180b80](5180b80)) * Add simulate_delay functionality ([#139](#139)) ([cca97fa](cca97fa)) * Add support for object files provided as bytes ([#94](#94)) ([c4cfac6](c4cfac6)) * add timing to builtin runtimes and batching options to softrz runtime ([#158](#158)) ([049e123](049e123)) * Build improvements ([#142](#142)) ([12f399b](12f399b)) * Interactive use of Selene from python ([#135](#135)) ([db3028d](db3028d)) * Make interfaces shared, add ArgReader ([#171](#171)) ([143e742](143e742)) * QIR support using QIR-QIS ([#114](#114)) ([70ab294](70ab294)) * Result stream handling refactor ([#93](#93)) ([607a55e](607a55e)) * Stim improvements (more ops + state printing) ([#115](#115)) ([867d5e5](867d5e5)) * test on QIS instead of relying on the upper stack ([#150](#150)) ([b80a9c4](b80a9c4)) * Traces for analytics ([#160](#160)) ([24b9978](24b9978)) * Use mingw instead of msvc for windows wheels ([#143](#143)) ([3d91514](3d91514)) * Use struct exports for all plugin types, separate error model and simulator ([#169](#169)) ([c44a5c9](c44a5c9)) ### Bug Fixes * add `___barrier` to Helios QIS for QIR emulation ([#136](#136)) ([dfbc4c3](dfbc4c3)) * avoid using qir_major_version string for QIR detection ([#123](#123)) ([deaa0dc](deaa0dc)) * classify lowered qir-qis bitcode as helios ([#157](#157)) ([67fca60](67fca60)) * make qir-qis optional and stabilize CI ([#164](#164)) ([e52c0b2](e52c0b2)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [0.3.0-alpha.1](selene-sim-v0.3.0-alpha.0...selene-sim-v0.3.0-alpha.1) (2026-06-12) ### ⚠ BREAKING CHANGES * Use struct exports for all plugin types, separate error model and simulator ([#169](#169)) * Add handling for an additional gateset ([#119](#119)) ### Features * Add __version__ attributes ([#137](#137)) ([379ae01](379ae01)) * Add event hook which records measurement results ([#104](#104)) ([01300ee](01300ee)), closes [#103](#103) * Add handling for an additional gateset ([#119](#119)) ([5180b80](5180b80)) * Add simulate_delay functionality ([#139](#139)) ([cca97fa](cca97fa)) * Add support for object files provided as bytes ([#94](#94)) ([c4cfac6](c4cfac6)) * add timing to builtin runtimes and batching options to softrz runtime ([#158](#158)) ([049e123](049e123)) * Better exception handling for parse_shots=False ([#70](#70)) ([3caf530](3caf530)) * Build improvements ([#142](#142)) ([12f399b](12f399b)) * Bump selene-core dependency and relax guppylang testing dependency ([#120](#120)) ([6375791](6375791)) * Cleanup error'd processes before log collection ([#98](#98)) ([77e698e](77e698e)) * **compiler:** Bump tket version; add wasm + gpu to the hugr-qis registry ([c69155d](c69155d)) * correct shot end strategy and error processing ([#91](#91)) ([93eaeb0](93eaeb0)) * Emit a nicer error when trying to emulate unsupported pytket ops ([#72](#72)) ([d88a28a](d88a28a)) * Fine-grained timeout configuration ([#69](#69)) ([072842e](072842e)) * Interactive use of Selene from python ([#135](#135)) ([db3028d](db3028d)) * Make interfaces shared, add ArgReader ([#171](#171)) ([143e742](143e742)) * QIR support using QIR-QIS ([#114](#114)) ([70ab294](70ab294)) * Quest performance improvements ([#133](#133)) ([af640e9](af640e9)) * random_advance ([#55](#55)) ([974b496](974b496)) * Record simulated delays in event hooks ([#154](#154)) ([608884a](608884a)) * Result stream handling refactor ([#93](#93)) ([607a55e](607a55e)) * Stim improvements (more ops + state printing) ([#115](#115)) ([867d5e5](867d5e5)) * Support state-dump passthrough on quantum replay simulator ([#108](#108)) ([1b01a01](1b01a01)) * test on QIS instead of relying on the upper stack ([#150](#150)) ([b80a9c4](b80a9c4)) * Test on QIS snapshots ([#134](#134)) ([c4d58ab](c4d58ab)) * Traces for analytics ([#160](#160)) ([24b9978](24b9978)) * update to tket-qsystem 0.20 ([#66](#66)) ([7191b07](7191b07)) * Use mingw instead of msvc for windows wheels ([#143](#143)) ([3d91514](3d91514)) * Use SeleneStartupError for errors before shot_start ([#170](#170)) ([85c2907](85c2907)) * Use struct exports for all plugin types, separate error model and simulator ([#169](#169)) ([c44a5c9](c44a5c9)) ### Bug Fixes * add `___barrier` to Helios QIS for QIR emulation ([#136](#136)) ([dfbc4c3](dfbc4c3)) * Add ENDING to allowed shot state on receiving meta information ([#100](#100)) ([fc4d673](fc4d673)) * Add numpy dependency back ([#127](#127)) ([c9a01eb](c9a01eb)) * avoid using qir_major_version string for QIR detection ([#123](#123)) ([deaa0dc](deaa0dc)) * classify lowered qir-qis bitcode as helios ([#157](#157)) ([67fca60](67fca60)) * **compiler:** error when entrypoint has arguments ([#84](#84)) ([604b131](604b131)) * **compiler:** update tket-qystem to fix CZ bug ([#78](#78)) ([3991f11](3991f11)) * correct post_runtime duration metric ([#74](#74)) ([0bef66a](0bef66a)) * make qir-qis optional and stabilize CI ([#164](#164)) ([e52c0b2](e52c0b2)) ### Documentation * add repository citation guidance and metadata ([#146](#146)) ([880a58b](880a58b)), closes [#145](#145) * Correct URL to guppy repository ([#121](#121)) ([27ded00](27ded00)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Jake Arkinstall <65358059+jake-arkinstall@users.noreply.github.com>
Addresses two recent feedback items:
The first item is managed by providing zig with a cache dir, which subsequently sets
ZIG_LOCAL_CACHE_DIRin the subprocess environment.The second is managed by providing a
complete_manifestbool flag tobuild(). When True, the manifest aims to be a complete picture of the build stages, for use with sharing and diagnosis. When False (default), less information is written to the manifest. At current this reduced information is in the form of stripping input artifact or intermediate artifacts in the manifest, keeping only the final binary.