Uh oh!
There was an error while loading. Please reload this page.
ci: cache vcpkg packages and compile each target in parallel - #721
Merged
Conversation
Breakdown of a 15m43s Windows CI job on #718: Install 3rd Party (vcpkg builds ICU) 10m04s Cmake Build all targets 3m40s checkout + generate solution 40s all 14 regression tests ~3s The tests are not what makes CI slow. Two thirds of it is vcpkg rebuilding ICU from source on every run, and most of the rest is the engine compiling one file at a time. 1. vcpkg binary caching. `vcpkg install` ran with no binary source configured, so nothing was reused between runs. Point VCPKG_BINARY_SOURCES at a files-based cache under the workspace and persist it with actions/cache, keyed on the vcpkg submodule commit (which pins the ports) plus vcpkg.json (which picks them). vcpkg verifies each package's ABI hash itself, so a partial restore-keys hit is safe: whatever does not match is simply rebuilt. Measured cost today: 10m04s on Windows, 4m11s on Linux, per job, per run. 2. /MP on MSVC. `cmake --build --parallel N` with the Visual Studio generator becomes msbuild /m:N, which parallelises PROJECTS -- so lite's 101 sources (Arun.cpp alone is ~14k lines) compiled serially no matter what --parallel said. Same defect fixed for generated analyzer code in 3.8.7, in the engine's own build. Clean Win32 Release build with --parallel 4: without /MP 189s with /MP4 49s (/MP4 to model a 4-core runner honestly) Also drops the "List $RUNNER_WORKSPACE before build" step: it recursively listed and printed the entire workspace, costing 8s and a very large log for no current purpose. No version bump: this changes how the engine is built and tested, not what it does. The binary is unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Where CI time actually goes
Step timings from the 15m43s Windows job on #718:
The tests are not what makes CI slow. Two thirds of it is vcpkg rebuilding ICU from source on every run, and most of the rest is the engine compiling one file at a time. Linux shows the same shape: 4m11s in vcpkg, 1m50s building.
1. vcpkg binary caching
vcpkg installran with no binary source configured, so nothing was ever reused between runs. This pointsVCPKG_BINARY_SOURCESat a files-based cache under the workspace and persists it withactions/cache.The key is the vcpkg submodule commit (which pins the ports, so it decides what the packages are) plus
vcpkg.json(which decides which ones).restore-keysfall back to looser prefixes — that's safe because vcpkg verifies each package's ABI hash itself, so anything that doesn't match is simply rebuilt rather than used incorrectly.Applied to all three platform workflows. The Linux key includes
matrix.idso the three Ubuntu variants don't fight over one entry.First run after merge is still a miss and pays full price; the saving starts on the second.
2.
/MPon MSVCcmake --build --parallel Nwith the Visual Studio generator becomesmsbuild /m:N, which parallelises projects — solite's 101 sources (Arun.cppalone is ~14k lines) compiled serially no matter what--parallelsaid. Same defect fixed for generated analyzer code in 3.8.7, this time in the engine's own build.Clean Win32 Release build,
--parallel 4:3. Dropped a debug step
List $RUNNER_WORKSPACE before buildrecursively listed and printed the entire workspace: 8 seconds and a very large log, for no current purpose.Notes
No version bump — this changes how the engine is built and tested, not what it does. The binary is unaffected.
The caching can only really be proven by merging and watching the second run, so I'd treat the first post-merge run as the test.
🤖 Generated with Claude Code