Skip to content

ci: cache vcpkg packages and compile each target in parallel - #721

Merged
ddehilster merged 1 commit into
masterfrom
perf/ci-build-speed
Aug 25, 2026
Merged

ci: cache vcpkg packages and compile each target in parallel#721
ddehilster merged 1 commit into
masterfrom
perf/ci-build-speed

Conversation

@ddehilster

Copy link
Copy Markdown
Member

Where CI time actually goes

Step timings from the 15m43s Windows job on #718:

steptime
Install 3rd Party (vcpkg builds ICU)10m 04s
Cmake Build all targets3m 40s
checkout + generate solution~40s
all 14 regression tests~3 seconds

The tests are not what makes CI slow. Two thirds of it is vcpkg rebuilding ICU from source on every run, and most of the rest is the engine compiling one file at a time. Linux shows the same shape: 4m11s in vcpkg, 1m50s building.

1. vcpkg binary caching

vcpkg install ran with no binary source configured, so nothing was ever reused between runs. This points VCPKG_BINARY_SOURCES at a files-based cache under the workspace and persists it with actions/cache.

The key is the vcpkg submodule commit (which pins the ports, so it decides what the packages are) plus vcpkg.json (which decides which ones). restore-keys fall back to looser prefixes — that's safe because vcpkg verifies each package's ABI hash itself, so anything that doesn't match is simply rebuilt rather than used incorrectly.

Applied to all three platform workflows. The Linux key includes matrix.id so the three Ubuntu variants don't fight over one entry.

First run after merge is still a miss and pays full price; the saving starts on the second.

2. /MP on MSVC

cmake --build --parallel N with the Visual Studio generator becomes msbuild /m:N, which parallelises projects — so lite's 101 sources (Arun.cpp alone is ~14k lines) compiled serially no matter what --parallel said. Same defect fixed for generated analyzer code in 3.8.7, this time in the engine's own build.

Clean Win32 Release build, --parallel 4:

without /MP 189s
with /MP4 49s (/MP4 rather than /MP, to model a 4-core runner honestly
-- bare /MP on my 22-core box gave 40s and would have
overstated the CI gain)

3. Dropped a debug step

List $RUNNER_WORKSPACE before build recursively listed and printed the entire workspace: 8 seconds and a very large log, for no current purpose.

Notes

No version bump — this changes how the engine is built and tested, not what it does. The binary is unaffected.

The caching can only really be proven by merging and watching the second run, so I'd treat the first post-merge run as the test.

🤖 Generated with Claude Code

Breakdown of a 15m43s Windows CI job on #718:
Install 3rd Party (vcpkg builds ICU) 10m04s
Cmake Build all targets 3m40s
checkout + generate solution 40s
all 14 regression tests ~3s
The tests are not what makes CI slow. Two thirds of it is vcpkg
rebuilding ICU from source on every run, and most of the rest is the
engine compiling one file at a time.
1. vcpkg binary caching. `vcpkg install` ran with no binary source
configured, so nothing was reused between runs. Point
VCPKG_BINARY_SOURCES at a files-based cache under the workspace and
persist it with actions/cache, keyed on the vcpkg submodule commit
(which pins the ports) plus vcpkg.json (which picks them). vcpkg
verifies each package's ABI hash itself, so a partial restore-keys
hit is safe: whatever does not match is simply rebuilt. Measured cost
today: 10m04s on Windows, 4m11s on Linux, per job, per run.
2. /MP on MSVC. `cmake --build --parallel N` with the Visual Studio
generator becomes msbuild /m:N, which parallelises PROJECTS -- so
lite's 101 sources (Arun.cpp alone is ~14k lines) compiled serially
no matter what --parallel said. Same defect fixed for generated
analyzer code in 3.8.7, in the engine's own build. Clean Win32
Release build with --parallel 4:
without /MP 189s
with /MP4 49s (/MP4 to model a 4-core runner honestly)
Also drops the "List $RUNNER_WORKSPACE before build" step: it recursively
listed and printed the entire workspace, costing 8s and a very large log
for no current purpose.
No version bump: this changes how the engine is built and tested, not
what it does. The binary is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ddehilster
ddehilster merged commit 5a106ae into masterAug 25, 2026
7 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@ddehilster