Skip to content

CI: Use ImageOS (ubuntu24) instead of runner.os (Linux) in cache keys#130200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hugovk merged 1 commit into from
Feb 17, 2025
Merged

CI: Use ImageOS (ubuntu24) instead of runner.os (Linux) in cache keys #130200

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 15 additions & 17 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -62,8 +62,6 @@ jobs:
with:
fetch-depth: 1
persist-credentials: false
- name: Runner image version
run: echo "IMAGE_VERSION=${ImageVersion}" >> "$GITHUB_ENV"
Comment on lines -65 to -66

@hugovkhugovkFeb 16, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was set but not used in this job.

- name: Check Autoconf and aclocal versions
run: |
grep "Generated by GNU Autoconf 2.72" configure
Expand DownExpand Up@@ -104,14 +102,14 @@ jobs:
with:
python-version: '3.x'
- name: Runner image version
run: echo "IMAGE_VERSION=${ImageVersion}" >> "$GITHUB_ENV"
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
- name: Restore config.cache
uses: actions/cache@v4
with:
path: config.cache
# Include env.pythonLocation in key to avoid changes in environment when setup-python updates Python
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ needs.build-context.outputs.config-hash }}-${{ env.pythonLocation }}
- name: Install Dependencies
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ needs.build-context.outputs.config-hash }}-${{ env.pythonLocation }}
- name: Install dependencies
run: sudo ./.github/workflows/posix-deps-apt.sh
- name: Add ccache to PATH
run: echo "PATH=/usr/lib/ccache:$PATH" >> "$GITHUB_ENV"
Expand DownExpand Up@@ -284,15 +282,15 @@ jobs:
with:
persist-credentials: false
- name: Runner image version
run: echo "IMAGE_VERSION=${ImageVersion}" >> "$GITHUB_ENV"
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
- name: Restore config.cache
uses: actions/cache@v4
with:
path: config.cache
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ needs.build-context.outputs.config-hash }}
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ needs.build-context.outputs.config-hash }}
- name: Register gcc problem matcher
run: echo "::add-matcher::.github/problem-matchers/gcc.json"
- name: Install Dependencies
- name: Install dependencies
run: sudo ./.github/workflows/posix-deps-apt.sh
- name: Configure OpenSSL env vars
run: |
Expand DownExpand Up@@ -347,7 +345,7 @@ jobs:
persist-credentials: false
- name: Register gcc problem matcher
run: echo "::add-matcher::.github/problem-matchers/gcc.json"
- name: Install Dependencies
- name: Install dependencies
run: sudo ./.github/workflows/posix-deps-apt.sh
- name: Configure OpenSSL env vars
run: |
Expand DownExpand Up@@ -379,12 +377,12 @@ jobs:
- name: Bind mount sources read-only
run: sudo mount --bind -o ro "$GITHUB_WORKSPACE" "$CPYTHON_RO_SRCDIR"
- name: Runner image version
run: echo "IMAGE_VERSION=${ImageVersion}" >> "$GITHUB_ENV"
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
- name: Restore config.cache
uses: actions/cache@v4
with:
path: ${{ env.CPYTHON_BUILDDIR }}/config.cache
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ needs.build-context.outputs.config-hash }}
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ needs.build-context.outputs.config-hash }}
- name: Configure CPython out-of-tree
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: |
Expand DownExpand Up@@ -466,15 +464,15 @@ jobs:
with:
persist-credentials: false
- name: Runner image version
run: echo "IMAGE_VERSION=${ImageVersion}" >> "$GITHUB_ENV"
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
- name: Restore config.cache
uses: actions/cache@v4
with:
path: config.cache
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ needs.build-context.outputs.config-hash }}
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ needs.build-context.outputs.config-hash }}
- name: Register gcc problem matcher
run: echo "::add-matcher::.github/problem-matchers/gcc.json"
- name: Install Dependencies
- name: Install dependencies
run: sudo ./.github/workflows/posix-deps-apt.sh
- name: Set up GCC-10 for ASAN
uses: egor-tensin/setup-gcc@v1
Expand DownExpand Up@@ -537,19 +535,19 @@ jobs:
with:
persist-credentials: false
- name: Runner image version
run: echo "IMAGE_VERSION=${ImageVersion}" >> "$GITHUB_ENV"
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
- name: Restore config.cache
uses: actions/cache@v4
with:
path: config.cache
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ needs.build-context.outputs.config-hash }}
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ needs.build-context.outputs.config-hash }}
- name: Register gcc problem matcher
run: echo "::add-matcher::.github/problem-matchers/gcc.json"
- name: Set build dir
run:
# an absolute path outside of the working directoy
echo "BUILD_DIR=$(realpath ${{ github.workspace }}/../build)" >> "$GITHUB_ENV"
- name: Install Dependencies
- name: Install dependencies
run: sudo ./.github/workflows/posix-deps-apt.sh
- name: Configure host build
run: ./configure --prefix="$BUILD_DIR/host-python"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/reusable-macos.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -35,12 +35,12 @@ jobs:
with:
persist-credentials: false
- name: Runner image version
run: echo "IMAGE_VERSION=${ImageVersion}" >> "$GITHUB_ENV"
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
- name: Restore config.cache
uses: actions/cache@v4
with:
path: config.cache
key: ${{ github.job }}-${{ inputs.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ inputs.config_hash }}
- name: Install Homebrew dependencies
run: |
brew install pkg-config openssl@3.0 xz gdbm tcl-tk@8 make
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/reusable-tsan.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -25,13 +25,13 @@ jobs:
with:
persist-credentials: false
- name: Runner image version
run: echo "IMAGE_VERSION=${ImageVersion}" >> "$GITHUB_ENV"
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
- name: Restore config.cache
uses: actions/cache@v4
with:
path: config.cache
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}
- name: Install Dependencies
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ inputs.config_hash }}
- name: Install dependencies
run: |
sudo ./.github/workflows/posix-deps-apt.sh
# Install clang-18
Expand All@@ -44,7 +44,7 @@ jobs:
sudo update-alternatives --set clang++ /usr/bin/clang++-17
# Reduce ASLR to avoid TSAN crashing
sudo sysctl -w vm.mmap_rnd_bits=28
- name: TSAN Option Setup
- name: TSAN option setup
run: |
echo "TSAN_OPTIONS=log_path=${GITHUB_WORKSPACE}/tsan_log suppressions=${GITHUB_WORKSPACE}/Tools/tsan/suppressions${{
fromJSON(inputs.free-threading)
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/reusable-ubuntu.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -78,12 +78,12 @@ jobs:
- name: Bind mount sources read-only
run: sudo mount --bind -o ro "$GITHUB_WORKSPACE" "$CPYTHON_RO_SRCDIR"
- name: Runner image version
run: echo "IMAGE_VERSION=${ImageVersion}" >> "$GITHUB_ENV"
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
- name: Restore config.cache
uses: actions/cache@v4
with:
path: ${{ env.CPYTHON_BUILDDIR }}/config.cache
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ inputs.config_hash }}
- name: Configure CPython out-of-tree
working-directory: ${{ env.CPYTHON_BUILDDIR }}
# `test_unpickle_module_race` writes to the source directory, which is
Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/reusable-wasi.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -53,14 +53,16 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: "Runner image version"
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
Comment on lines +56 to +57

@hugovkhugovkFeb 16, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wasn't being set in this job, so env.IMAGE_VERSION was evaluating as an empty string in the keys below.

- name: "Restore Python build config.cache"
uses: actions/cache@v4
with:
path: ${{ env.CROSS_BUILD_PYTHON }}/config.cache
# Include env.pythonLocation in key to avoid changes in environment when setup-python updates Python.
# Include the hash of `Tools/wasm/wasi.py` as it may change the environment variables.
# (Make sure to keep the key in sync with the other config.cache step below.)
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ env.WASI_SDK_VERSION }}-${{ env.WASMTIME_VERSION }}-${{ inputs.config_hash }}-${{ hashFiles('Tools/wasm/wasi.py') }}-${{ env.pythonLocation }}
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ env.WASI_SDK_VERSION }}-${{ env.WASMTIME_VERSION }}-${{ inputs.config_hash }}-${{ hashFiles('Tools/wasm/wasi.py') }}-${{ env.pythonLocation }}
- name: "Configure build Python"
run: python3 Tools/wasm/wasi.py configure-build-python -- --config-cache --with-pydebug
- name: "Make build Python"
Expand All@@ -70,7 +72,7 @@ jobs:
with:
path: ${{ env.CROSS_BUILD_WASI }}/config.cache
# Should be kept in sync with the other config.cache step above.
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ env.WASI_SDK_VERSION }}-${{ env.WASMTIME_VERSION }}-${{ inputs.config_hash }}-${{ hashFiles('Tools/wasm/wasi.py') }}-${{ env.pythonLocation }}
key: ${{ github.job }}-${{ env.IMAGE_OS_VERSION }}-${{ env.WASI_SDK_VERSION }}-${{ env.WASMTIME_VERSION }}-${{ inputs.config_hash }}-${{ hashFiles('Tools/wasm/wasi.py') }}-${{ env.pythonLocation }}
- name: "Configure host"
# `--with-pydebug` inferred from configure-build-python
run: python3 Tools/wasm/wasi.py configure-host -- --config-cache
Expand Down