Is your feature request related to a problem or challenge?
We've faced issues with storage space on our CI runners, so we introduced steps to free up space for them across various actions:
| # Run extended tests (with feature 'extended_tests') |
| linux-test-extended: |
| name: cargo test 'extended_tests' (amd64) |
| needs: [linux-build-lib] |
| runs-on: ubuntu-latest |
| # note: do not use amd/rust container to preserve disk space |
| steps: |
| - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 |
| with: |
| ref: ${{ github.event.inputs.pr_head_sha }} # will be empty if triggered by push |
| submodules: true |
| fetch-depth: 1 |
| - name: Free Disk Space (Ubuntu) |
| uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1 |
| - name: Install Rust |
| run: | |
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y |
| source $HOME/.cargo/env |
| rustup toolchain install |
| - name: Install Protobuf Compiler |
| run: sudo apt-get install -y protobuf-compiler |
| # For debugging, test binaries can be large. |
| - name: Show available disk space |
| run: | |
| df -h |
| # Library and integration tests |
| linux-test: |
| name: cargo test (amd64) |
| needs: linux-build-lib |
| runs-on: ubuntu-latest |
| container: |
| image: amd64/rust |
| volumes: |
| - /usr/local:/host/usr/local |
| steps: |
| - name: Remove unnecessary preinstalled software |
| run: | |
| echo "Disk space before cleanup:" |
| df -h |
| # remove tool cache: about 8.5GB (github has host /opt/hostedtoolcache mounted as /__t) |
| rm -rf /__t/* || true |
| # remove Haskell runtime: about 6.3GB (host /usr/local/.ghcup) |
| rm -rf /host/usr/local/.ghcup || true |
| # remove Android library: about 7.8GB (host /usr/local/lib/android) |
| rm -rf /host/usr/local/lib/android || true |
| echo "Disk space after cleanup:" |
| df -h |
| linux-test-example: |
| name: cargo examples (amd64) |
| needs: linux-build-lib |
| runs-on: ubuntu-latest |
| container: |
| image: amd64/rust |
| steps: |
| - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 |
| with: |
| submodules: true |
| fetch-depth: 1 |
| - name: Setup Rust toolchain |
| uses: ./.github/actions/setup-builder |
| with: |
| rust-version: stable |
| - name: Rust Dependency Cache |
| uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1 |
| with: |
| save-if: ${{ github.ref_name == 'main' }} |
| shared-key: "amd-ci-linux-test-example" |
| - name: Remove unnecessary preinstalled software |
| run: | |
| echo "Disk space before cleanup:" |
| df -h |
| apt-get clean |
| rm -rf /__t/CodeQL |
| rm -rf /__t/PyPy |
| rm -rf /__t/Java_Temurin-Hotspot_jdk |
| rm -rf /__t/Python |
| rm -rf /__t/go |
| rm -rf /__t/Ruby |
| echo "Disk space after cleanup:" |
| df -h |
Describe the solution you'd like
We should look into consolidating them together if possible so can apply to all our runners/actions.
Need to consider that some actions are on bare machine runners and others are on containers.
Describe alternatives you've considered
No response
Additional context
See #18709
Is your feature request related to a problem or challenge?
We've faced issues with storage space on our CI runners, so we introduced steps to free up space for them across various actions:
datafusion/.github/workflows/extended.yml
Lines 89 to 113 in 302dbc8
datafusion/.github/workflows/rust.yml
Lines 267 to 288 in 302dbc8
datafusion/.github/workflows/rust.yml
Lines 359 to 391 in 302dbc8
Describe the solution you'd like
We should look into consolidating them together if possible so can apply to all our runners/actions.
Need to consider that some actions are on bare machine runners and others are on containers.
Describe alternatives you've considered
No response
Additional context
See #18709