Skip to content

CI: try free up space in Rust / cargo test (amd64) action - #18709

Merged
Jefffrey merged 5 commits into
apache:mainfrom
Jefffrey:fix-ci-123
Nov 15, 2025
Merged

CI: try free up space in Rust / cargo test (amd64) action#18709
Jefffrey merged 5 commits into
apache:mainfrom
Jefffrey:fix-ci-123

Conversation

@Jefffrey

@JefffreyJefffrey commented Nov 14, 2025

Copy link
Copy Markdown
Contributor

Closes#18692 (hopefully)

Trying to get CI to pass consistently, try various techniques.

@github-actionsgithub-actionsBot added the development-process Related to development process of DataFusion label Nov 14, 2025
@Jefffrey

Jefffrey commented Nov 14, 2025

Copy link
Copy Markdown
ContributorAuthor

Oh not sure it (jlumbroso/free-disk-space) works if the action runs in a container 🤔

Edit: jlumbroso/free-disk-space#21

@JefffreyJefffrey changed the title Use jlumbroso/free-disk-space in CI setup-builder stepCI: try free up space in Rust / cargo test (amd64) actionNov 14, 2025
@Jefffrey

Copy link
Copy Markdown
ContributorAuthor

Taking inspiration from examples:

- name: Remove unnecessary preinstalled software
run: |
echo "Disk space before cleanup:"
df -h
apt-get clean
rm -rf /__t/CodeQL
rm -rf /__t/PyPy
rm -rf /__t/Java_Temurin-Hotspot_jdk
rm -rf /__t/Python
rm -rf /__t/go
rm -rf /__t/Ruby
echo "Disk space after cleanup:"
df -h

Which usually cleans about 5gb:

Run echo "Disk space before cleanup:"
Disk space before cleanup:
Filesystem Size Used Avail Use% Mounted on
overlay 72G 58G 14G 81% /
tmpfs 64M 0 64M 0% /dev
shm 64M 0 64M 0% /dev/shm
/dev/root 72G 58G 14G 81% /__t
tmpfs 3.2G 1.2M 3.2G 1% /run/docker.sock
tmpfs 7.9G 0 7.9G 0% /proc/acpi
tmpfs 7.9G 0 7.9G 0% /proc/scsi
tmpfs 7.9G 0 7.9G 0% /sys/firmware
Disk space after cleanup:
Filesystem Size Used Avail Use% Mounted on
overlay 72G 53G 19G 74% /
tmpfs 64M 0 64M 0% /dev
shm 64M 0 64M 0% /dev/shm
/dev/root 72G 53G 19G 74% /__t
tmpfs 3.2G 1.2M 3.2G 1% /run/docker.sock
tmpfs 7.9G 0 7.9G 0% /proc/acpi
tmpfs 7.9G 0 7.9G 0% /proc/scsi
tmpfs 7.9G 0 7.9G 0% /sys/firmware

But also adding what was suggested here: jlumbroso/free-disk-space#21

Combine together, seems to clean 24gb:

Run echo "Disk space before cleanup:"
Disk space before cleanup:
Filesystem Size Used Avail Use% Mounted on
overlay 72G 55G 18G 77% /
tmpfs 64M 0 64M 0% /dev
shm 64M 0 64M 0% /dev/shm
/dev/root 72G 55G 18G 77% /__t
tmpfs 3.2G 1.2M 3.2G 1% /run/docker.sock
tmpfs 7.9G 0 7.9G 0% /proc/acpi
tmpfs 7.9G 0 7.9G 0% /proc/scsi
tmpfs 7.9G 0 7.9G 0% /sys/firmware
Disk space after cleanup:
Filesystem Size Used Avail Use% Mounted on
overlay 72G 31G 42G 43% /
tmpfs 64M 0 64M 0% /dev
shm 64M 0 64M 0% /dev/shm
/dev/root 72G 31G 42G 43% /__t
tmpfs 3.2G 1.2M 3.2G 1% /run/docker.sock
tmpfs 7.9G 0 7.9G 0% /proc/acpi
tmpfs 7.9G 0 7.9G 0% /proc/scsi
tmpfs 7.9G 0 7.9G 0% /sys/firmware

🤯

@Jefffrey
Jefffrey marked this pull request as ready for review November 14, 2025 16:40

@2010YOUY012010YOUY01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the fix!

I suggest we merge it after CI passes, many recent PRs failed CI due to this reason.

@Jefffrey

Copy link
Copy Markdown
ContributorAuthor

Thank you for the fix!

I suggest we merge it after CI passes, many recent PRs failed CI due to this reason.

Thanks, looks like there's been 3 consecutively successful runs so the fix seems to work.

@Jefffrey
Jefffrey added this pull request to the merge queueNov 15, 2025
Merged via the queue into apache:main with commit 302dbc8Nov 15, 2025
28 checks passed
@Jefffrey
Jefffrey deleted the fix-ci-123 branch November 15, 2025 02:42
jizezhang pushed a commit to jizezhang/datafusion that referenced this pull request Nov 15, 2025
…8709)
Closesapache#18692 (hopefully)
Trying to get CI to pass consistently, try various techniques.
xudong963 pushed a commit to massive-com/arrow-datafusion that referenced this pull request Nov 17, 2025
…8709)
Closesapache#18692 (hopefully)
Trying to get CI to pass consistently, try various techniques.
logan-keede pushed a commit to logan-keede/datafusion that referenced this pull request Nov 23, 2025
…8709)
Closesapache#18692 (hopefully)
Trying to get CI to pass consistently, try various techniques.
alamb pushed a commit to alamb/datafusion that referenced this pull request Dec 1, 2025
…8709)
Closesapache#18692 (hopefully)
Trying to get CI to pass consistently, try various techniques.
xudong963 pushed a commit that referenced this pull request Dec 2, 2025
…on (#18709) (#19037)
## Which issue does this PR close?
- part of #18843 - Backports #18709 from
@Jefffrey
## Rationale for this change
@tobixdev is hitting a CI failure due to out of space on a different PR:
-
#19017 (comment)@Jefffrey fixed this on main with
- #18709 So let's backport that change to branch-51
## What changes are included in this PR?
- Backport #18709 to branch-51
## Are these changes tested?
by ci
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
## Are there any user-facing changes?
no
Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>
alamb pushed a commit to influxdata/arrow-datafusion that referenced this pull request Dec 3, 2025
…8709)
Closesapache#18692 (hopefully)
Trying to get CI to pass consistently, try various techniques.
erratic-pattern pushed a commit to influxdata/arrow-datafusion that referenced this pull request Dec 6, 2025
…8709)
Closesapache#18692 (hopefully)
Trying to get CI to pass consistently, try various techniques.
erratic-pattern pushed a commit to influxdata/arrow-datafusion that referenced this pull request Dec 26, 2025
…8709)
Closesapache#18692 (hopefully)
Trying to get CI to pass consistently, try various techniques.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-processRelated to development process of DataFusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI failing on main due to storage issues

2 participants

@Jefffrey@2010YOUY01