Skip to content

[branch-50] chore: Fix no space left on device - #18141

Merged
alamb merged 7 commits into
apache:branch-50from
comphead:dev1
Oct 18, 2025
Merged

[branch-50] chore: Fix no space left on device#18141
alamb merged 7 commits into
apache:branch-50from
comphead:dev1

Conversation

@comphead

@compheadcomphead commented Oct 17, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actionsgithub-actionsBot added the development-process Related to development process of DataFusion label Oct 17, 2025
@alambalamb changed the title chore: Fix no space left on device[branch-50] chore: Fix no space left on deviceOct 17, 2025
@alamb

alamb commented Oct 17, 2025

Copy link
Copy Markdown
Contributor

Thanks @comphead

I took the liberty of pushing a commit to remove calling docker to try and fix the CI failure: https://github.com/apache/datafusion/actions/runs/18604053825/job/53049662273?pr=18141

Run echo "Disk space before cleanup:"
Disk space before cleanup:
Filesystem Size Used Avail Use% Mounted on
overlay 72G 54G 18G 75% /
tmpfs 64M 0 64M 0% /dev
shm 64M 0 64M 0% /dev/shm
/dev/root 72G 54G 18G 75% /__t
tmpfs 3.2G 1.2M 3.2G 1% /run/docker.sock
tmpfs 7.9G 0 7.9G 0% /proc/acpi
tmpfs 7.9G 0 7.9G 0% /proc/scsi
tmpfs 7.9G 0 7.9G 0% /sys/firmware
/__w/_temp/afee5de1-f9bf-44bd-96d5-94114efa6d4e.sh: 3: docker: not found

@alamb

alamb commented Oct 17, 2025

Copy link
Copy Markdown
Contributor

Note this PR also updates the datafusion-testing pin, probably incorrectly

@alamb

alamb commented Oct 17, 2025

Copy link
Copy Markdown
Contributor

I am going to see if I can find some other consumers of space via #18143

@comphead

comphead commented Oct 17, 2025

Copy link
Copy Markdown
ContributorAuthor

I am going to see if I can find some other consumers of space via #18143

Great, yep, having just 1 pipeline is time efficient

@alamb

alamb commented Oct 17, 2025

Copy link
Copy Markdown
Contributor

I am going to see if I can find some other consumers of space via #18143

Great, yep, having just 1 pipeline is time efficient

Here is some findings (as you say we can probably delete PyPy, Python, Go, Ruby and the CodeQL and get back several GB

742M	/__t/node
1.6G	/__t/CodeQL
520M	/__t/PyPy
24K	/__t/Java_Temurin-Hotspot_jdk
1.5G	/__t/Python
1.1G	/__t/go
218M	/__t/Ruby
5.5G	/__t
170M	/__e/node20
214M	/__e/node24
99M	/__e/node20_alpine
130M	/__e/node24_alpine
611M	/__e
92K	/__w/_temp
953M	/__w/datafusion
16K	/__w/_PipelineMapping
14M	/__w/_actions
966M	/__w

@comphead

comphead commented Oct 17, 2025

Copy link
Copy Markdown
ContributorAuthor

@alamb looks like this go to go, WDYT should we also backport it to main to avoid such failures for other releases?

alamb
alamb approved these changes Oct 18, 2025
View reviewed changes

@alambalamb left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @comphead !

@alamb looks like this go to go, WDYT should we also backport it to main to avoid such failures for other releases?

Yes, I will do so

Update: here is the PR

@alamb
alamb merged commit 28ad4ef into apache:branch-50Oct 18, 2025
29 checks passed
alamb added a commit to alamb/datafusion that referenced this pull request Oct 18, 2025
@comphead@alamb
[branch-50] chore: Fixno space left on device(apache#18141)
d3cd076
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closesapache#123` indicates that this PR will close issue apache#123.
-->
- Closesapache#18135 ## Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
This was referenced Oct 18, 2025
Merged
Merged
Merged
Merged
Merged
Merged
Closed
github-merge-queueBot pushed a commit that referenced this pull request Oct 18, 2025
@alamb@comphead
[main] chore: Fixno space left on device(#18141) (#18151)
28a6854
## Which issue does this PR close?
- related to #18135 - ## Rationale for this change
Our example job was failing due to out of space on other branches, so
let's remove some unecessary pre-installed software
## What changes are included in this PR?
- forward port changes from
#18141 to main
## Are these changes tested?
It is part of CI
## Are there any user-facing changes?
No
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
tschwarzinger pushed a commit to tschwarzinger/datafusion that referenced this pull request Nov 2, 2025
@alamb@comphead@tschwarzinger
[main] chore: Fixno space left on device(apache#18141) (apache#18151
07a9cf3
)
## Which issue does this PR close?
- related to apache#18135 - ## Rationale for this change
Our example job was failing due to out of space on other branches, so
let's remove some unecessary pre-installed software
## What changes are included in this PR?
- forward port changes from
apache#18141 to main
## Are these changes tested?
It is part of CI
## Are there any user-facing changes?
No
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
xudong963 pushed a commit to massive-com/arrow-datafusion that referenced this pull request Nov 14, 2025
@alamb@comphead@xudong963
[main] chore: Fixno space left on device(apache#18141) (apache#18151
6045c77
)
## Which issue does this PR close?
- related to apache#18135 - ## Rationale for this change
Our example job was failing due to out of space on other branches, so
let's remove some unecessary pre-installed software
## What changes are included in this PR?
- forward port changes from
apache#18141 to main
## Are these changes tested?
It is part of CI
## Are there any user-facing changes?
No
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
EeshanBembi pushed a commit to EeshanBembi/datafusion that referenced this pull request Nov 24, 2025
@alamb@comphead@EeshanBembi
[main] chore: Fixno space left on device(apache#18141) (apache#18151
6ea7e31
)
## Which issue does this PR close?
- related to apache#18135 - ## Rationale for this change
Our example job was failing due to out of space on other branches, so
let's remove some unecessary pre-installed software
## What changes are included in this PR?
- forward port changes from
apache#18141 to main
## Are these changes tested?
It is part of CI
## Are there any user-facing changes?
No
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alambalambalamb approved these changes

Assignees

No one assigned

Labels

development-processRelated to development process of DataFusion

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@comphead@alamb