[SPARK-58871][INFRA] Add timeout to disk cleanup steps - #58121
Closed
zhengruifeng wants to merge 1 commit into
Closed
[SPARK-58871][INFRA] Add timeout to disk cleanup steps#58121zhengruifeng wants to merge 1 commit into
zhengruifeng wants to merge 1 commit into
Conversation
zhengruifeng
requested review from
HyukjinKwon,
LuciferYang,
dongjoon-hyun and
gaogaotiantian
August 19, 2026 16:15
LuciferYang
approved these changes
Aug 19, 2026
dongjoon-hyun
approved these changes
Aug 19, 2026
uros-b
approved these changes
Aug 19, 2026
HyukjinKwon
approved these changes
Aug 19, 2026
zhengruifeng
added a commit
that referenced
this pull request
Aug 20, 2026
### What changes were proposed in this pull request? This PR adds a 10-minute timeout and `continue-on-error: true` to the `Free up disk space` steps in the GitHub Actions build workflow. JIRA: SPARK-58871 ### Why are the changes needed? The disk cleanup step is an optimization. If it hangs, the job can spend its entire timeout before reaching setup or tests. Continuing after the cleanup step times out lets CI proceed with a higher disk space risk instead of cancelling the whole job during cleanup. For example, this CI job was cancelled while running the disk cleanup step: https://github.com/zhengruifeng/spark/actions/runs/32230206211/job/96008694804 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Not run. This is a GitHub Actions workflow-only change. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) Closes #58121 from zhengruifeng/minor-free-disk-space-timeout. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com> (cherry picked from commit d8403fe) Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
zhengruifeng
added a commit
that referenced
this pull request
Aug 20, 2026
### What changes were proposed in this pull request? This PR adds a 10-minute timeout and `continue-on-error: true` to the `Free up disk space` steps in the GitHub Actions build workflow. JIRA: SPARK-58871 ### Why are the changes needed? The disk cleanup step is an optimization. If it hangs, the job can spend its entire timeout before reaching setup or tests. Continuing after the cleanup step times out lets CI proceed with a higher disk space risk instead of cancelling the whole job during cleanup. For example, this CI job was cancelled while running the disk cleanup step: https://github.com/zhengruifeng/spark/actions/runs/32230206211/job/96008694804 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Not run. This is a GitHub Actions workflow-only change. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) Closes #58121 from zhengruifeng/minor-free-disk-space-timeout. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com> (cherry picked from commit d8403fe) Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
zhengruifeng
added a commit
that referenced
this pull request
Aug 20, 2026
### What changes were proposed in this pull request? This PR adds a 10-minute timeout and `continue-on-error: true` to the `Free up disk space` steps in the GitHub Actions build workflow. JIRA: SPARK-58871 ### Why are the changes needed? The disk cleanup step is an optimization. If it hangs, the job can spend its entire timeout before reaching setup or tests. Continuing after the cleanup step times out lets CI proceed with a higher disk space risk instead of cancelling the whole job during cleanup. For example, this CI job was cancelled while running the disk cleanup step: https://github.com/zhengruifeng/spark/actions/runs/32230206211/job/96008694804 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Not run. This is a GitHub Actions workflow-only change. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) Closes #58121 from zhengruifeng/minor-free-disk-space-timeout. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com> (cherry picked from commit d8403fe) Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
zhengruifeng
added a commit
that referenced
this pull request
Aug 20, 2026
### What changes were proposed in this pull request? This PR adds a 10-minute timeout and `continue-on-error: true` to the `Free up disk space` steps in the GitHub Actions build workflow. JIRA: SPARK-58871 ### Why are the changes needed? The disk cleanup step is an optimization. If it hangs, the job can spend its entire timeout before reaching setup or tests. Continuing after the cleanup step times out lets CI proceed with a higher disk space risk instead of cancelling the whole job during cleanup. For example, this CI job was cancelled while running the disk cleanup step: https://github.com/zhengruifeng/spark/actions/runs/32230206211/job/96008694804 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Not run. This is a GitHub Actions workflow-only change. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) Closes #58121 from zhengruifeng/minor-free-disk-space-timeout. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com> (cherry picked from commit d8403fe) Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
zhengruifeng
added a commit
that referenced
this pull request
Aug 20, 2026
### What changes were proposed in this pull request? This PR adds a 10-minute timeout and `continue-on-error: true` to the `Free up disk space` steps in the GitHub Actions build workflow. JIRA: SPARK-58871 ### Why are the changes needed? The disk cleanup step is an optimization. If it hangs, the job can spend its entire timeout before reaching setup or tests. Continuing after the cleanup step times out lets CI proceed with a higher disk space risk instead of cancelling the whole job during cleanup. For example, this CI job was cancelled while running the disk cleanup step: https://github.com/zhengruifeng/spark/actions/runs/32230206211/job/96008694804 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Not run. This is a GitHub Actions workflow-only change. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Codex (GPT-5) Closes #58121 from zhengruifeng/minor-free-disk-space-timeout. Authored-by: Ruifeng Zheng <ruifengz@apache.org> Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com> (cherry picked from commit d8403fe) Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
Contributor
Author
nchammas
added a commit
that referenced
this pull request
Aug 25, 2026
### What changes were proposed in this pull request? Tolerate failures from `apt-get` remove directly. Refactor the list of packages to remove into an array. Remove the call to `apt-get update`. ### Why are the changes needed? #53226 added the call to `update` specifically to fix intermittent failures when calling `remove`. That `update`, however, had its own issues and hit up against actions/runner-images#14594. This in turn triggered the addition of a timeout to the cleanup in #58121. If we tolerate `remove` failures directly, we don't need the `update` anymore. I don't think we need the timeout either, but I've left that alone. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? CI on this PR. ### Was this patch authored or co-authored using generative AI tooling? I wrote this with assistance from GitHub Copilot. Closes #58206 from nchammas/free-disk-space-cleanup. Authored-by: Nicholas Chammas <nicholas.chammas@gmail.com> Signed-off-by: Nicholas Chammas <nicholas.chammas@gmail.com>
nchammas
added a commit
that referenced
this pull request
Aug 25, 2026
### What changes were proposed in this pull request? Tolerate failures from `apt-get` remove directly. Refactor the list of packages to remove into an array. Remove the call to `apt-get update`. ### Why are the changes needed? #53226 added the call to `update` specifically to fix intermittent failures when calling `remove`. That `update`, however, had its own issues and hit up against actions/runner-images#14594. This in turn triggered the addition of a timeout to the cleanup in #58121. If we tolerate `remove` failures directly, we don't need the `update` anymore. I don't think we need the timeout either, but I've left that alone. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? CI on this PR. ### Was this patch authored or co-authored using generative AI tooling? I wrote this with assistance from GitHub Copilot. Closes #58206 from nchammas/free-disk-space-cleanup. Authored-by: Nicholas Chammas <nicholas.chammas@gmail.com> Signed-off-by: Nicholas Chammas <nicholas.chammas@gmail.com> (cherry picked from commit 6f89924) Signed-off-by: Nicholas Chammas <nicholas.chammas@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR adds a 10-minute timeout and
continue-on-error: trueto theFree up disk spacesteps in the GitHub Actions build workflow.JIRA: SPARK-58871
Why are the changes needed?
The disk cleanup step is an optimization. If it hangs, the job can spend its
entire timeout before reaching setup or tests. Continuing after the cleanup
step times out lets CI proceed with a higher disk space risk instead of
cancelling the whole job during cleanup.
For example, this CI job was cancelled while running the disk cleanup step:
https://github.com/zhengruifeng/spark/actions/runs/32230206211/job/96008694804
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Not run. This is a GitHub Actions workflow-only change.
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)