Skip to content

[SPARK-55115][INFRA] Use master branch's base Dockerfile for release builds - #53890

Closed
cloud-fan wants to merge 1 commit into
apache:masterfrom
cloud-fan:use-master-dockerfile-for-release
Closed

[SPARK-55115][INFRA] Use master branch's base Dockerfile for release builds#53890
cloud-fan wants to merge 1 commit into
apache:masterfrom
cloud-fan:use-master-dockerfile-for-release

Conversation

@cloud-fan

@cloud-fancloud-fan commented Jan 21, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Modified the release GitHub Actions workflow to always use master branch's Dockerfile.base when building the release Docker image, instead of using the Dockerfile.base from the release branch.

Why are the changes needed?

Old branch Dockerfiles can become unmaintainable over time due to:

  • Expired GPG keys (e.g., Node.js 12 repository keys)
  • Outdated base images with broken package dependencies
  • Package version conflicts with aging OS versions (e.g., Ubuntu 20.04)

The master branch's Dockerfile.base is actively maintained and uses modern base images (Ubuntu 22.04), making release builds more reliable. Only the base Dockerfile is pulled from master, while the main Dockerfile remains from the release branch to preserve any release-specific configurations.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Tested locally for all the active branches with all the release steps (except for uploading)

Was this patch authored or co-authored using generative AI tooling?

Yes. cursor 2.3.41

@github-actions

github-actionsBot commented Jan 21, 2026

Copy link
Copy Markdown

JIRA Issue Information

=== Improvement SPARK-55115 ===
Summary: Use master branch's Dockerfile for release builds
Assignee: Wenchen Fan
Status: Resolved
Affected: ["4.2.0"]


This comment was automatically generated by GitHub Actions

@cloud-fan
cloud-fan marked this pull request as draft January 21, 2026 18:34
Comment thread.github/workflows/release.yml Outdated
# Old branch Dockerfiles may have broken dependencies (expired GPG keys, outdated base images, etc.)
run: |
git fetch origin master --depth=1
git checkout origin/master -- dev/create-release/spark-rm/

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I think we shoudn't? the dependencies might make the build fail. e.g., python requires some upper/lower bound depdencnies

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update it to only use the base docker file later.

@cloud-fan
cloud-fanforce-pushed the use-master-dockerfile-for-release branch from 1c4b6d2 to bbb79f8CompareJanuary 23, 2026 09:46
@cloud-fancloud-fan changed the title [SPARK-55115][INFRA] Use master branch's Dockerfile for release builds[SPARK-55115][INFRA] Use master branch's base Dockerfile for release buildsJan 23, 2026
@cloud-fan
cloud-fan marked this pull request as ready for review January 23, 2026 09:47
@cloud-fan

Copy link
Copy Markdown
ContributorAuthor

release infra change, no need to wait for CI. Thanks for the review, merging to master!

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@cloud-fan@HyukjinKwon