Skip to content

[SPARK-59086][SPARK-54466][BUILD] Extract common build env setup into composite action - #58334

Open
nchammas wants to merge 9 commits into
apache:masterfrom
nchammas:setup-build-env
Open

[SPARK-59086][SPARK-54466][BUILD] Extract common build env setup into composite action#58334
nchammas wants to merge 9 commits into
apache:masterfrom
nchammas:setup-build-env

Conversation

@nchammas

@nchammasnchammas commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

  1. Extract the repeated build environment setup across several workflows into a common composite action.
  2. Collapse the various cache keys used into a small set in the composite action.
  3. Remove the various skips and comments related to macOS.

Related: #57187

Why are the changes needed?

  1. We should capture repeated workflow steps in a composite action to make them easier to maintain over time and to prevent unintentional drift across the various copies of the steps.
  2. I intentionally collapsed the various cache keys because I didn't see that they were serving a purpose. If we find over time that there are cache misses costing us build time, we can update the composite action and, if necessary, parametrize some of the cache keys. Until then, however, I erred on the side of simplicity.
  3. The original issue that triggered the skip of macOS has since been resolved. I also manually ran the affected workflows and confirmed they complete successfully on macOS. (build_maven_java21_macos26.yml, build_python_3.12_macos26.yml)

Does this PR introduce any user-facing change?

No.

How was this patch tested?

PR CI:

As far as the runtimes go, I don't see any notable differences for any individual job. (Disregard the overall graph runtime since it includes waiting for runners to pick up the jobs.)

For the scheduled workflows that don't run as part of PR CI tests, I manually edited them to run on my fork:

They both ran successfully. (The failure on the extended SQL tests in first run is unrelated to this PR.)

Was this patch authored or co-authored using generative AI tooling?

I wrote this with assistance from GitHub Copilot.

@nchammasnchammas changed the title [BUILD] Extract common build env setup into composite action[SPARK-59086][BUILD] Extract common build env setup into composite actionAug 28, 2026
@nchammasnchammas changed the title [SPARK-59086][BUILD] Extract common build env setup into composite action[SPARK-59086][SPARK-54466][BUILD] Extract common build env setup into composite actionAug 29, 2026
@nchammas
nchammas marked this pull request as ready for review August 29, 2026 04:18
@nchammas

Copy link
Copy Markdown
ContributorAuthor

Tagging for review the same folks who reviewed #57187.

@nchammas

Copy link
Copy Markdown
ContributorAuthor

@dongjoon-hyun - Would you like me to split out the macOS-related cleanup into its own PR?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@nchammas