GH-40077: [CI] Use GitHub hosted M1 macOS runner

[kou]

Rationale for this change

If we use GitHub hosted M1 macOS runner instead of self-hosted M1 macOS runner, we don't need to maintain self-hosted M1 macOS runner.

What changes are included in this PR?

Use GitHub hosted M1 macOS runner except r-binary-packages's r-packages job. It uses macOS 11 but GitHub hosted M1 macOS runner is 13 or 14.

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

• GitHub Issue: [CI] Move M1 self-hosted runners to newly available GitHub M1 runners #40077

[github-actions]

⚠️ GitHub issue #40077has been automatically assigned in GitHub to PR creator.

[kou]

@github-actions crossbow submit java-jars wheel-macos-*-arm64 r-binary-packages

[kou]

@github-actions crossbow submit verify-rc-source-*-macos-arm64

[kou]

Can we use macos-13-arm64 or macos-14-arm64 to verify RC of wheels?

arrow/dev/tasks/tasks.yml

Lines 1007 to 1014 in 6a7a6ee

	verify-rc-binaries-wheels-macos-11-arm64:
	ci: github
	template: verify-rc/github.macos.arm64.yml
	params:
	env:
	PYTEST_ADDOPTS: "-k 'not test_cancellation'"
	github_runner: ["self-hosted", "macOS", "arm64"]
	target: "wheels"

Is macos-11 important here?

[kou]

@github-actions crossbow submit java-jars wheel-macos--arm64 r-binary-packages verify-rc-source--macos-arm64

[assignUser]

Imo I think we can drop the self hosted M1 for r-binary-packages as well, then we can retire the runner completely. It don't recall ever having an issue with the package related to the macos version (and x86 is even run on macos 13). The issues are usually with the environment setup (e.g. gnu libtool being on the path etc.)

[kou]

OK. Let's try with GitHub hosted runner.

[kou]

@github-actions crossbow submit java-jars wheel-macos-big-sur-cp310-arm64 r-binary-packages verify-rc-source-cpp-macos-arm64

[kou]

@github-actions crossbow submit wheel-macos-big-sur-cp310-* r-binary-packages

[kou]

@github-actions crossbow submit wheel-macos-big-sur-cp310-* r-binary-packages

[kou]

@github-actions crossbow submit wheel-macos--cp310-

[kou]

@github-actions crossbow submit wheel-macos--cp310-

[kou]

@github-actions crossbow submit wheel-macos--cp310-

[kou]

@assignUser Could you check the artifacts of https://github.com/ursacomputing/crossbow/actions/runs/8220036683 ? It uses GitHub hosted runners.

[kou]

@github-actions crossbow submit wheel-macos--cp310-

[pitrou]

Could we also add a PR C++ job on macOS ARM64? (or perhaps as a separate PR)

[kou]

Hmm. It seems that we need #40418 for the wheel failures.

[kou]

@github-actions crossbow submit wheel-macos--cp38-

[github-actions]

Revision: a9db9da71dd94785eaa01fffbe7a0c2ec045f996

Submitted crossbow builds: ursacomputing/crossbow @ actions-d07c449d1f

Task	Status
wheel-macos-big-sur-cp38-arm64
wheel-macos-catalina-cp38-amd64

[pitrou]

I suppose the "install packages" step in the arm64 wheel builds will get faster once vcpkg results are cached?

[kou]

I suppose the "install packages" step in the arm64 wheel builds will get faster once vcpkg results are cached?

Yes. The previous arm64 template doesn't use NuGet based cache but I unified amd64/arm64 templates in this PR. So the arm64 template can also use NuGet based cache. It'll reduce build time.

[kou]

Should we wait for #40418 before we merge this?

[pitrou]

I think it would be better to wait for #40418.

[kou]

@github-actions crossbow submit wheel-macos-*-arm64

[github-actions]

Revision: de8ae54

Submitted crossbow builds: ursacomputing/crossbow @ actions-7a78f0a41d

Task	Status
wheel-macos-big-sur-cp310-arm64
wheel-macos-big-sur-cp311-arm64
wheel-macos-big-sur-cp312-arm64
wheel-macos-big-sur-cp38-arm64
wheel-macos-big-sur-cp39-arm64

[kou]

Green. I'll merge this.

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 7d290bd.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.