Link LLVM dynamically on aarch64-apple-darwin

[sgasho]

--set llvm.link-shared=true on dist-aarch64-apple to link LLVM dynamically on MacOS.
Also, I fixed some link problems, by normalizing the libLLVM.dylib name, and creating a symlink(On Linux, we use linker script, but we can not use it for MacOS. ld64 does not recognize linker scripts) from versioned dylib to unversioned libLLVM.dylib to build successfully.

How I verified on my Mac

1. remove the old build dir

rm -rf build/

2. configure for rust-dev dist

./configure \
--set llvm.download-ci-llvm=false \
--set llvm.link-shared=true \
--release-channel=nightly

3. dist

./x dist rust-dev --host=aarch64-apple-darwin --target=aarch64-apple-darwin

4. move tarball to /tmp(or, wherever else except for inside build/) and delete the current build dir

We create build/ for both rust-dev dist, and local build tests. Once dist finished, we have to clear the directory so we can use it for build tests

cp build/dist/rust-dev-nightly-aarch64-apple-darwin.tar.xz /tmp/
rm -rf build

I saw a symlink created as expected

6. modify download.rs temporary so it gets the locally built tarball at step3

#[cfg(not(test))]fndownload_ci_llvm(&self,_llvm_sha:&str){let tarball = PathBuf::from("/tmp/rust-dev-nightly-aarch64-apple-darwin.tar.xz");let llvm_root = self.ci_llvm_root();self.unpack(&tarball,&llvm_root,"rust-dev");// let llvm_assertions = self.llvm_assertions;// let cache_prefix = format!("llvm-{llvm_sha}-{llvm_assertions}");// let cache_dst =// self.bootstrap_cache_path.as_ref().cloned().unwrap_or_else(|| self.out.join("cache"));// let rustc_cache = cache_dst.join(cache_prefix);// if !rustc_cache.exists() {// t!(fs::create_dir_all(&rustc_cache));// }// let base = if llvm_assertions {// &self.stage0_metadata.config.artifacts_with_llvm_assertions_server// } else {// &self.stage0_metadata.config.artifacts_server// };// let version = self.artifact_version_part(llvm_sha);// let filename = format!("rust-dev-{}-{}.tar.xz", version, self.host_target.triple);// let tarball = rustc_cache.join(&filename);// if !tarball.exists() {// let help_on_error = "ERROR: failed to download llvm from ci// HELP: There could be two reasons behind this:// 1) The host triple is not supported for `download-ci-llvm`.// 2) Old builds get deleted after a certain time.// HELP: In either case, disable `download-ci-llvm` in your bootstrap.toml:// [llvm]// download-ci-llvm = false// ";// self.download_file(&format!("{base}/{llvm_sha}/{filename}"), &tarball, help_on_error);// }// let llvm_root = self.ci_llvm_root();// self.unpack(&tarball, &llvm_root, "rust-dev");}

7. configure for ./x build

rm -rf bootstrap.toml // delete existing config of rust-dev dist
// maybe we don't need to specify llvm.download-ci-llvm=true, since I bumped download-ci-llvm-stamp.// but anyway, I explicitly specified it when I tested it./configure \ --set llvm.download-ci-llvm=true \ --set llvm.link-shared=true \ --release-channel=nightly

8. ./x build

./x build (library)

Result: build completed successfully

r? @Kobzol

cc: @ZuseZ4

[rustbot]

Warning

If you are changing how CI LLVM is built or linked, make sure to bump
src/bootstrap/download-ci-llvm-stamp.

cc @jieyouxu

[rustbot]

Kobzol is not on the review rotation at the moment.
They may take a while to respond.

[ZuseZ4]

@bors r+ rollup=never

The last time we tried this change we broke rustc contributors, since download-llvm-ci failed for them.
The pr author since reproduced that in another PR in CI, and also locally. This PR fixes failures locally and in CI, so it's worth trying again.

We should do the same for the x86 apple runner once we verified it works.

[rust-bors]

📌 Commit 3af88c5 has been approved by ZuseZ4

It is now in the queue for this repository.

[rust-bors]

☀️ Test successful - CI
Approved by: ZuseZ4
Duration: 5h 4m 24s
Pushing bef8e62 to main...

[github-actions]

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 14210df (parent) -> bef8e62 (this PR)

Test differences

Show 4 test diffs

Stage 2

• [run-make] tests/run-make/compressed-debuginfo-zstd: pass -> ignore (ignored if LLVM wasn't build with zstd for ELF section compression or LLVM is not the default codegen backend) (J0)

Additionally, 3 doctest diffs were found. These are ignored, as they are noisy.

Job group index

• J0: x86_64-gnu-nopt

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
test-dashboard bef8e620f19adbfd1530e916ab8caa296ef9c3ee --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

1. x86_64-gnu-gcc-core-tests: 9m 43s -> 1h 8m (+602.5%)
2. pr-check-1: 30m 8s -> 1h 26m (+186.2%)
3. dist-aarch64-apple: 1h 59m -> 4h 58m (+149.6%)
4. pr-check-2: 39m 10s -> 1h 34m (+142.3%)
5. x86_64-rust-for-linux: 47m 22s -> 1h 41m (+114.5%)
6. x86_64-gnu-gcc: 1h 1m -> 2h 11m (+114.0%)
7. x86_64-gnu-tools: 1h -> 2h 1m (+100.2%)
8. x86_64-gnu-miri: 1h 28m -> 2h 54m (+96.5%)
9. arm-android: 1h 43m -> 2h 59m (+73.0%)
10. test-various: 1h 50m -> 3h 5m (+68.3%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

[rust-timer]

Finished benchmarking commit (bef8e62): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This perf run didn't have relevant results for this metric.

Max RSS (memory usage)

Results (primary -3.1%, secondary 4.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.2%	[2.5%, 5.8%]	2
Improvements ✅ (primary)	-3.1%	[-3.1%, -3.1%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-3.1%	[-3.1%, -3.1%]	1

Cycles

Results (primary 2.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.3%, 2.3%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.3%	[2.3%, 2.3%]	1

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 511.515s -> 518.865s (1.44%)
Artifact size: 400.77 MiB -> 400.72 MiB (-0.01%)