ci: Move dist-aarch64-linux to an aarch64 runner by mrkajetanp · Pull Request #135262 · rust-lang/rust

mrkajetanp · 2025-01-08T17:46:41Z

Move the dist-aarch64-linux CI job to an aarch64 runner instead of cross-compiling it from an x86 one. This will make it possible to perform optimisations such as LTO, PGO and BOLT later on.

r? @Kobzol

Reland of #133809 now that the higher page sizes are fixed.

try-job: dist-aarch64-linux
try-job: dist-x86_64-linux
try-job: dist-i686-linux

mrkajetanp · 2025-01-08T17:57:35Z

Same as before, plus a couple small changes to account for the LTO+clang workarounds that were merged in the meantime

tgross35 · 2025-01-09T01:05:33Z

@bors try

…=<try> ci: Move dist-aarch64-linux to an aarch64 runner Move the dist-aarch64-linux CI job to an aarch64 runner instead of cross-compiling it from an x86 one. This will make it possible to perform optimisations such as LTO, PGO and BOLT later on. r? `@Kobzol` Reland of rust-lang#133809 now that the higher page sizes are fixed. try-job: dist-aarch64-linux try-job: dist-x86_64-linux try-job: dist-i686-linux

bors · 2025-01-09T01:06:44Z

⌛ Trying commit 7d9903c with merge a8e7b21...

bors · 2025-01-09T07:06:44Z

💥 Test timed out

mrkajetanp · 2025-01-09T10:22:34Z

Do we need a longer timeout here again?

Kobzol · 2025-01-09T14:17:11Z

We probably turned that one to a free runner recently-ish, so it times out now. With the cache primed, it should be fine.

@bors try

bors · 2025-01-09T14:18:22Z

⌛ Trying commit 7d9903c with merge 4c7352d...

…=<try> ci: Move dist-aarch64-linux to an aarch64 runner Move the dist-aarch64-linux CI job to an aarch64 runner instead of cross-compiling it from an x86 one. This will make it possible to perform optimisations such as LTO, PGO and BOLT later on. r? `@Kobzol` Reland of rust-lang#133809 now that the higher page sizes are fixed. try-job: dist-aarch64-linux try-job: dist-x86_64-linux try-job: dist-i686-linux

bors · 2025-01-09T20:05:23Z

☔ The latest upstream changes (presumably #135286) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2025-01-09T20:18:22Z

💥 Test timed out

mrkajetanp · 2025-01-09T21:13:50Z

Ahh I see, I think we need to apply the same fix as in #134690 to i686 as well.

But then it already seems like it was tested there when that PR was merged? build-clang.sh is failing with some errors mentioning Gold and that's causing the timeout in any case.

tgross35 · 2025-01-09T22:02:55Z

There is the path src/ci/docker/host-*/disabled for jobs that are still useful but don't run in CI. I'm not sure what the policy is, but maybe it would be good to move this there instead of removing it? I would find it occasionally useful for verifying aarch64 builds on x86 machines. (non-dist would be better yet, but leaving the dist job around still seems convenient).

Jobs in the disabled directory rot fast, and they can be restored from git history, so I personally don't see a lot of value in this. You can always copy-paste the contents locally to test it.

Kobzol · 2025-01-10T11:21:08Z

 -DLLVM_INCLUDE_EXAMPLES=OFF \
 -DLLVM_ENABLE_PROJECTS="clang;lld;compiler-rt;bolt" \
- -DLLVM_BINUTILS_INCDIR="/rustroot/lib/gcc/x86_64-pc-linux-gnu/$GCC_VERSION/plugin/include/" \
+ -DLLVM_BINUTILS_INCDIR="/rustroot/lib/gcc/$GCC_BUILD_TARGET/$GCC_VERSION/plugin/include/" \


It seems to me that this is the only change that happened to the 32-bit runner. I think that the problem is that only the 64-bit version of binutils is installed in the CentOS image:
2025-01-09T14:25:36.9324939Z #13 5.216 ---> Package binutils.x86_64 0:2.27-44.base.el7 will be updated 2025-01-09T14:25:36.9325811Z #13 5.217 ---> Package binutils.x86_64 0:2.27-44.base.el7_9.1 will be an update
So either this should be reverted (keep it hardcoded for 64-bit), or the 32-bit image should install also the 32-bit version of binutils.

I'm fairly sure the problem is here
-DLLVM_BINUTILS_INCDIR="/rustroot/lib/gcc/$GCC_BUILD_TARGET/$GCC_VERSION/plugin/include/" \
Because GCC_BUILD_TARGET will be i686 for the i686 job but the plugin/include will end up under x86_64 because that's where the job is running. I'm running a local build to confirm atm

I don't think that the cause is necessarily that the host is x64, rather than just the binutils package is installed only in the x64 version, not the i686 version. So downloading a binutils.i686 CentOS package should hopefully fix it. If not, I would just hardcode the previous x64 path.

rustbot · 2025-01-10T13:59:22Z

Some changes occurred in src/tools/cargo

cc @ehuss

mrkajetanp · 2025-01-10T14:01:36Z

ah sorry about that

Move the dist-aarch64-linux CI job to an aarch64 runner instead of cross-compiling it from an x86 one. This will make it possible to perform optimisations such as LTO, PGO and BOLT later on.

Kobzol · 2025-01-10T14:56:28Z

It seems like you did both, i.e. install 32-bit binutils, but use the x64 path? :D

Anyway, let's see if it helped.

@bors try

bors · 2025-01-10T14:57:40Z

⌛ Trying commit 307480a with merge c2013d9...

…=<try> ci: Move dist-aarch64-linux to an aarch64 runner Move the dist-aarch64-linux CI job to an aarch64 runner instead of cross-compiling it from an x86 one. This will make it possible to perform optimisations such as LTO, PGO and BOLT later on. r? `@Kobzol` Reland of rust-lang#133809 now that the higher page sizes are fixed. try-job: dist-aarch64-linux try-job: dist-x86_64-linux try-job: dist-i686-linux

mrkajetanp · 2025-01-10T15:00:08Z

Well I figured the builds take so long to run that I might as well try the more conservative approach :) I think if it needed the i686 binutils it probably would have failed sooner, but it failing on the linking stage of building clang suggests that it's the linker plugin include that was messed up

bors · 2025-01-10T19:38:10Z

☀️ Try build successful - checks-actions
Build commit: c2013d9 (c2013d90324bd57e903ea300e181b37f130d4b74)

…=Kobzol ci: Move dist-aarch64-linux to an aarch64 runner Move the dist-aarch64-linux CI job to an aarch64 runner instead of cross-compiling it from an x86 one. This will make it possible to perform optimisations such as LTO, PGO and BOLT later on. r? `@Kobzol` Reland of rust-lang#133809 now that the higher page sizes are fixed. try-job: dist-aarch64-linux try-job: dist-x86_64-linux try-job: dist-i686-linux

bors · 2025-01-11T20:45:18Z

⌛ Testing commit 9ba34f8 with merge 724f36a...

rust-log-analyzer · 2025-01-11T21:45:01Z

The job dist-x86_64-msvc failed! Check out the build log: (web)(plain)

Click to see the possible cause of the failure (guessed by this bot)

bors · 2025-01-11T21:45:37Z

💔 Test failed - checks-actions

mrkajetanp · 2025-01-11T23:58:14Z

Huh, I don' think this is related to the PR?

2025-01-11T21:44:43.5293056Z �[1m�[31merror�[0m�[1m:�[0m failed to remove file `C:\a\rust\rust\build\x86_64-pc-windows-msvc\stage1-rustc\x86_64-pc-windows-msvc\release\rustc-main.exe`
2025-01-11T21:44:43.5293778Z 2025-01-11T21:44:43.5293891Z Caused by:
2025-01-11T21:44:43.5294173Z Access is denied. (os error 5)

But then I know nothing about windows, so if someone who does could chime in that'd be great..

Kobzol · 2025-01-12T00:01:21Z

Yeah, that's sadly a known flaky failure.

@bors retry

bors · 2025-01-12T01:27:07Z

⌛ Testing commit 9ba34f8 with merge afb3e7e...

…=Kobzol ci: Move dist-aarch64-linux to an aarch64 runner Move the dist-aarch64-linux CI job to an aarch64 runner instead of cross-compiling it from an x86 one. This will make it possible to perform optimisations such as LTO, PGO and BOLT later on. r? `@Kobzol` Reland of rust-lang#133809 now that the higher page sizes are fixed. try-job: dist-aarch64-linux try-job: dist-x86_64-linux try-job: dist-i686-linux

rust-log-analyzer · 2025-01-12T02:33:35Z

The job x86_64-gnu-llvm-19-1 failed! Check out the build log: (web)(plain)

Click to see the possible cause of the failure (guessed by this bot)

bors · 2025-01-12T02:34:11Z

💔 Test failed - checks-actions

mrkajetanp · 2025-01-12T02:46:09Z

It seems the CI gods are not with us tonight

2025-01-12T02:33:21.7595180Z �[1m�[31merror�[0m�[1m:�[0m failed to download from `https://static.crates.io/crates/rand_xorshift/0.3.0/download`
2025-01-12T02:33:21.7595798Z 2025-01-12T02:33:21.7595923Z Caused by:
2025-01-12T02:33:21.7596566Z [7] Could not connect to server (Failed to connect to static.crates.io port 443 after 1019 ms: Could not connect to server)
2025-01-12T02:33:21.7619188Z Build completed unsuccessfully in 0:57:06

Kobzol · 2025-01-12T08:19:26Z

@bors retry

bors · 2025-01-12T11:56:07Z

⌛ Testing commit 9ba34f8 with merge c0f6a1c...

bors · 2025-01-12T14:42:03Z

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing c0f6a1c to master...

rust-timer · 2025-01-12T15:58:32Z

Finished benchmarking commit (c0f6a1c): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.3%	[-0.4%, -0.3%]	3
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results (secondary -3.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.0%	[-3.0%, -3.0%]	1
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 761.975s -> 761.573s (-0.05%)
Artifact size: 326.11 MiB -> 326.13 MiB (0.01%)

rustbot assigned Kobzol Jan 8, 2025

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Jan 8, 2025

mrkajetanp force-pushed the ci-aarch64-dist-reland branch from 528f263 to 7d9903cCompare January 8, 2025 17:51

ehuss mentioned this pull request Jan 8, 2025
Add COPYRIGHT-*.html files to distribution and update COPYRIGHT#133461
Merged

tgross35 reviewed Jan 9, 2025
View reviewed changes

Kobzol reviewed Jan 10, 2025
View reviewed changes

mrkajetanp force-pushed the ci-aarch64-dist-reland branch from 7d9903c to 1ee0062Compare January 10, 2025 13:59

ci: Move dist-aarch64-linux to an aarch64 runner
dce8c06
Move the dist-aarch64-linux CI job to an aarch64 runner instead of cross-compiling it from an x86 one. This will make it possible to perform optimisations such as LTO, PGO and BOLT later on.

mrkajetanp force-pushed the ci-aarch64-dist-reland branch from 1ee0062 to 307480aCompare January 10, 2025 14:13

bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Jan 11, 2025

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 11, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 12, 2025

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 12, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 12, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 12, 2025

bors merged commit c0f6a1c into rust-lang:masterJan 12, 2025

rustbot added this to the 1.86.0 milestone Jan 12, 2025

adamgemmell mentioned this pull request Jan 16, 2025
Tracking issue for release notes of #133807: ci: Enable opt-dist for dist-aarch64-linux builds #135482
Closed
3 tasks

saethlin mentioned this pull request Jan 23, 2025
Large const allocations no longer lead to graceful errors on aarch64 #135952
Open

mrkajetanp mentioned this pull request Jan 24, 2025
ci: Enable opt-dist for dist-aarch64-linux builds #133807
Merged

Uh oh!

Uh oh!

Conversation

mrkajetanp commented Jan 8, 2025

Uh oh!

mrkajetanp commented Jan 8, 2025

Uh oh!

tgross35 commented Jan 9, 2025

Uh oh!

bors commented Jan 9, 2025

Uh oh!

bors commented Jan 9, 2025

Uh oh!

mrkajetanp commented Jan 9, 2025

Uh oh!

Kobzol commented Jan 9, 2025

Uh oh!

bors commented Jan 9, 2025

Uh oh!

bors commented Jan 9, 2025

Uh oh!

bors commented Jan 9, 2025

Uh oh!

mrkajetanp commented Jan 9, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tgross35Jan 9, 2025

Choose a reason for hiding this comment

Uh oh!

KobzolJan 10, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KobzolJan 10, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrkajetanpJan 10, 2025

Choose a reason for hiding this comment

Uh oh!

KobzolJan 10, 2025

Choose a reason for hiding this comment

Uh oh!

rustbot commented Jan 10, 2025

Uh oh!

mrkajetanp commented Jan 10, 2025

Uh oh!

Kobzol commented Jan 10, 2025

Uh oh!

bors commented Jan 10, 2025

Uh oh!

mrkajetanp commented Jan 10, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bors commented Jan 10, 2025

Uh oh!

bors commented Jan 11, 2025

Uh oh!

rust-log-analyzer commented Jan 11, 2025

Uh oh!

bors commented Jan 11, 2025

Uh oh!

mrkajetanp commented Jan 11, 2025

Uh oh!

Kobzol commented Jan 12, 2025

Uh oh!

bors commented Jan 12, 2025

Uh oh!

rust-log-analyzer commented Jan 12, 2025

Uh oh!

bors commented Jan 12, 2025

Uh oh!

mrkajetanp commented Jan 12, 2025

Uh oh!

Kobzol commented Jan 12, 2025

Uh oh!

bors commented Jan 12, 2025

Uh oh!

bors commented Jan 12, 2025

Uh oh!

mrkajetanp commented Jan 9, 2025•
edited
Loading

KobzolJan 10, 2025•
edited
Loading

KobzolJan 10, 2025•
edited
Loading

mrkajetanp commented Jan 10, 2025•
edited
Loading