rustbuild: Fix cross compiling to FreeBSD by alexcrichton · Pull Request #32239 · rust-lang/rust

alexcrichton · 2016-03-14T05:29:29Z

This commit fixes our support for cross compiling a compiler to run on FreeBSD.
Over the weekend I managed to get a cross compiler from Linux to FreeBSD 1
which I hope to soon use to start producing FreeBSD nightly compilers. With the
make dist support added in #32237 we should be able to produce standard
rustc/rust-std packages for FreeBSD through a new slave with this cross compiler.

Currently, however, we don't "Just Work" when cross compiling FreeBSD and a
number of changes were required (part of this PR). They include:

A few build fixes were needed in LLVM. Our own branch has been rebased on the
actual 3.8 release and I applied one extra commit 2 which contains two fixes:
1. The LLVM CMake build system passes the -Wl,-z,defs flag on many
  platforms, but not when CMAKE_SYSTEM_NAME is "FreeBSD". Unfortunately
  this doesn't take into account when we're cross compiling, and as predicted
  the build will fail if -Wl,-z,defs is passed (see 3 for more info). To
  fix this we test TARGET_TRIPLE instead of the CMAKE_SYSTEM_NAME which
  is what we're compiling for which fixes the problem.
2. The PATH_MAX constant is apparently defined in a different location than
  many other Unix systems, so a file which required this just needed some
  help to keep compiling.
Support for compiling compiler-rt with CMake has been added to rustbuild. It
looks like it just emulates Linux in what it compiles as it didn't seem to
naturally produce anything else... At least the architecture is right, so
seems good for now at least!

rust-highfive · 2016-03-14T05:29:40Z

r? @brson

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · 2016-03-14T05:30:04Z

cc @dhuseby

I hope to soon set up FreeBSD nightlies after this lands (assuming it all continues to keep working), and perhaps soon after use the groundwork to tackle OpenBSD next! (cc @semarie)

This commit fixes our support for cross compiling a compiler to run on FreeBSD. Over the weekend I managed to get a cross compiler from Linux to FreeBSD [1] which I hope to soon use to start producing FreeBSD nightly compilers. With the `make dist` support added in rust-lang#32237 we should be able to produce standard rustc/rust-std packages for FreeBSD through a new slave with this cross compiler. Currently, however, we don't "Just Work" when cross compiling FreeBSD and a number of changes were required (part of this PR). They include: * A few build fixes were needed in LLVM. Our own branch has been rebased on the actual 3.8 release and I applied one extra commit [2] which contains two fixes: 1. The LLVM CMake build system passes the `-Wl,-z,defs` flag on many platforms, but *not* when `CMAKE_SYSTEM_NAME` is "FreeBSD". Unfortunately this doesn't take into account when we're cross compiling, and as predicted the build will fail if `-Wl,-z,defs` is passed (see [3] for more info). To fix this we test `TARGET_TRIPLE` instead of the `CMAKE_SYSTEM_NAME` which is what we're compiling for which fixes the problem. 2. The `PATH_MAX` constant is apparently defined in a different location than many other Unix systems, so a file which required this just needed some help to keep compiling. * Support for compiling compiler-rt with CMake has been added to rustbuild. It looks like it just emulates Linux in what it compiles as it didn't seem to naturally produce anything else... At least the architecture is right, so seems good for now at least! [1]: https://github.com/alexcrichton/port-of-rust/blob/master/prebuilt/freebsd/Dockerfile [2]: rust-lang/llvm@be89e4b5 [3]: https://bugs.webkit.org/show_bug.cgi?id=138420

brson · 2016-03-14T22:08:45Z

@bors r+

bors · 2016-03-14T22:08:46Z

📌 Commit 155735a has been approved by brson

bors · 2016-03-15T10:21:40Z

⌛ Testing commit 155735a with merge 483fc71...

rustbuild: Fix cross compiling to FreeBSD This commit fixes our support for cross compiling a compiler to run on FreeBSD. Over the weekend I managed to get a cross compiler from Linux to FreeBSD [1] which I hope to soon use to start producing FreeBSD nightly compilers. With the `make dist` support added in #32237 we should be able to produce standard rustc/rust-std packages for FreeBSD through a new slave with this cross compiler. Currently, however, we don't "Just Work" when cross compiling FreeBSD and a number of changes were required (part of this PR). They include: * A few build fixes were needed in LLVM. Our own branch has been rebased on the actual 3.8 release and I applied one extra commit [2] which contains two fixes: 1. The LLVM CMake build system passes the `-Wl,-z,defs` flag on many platforms, but *not* when `CMAKE_SYSTEM_NAME` is "FreeBSD". Unfortunately this doesn't take into account when we're cross compiling, and as predicted the build will fail if `-Wl,-z,defs` is passed (see [3] for more info). To fix this we test `TARGET_TRIPLE` instead of the `CMAKE_SYSTEM_NAME` which is what we're compiling for which fixes the problem. 2. The `PATH_MAX` constant is apparently defined in a different location than many other Unix systems, so a file which required this just needed some help to keep compiling. * Support for compiling compiler-rt with CMake has been added to rustbuild. It looks like it just emulates Linux in what it compiles as it didn't seem to naturally produce anything else... At least the architecture is right, so seems good for now at least! [1]: https://github.com/alexcrichton/port-of-rust/blob/master/prebuilt/freebsd/Dockerfile [2]: rust-lang/llvm@be89e4b5 [3]: https://bugs.webkit.org/show_bug.cgi?id=138420

bors · 2016-03-15T13:36:45Z

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-opt-rustbuild, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-mac-ios-opt, auto-win-gnu-32-nopt-t, auto-win-gnu-32-opt, auto-win-gnu-64-nopt-t, auto-win-gnu-64-opt, auto-win-msvc-32-opt, auto-win-msvc-64-opt

The LLVM change [1] in rust-lang#32239 unfortunately broke the LLVM build on MinGW, so this LLVM submodule update brings in one more fix [2] which should hopefully remedy that. Once this lands we should be able to immediately start gating on this to prevent it from happening again. [1]: rust-lang/llvm@be89e4b5 [2]: rust-lang/llvm@3dcd2c84

rustbuild: Fix LLVM compile on MinGW The LLVM change [1] in #32239 unfortunately broke the LLVM build on MinGW, so this LLVM submodule update brings in one more fix [2] which should hopefully remedy that. Once this lands we should be able to immediately start gating on this to prevent it from happening again. [1]: rust-lang/llvm@be89e4b5 [2]: rust-lang/llvm@3dcd2c84

dhuseby · 2016-03-21T21:24:13Z

@alexcrichton this is great! will this allow me to take the FreeBSD buildbot down once it is in place and working?

alexcrichton · 2016-03-21T21:59:11Z

@dhuseby it's up to you I think. We're not producing FreeBSD nightlies and standard libraries (as of yesterday), but we're unfortunately not running any tests just yet. If you want to keep the bots online to run tests we can do that, but for nightlies we'll at least have that taken care of

dhuseby · 2016-04-02T18:47:28Z

@alexcrichton so this is building nightlies and snapshots? what about cargo nightlies and snapshots?

alexcrichton · 2016-04-02T23:42:42Z

Yeah we've got FreeBSD nightlies (not snapshots) for both rustc and Cargo now

dhuseby · 2016-04-05T17:05:31Z

@alexcrichton so do these show up in the buildbot builders lists? if we have these for linux/windows/mac, why do we still have buildbots for those platforms?

alexcrichton · 2016-04-05T17:14:02Z

Yeah the auto-linux-64-x-freebsd and auto-linux-64-x-netbsd builders are the auto builders, and the nightly-dist-rustc-cross-linux is just one massive builder creating a mess of compilers.

I left the other FreeBSD bots for now because the cross-compiled ones aren't running tests (whereas the current ones are), but if you'd like I can turn those off.

dhuseby · 2016-04-11T05:59:24Z

@alexcrichton so what is it going to take to get them to start running tests?

dhuseby · 2016-04-11T05:59:58Z

@alexcrichton I suppose that's not possible unless we have a machine running freebsd right?

alexcrichton · 2016-04-11T16:21:31Z

Yeah we don't have any time frame for getting a FreeBSD bot, but we could in theory also run them under qemu somehow (or some subset). Either way that's pretty far out and not currently planned.

dhuseby · 2016-04-14T01:13:00Z

@alexcrichton i've been interested in figuring out how to make gitian work with the bsd OS's just like it does with Linux now. Currently, it relies on qemu to boot a linux image, install dependencies, clone the code, run a build, and extract the build outputs. If we could make it also drive qemu to boot a bsd image, install dependencies, clone the code, run a build, and extract the build outputs, we could get away from cross-compiling from Linux.

What do you think?

alexcrichton · 2016-04-14T06:52:05Z

Hey so long as it works and id easy to manage I'm game :).

I've had bad luck with performance in the past running code in qemu (took forever to finish), but maybe I was passing the wrong options or there's some optimizations along the way?

dhuseby · 2016-04-20T17:40:47Z

@alexcrichton All of my buildbot slaves are run as qemu vm's (with kvm acceleration) on my server. It seems to run fine.

alexcrichton · 2016-04-20T18:44:04Z

Interesting! Seems like we should definitely at least try to get these running!

rust-highfive assigned brson Mar 14, 2016

alexcrichton force-pushed the fix-cross-to-freebsd branch from 1bb076e to 155735aCompare March 14, 2016 22:07

bors merged commit 155735a into rust-lang:masterMar 15, 2016

alexcrichton mentioned this pull request Mar 15, 2016
rustbuild: Fix LLVM compile on MinGW #32266
Merged

alexcrichton deleted the fix-cross-to-freebsd branch March 21, 2016 21:58

Uh oh!

Uh oh!

Conversation

alexcrichton commented Mar 14, 2016

Uh oh!

rust-highfive commented Mar 14, 2016

Uh oh!

alexcrichton commented Mar 14, 2016

Uh oh!

brson commented Mar 14, 2016

Uh oh!

bors commented Mar 14, 2016

Uh oh!

bors commented Mar 15, 2016

Uh oh!

bors commented Mar 15, 2016

Uh oh!

dhuseby commented Mar 21, 2016

Uh oh!

alexcrichton commented Mar 21, 2016

Uh oh!

dhuseby commented Apr 2, 2016

Uh oh!

alexcrichton commented Apr 2, 2016

Uh oh!

dhuseby commented Apr 5, 2016

Uh oh!

alexcrichton commented Apr 5, 2016

Uh oh!

dhuseby commented Apr 11, 2016

Uh oh!

dhuseby commented Apr 11, 2016

Uh oh!

alexcrichton commented Apr 11, 2016

Uh oh!

dhuseby commented Apr 14, 2016

Uh oh!

alexcrichton commented Apr 14, 2016

Uh oh!

dhuseby commented Apr 20, 2016

Uh oh!

alexcrichton commented Apr 20, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants