bootstrap: Always build for host, even when target is given

[tmandry]

This changes the behavior from not building for host whenever an
explicit target is specified. I find this much less confusing.

You can still disable host steps by passing an explicit empty list for
host.

Fixes#76990.

r? @Mark-Simulacrum

[Mark-Simulacrum]

This shouldn't be necessary anymore, the field should be removable with no effect.

[tmandry]

I tried that at first but it changed the dist behavior in the test: it causes us to distribute a rustc for the build target ("A" in the test) as well as sources. Is that desirable?

[Mark-Simulacrum]

I think that's probably because the tests are always configuring the build triple as part of the host triples, whereas here that's not the case -- https://github.com/Mark-Simulacrum/rust/blob/4de836e2141c418d37e556bc80c4ad0127d74b71/src/bootstrap/builder/tests.rs#L23-L26.

We should, I think, remove this field and update the tests to avoid that buggy behavior (probably by explicitly adding the build triple to all of the tests that need it, i.e., those that don't explicitly override skip_only_host_steps.

[tmandry]

Ah that makes sense, done. I also made them list all their targets explicitly for consistency.

[Mark-Simulacrum]

Could you also add a note to the CHANGELOG? https://github.com/rust-lang/rust/blob/master/src/bootstrap/CHANGELOG.md

No need to add a new version, just append to the list of changes.

r=me with that done.

[Mark-Simulacrum]

Hm, actually, no -- I think this probably also means that we need to go and update a bunch of our builders to explicitly pass --host '' or --host '[]' or something like that if they're only passing target right now, otherwise we'll bump build times and all up. I'm not sure what the best way to do that is, though -- maybe just rg '\-\-target' src/ci/ and trying to catch all of those cases?

I'm not sure if this is a sign we're "doing the wrong thing" or not -- I think probably no, because in most cases you do want to test or distribute for both targets and hosts, just in our CI the way we parallelize things we want to avoid that because we're taking care of the host part in one place and then spreading the targets out.

[tmandry]

No need to add a new version, just append to the list of changes.

Should we consider it a breaking change though?

[Mark-Simulacrum]

Hm, it's a good question. I guess the answer is probably yes. It should be relatively easy to bump the version number -- I think just https://github.com/rust-lang/rust/blob/master/src/bootstrap/bin/main.rs#L30 needs to change but not 100% sure.

cc @jyn514 as well

[jyn514]

You also need to add it to the changelog itself: https://github.com/rust-lang/rust/blob/master/src/bootstrap/CHANGELOG.md. Those should be the only two places I think.

[jyn514]

Hm, actually, no -- I think this probably also means that we need to go and update a bunch of our builders to explicitly pass --host '' or --host '[]' or something like that if they're only passing target right now, otherwise we'll bump build times and all up. I'm not sure what the best way to do that is, though -- maybe just rg '--target' src/ci/ and trying to catch all of those cases?

Another use case for https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/x.2Epy.20CI.20defaults ;)

[Mark-Simulacrum]

I don't think defaults would be a good fit here, they would probably confuse things more than help - better to have both overriden in the same place.

[tmandry]

Hmm, thinking about this more and I'm pretty on the fence about what to do.

For config.toml it seems like we're reverting back to what we had 1-2 weeks ago, so having people bump changelog-seen feels like overkill.

For the command line we could consider it a breaking change, but we could also say that building more than we used to in some cases isn't actually breaking. Still, we do want people who might be impacted to know about it.

[jyn514]

Would it help if instead of a binary breaking/not-breaking you printed a short description of the change?

[tmandry]

Do you mean in the changelog? Or in check_version()?

A changelog entry would probably look something like:

host now defaults to the value of build in all cases. Previously we defaulted host to an empty list when target was overridden and to build otherwise.

[jyn514]

I meant in check_version.

In general I think nagging more is better than nagging less. The thing to avoid is people changing the number without reading the changelog, and I don't think we've had enough changes for that yet.

[tmandry]

Right, and because these changes are "live" as soon as they merge, I think we're always going to have the issue of there being only a single change for each version. Which means we might want to tweak the changelog format a bit. Maybe grouping changes by month would make sense?

I do like the idea of sticking the change text in check_version(), i.e. a programmatic changelog to match the text one. One other idea: it would be nice if there were a flag or command to bump the version in your config.toml for you =)

[tmandry]

Okay, updated the changelog and bumped the version. Outputting breaking changes from x.py and adding an "ack" feature would be good for a follow up change.

[tmandry]

@Mark-Simulacrum I'll let you take a final look since I updated all the CI scripts

[bors]

☔ The latest upstream changes (presumably #77224) made this pull request unmergeable. Please resolve the merge conflicts.

Note that reviewers usually do not review pull requests until merge conflicts are resolved! Once you resolve the conflicts, you should change the labels applied by bors to indicate that your PR is ready for review. Post this as a comment to change the labels:

@rustbot modify labels: +S-waiting-on-review -S-waiting-on-author

[jyn514]

Looks like you need --host="" instead of --host somewhere.

[tmandry]

Strange, maybe some version of Docker wasn't passing through empty '' args? I changed everything to use --host='' to see if that fixes things.

@bors r=Mark-Simulacrum rollup=never

[bors]

📌 Commit 63eaa60 has been approved by Mark-Simulacrum

[bors]

⌛ Testing commit 63eaa60 with merge cd6acf7a460fd4dd80f83443261995d6907c07b7...

[bors]

💔 Test failed - checks-actions

[tmandry]

thread 'main' panicked at '
couldn't find required command: "arm-linux-gnueabihf-g++"
', src/bootstrap/sanity.rs:59:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failed to run: /checkout/obj/build/bootstrap/debug/bootstrap --stage 2 test --host= --target arm-unknown-linux-gnueabihf

I want this to be a spurious error but it probably isn't. Maybe there's a subtle change in the behavior that we haven't accounted for.

[Mark-Simulacrum]

Well, one immediate problem is that --host='' appears to configure an empty string as the triple, at least based on some quick dbg!ing locally.

[src/bootstrap/flags.rs:577] split(&matches.opt_strs("host")).into_iter().map(|x|
TargetSelection::from_user(&x)).collect::<Vec<_>>() = [
TargetSelection {
triple: "",
file: None,
},
]

We should probably just filter out empty strings in the split function in flags.rs, that seems to have fixed that failure locally for me at least.

[tmandry]

Nice catch! I seem to have forgotten that I can reproduce these locally with Docker.

[tmandry]

Pushed a change with your suggestion.

[Mark-Simulacrum]

That said the build still fails for me locally with zlib.h header missing, but not sure if that's related to this PR. I think it might be changes to the docker environment or something like that, because adding zlib1g-dev to the armhf-gnu docker container does seem to fix things. Maybe an upstream change to 16.04? Seems unlikely, but not entirely implausible, and I'm fine with hitting CI to find out.

[Mark-Simulacrum]

TIL about carp streamer, https://codepoints.net/U+1F38F, though how that relates to "flags" beyond some visual similarity I'm not sure.

Anyway, @bors r+ rollup=never

[bors]

📌 Commit bf7aeaa has been approved by Mark-Simulacrum

[bors]

⌛ Testing commit bf7aeaa with merge 0d9afb6...

[bors]

☀️ Test successful - checks-actions, checks-azure
Approved by: Mark-Simulacrum
Pushing 0d9afb6 to master...