Detect configuration for LLVM during setup

[alarsyo]

This is a first draft to address #77579, setting download-ci-llvm to true on Linux, but I could also implement the if-available setting mentioned in the issue.

On other platforms I was thinking about using the which crate, if adding a dependency on it is considered okay of course, to detect the presence of llvm-config in the path, and use it if found. Still a work in progress of course.

[rust-highfive]

r? @Mark-Simulacrum

(rust_highfive has picked a reviewer for you, use r? to override)

[Mark-Simulacrum]

I would not want to add a dependency for this sort of thing, it shouldn't be necessary to just scan PATH and append llvm-config to it.

I don't think appending to the file is going to work well, I think the if-available approach is the one we should pursue.

[alarsyo]

Is that better? Hopefully the "if-available" option is now handled everywhere needed, and I just need to write the llvm-config detection logic for other platforms

[bors]

☔ The latest upstream changes (presumably #77867) made this pull request unmergeable. Please resolve the merge conflicts.

Note that reviewers usually do not review pull requests until merge conflicts are resolved! Once you resolve the conflicts, you should change the labels applied by bors to indicate that your PR is ready for review. Post this as a comment to change the labels:

@rustbot modify labels: +S-waiting-on-review -S-waiting-on-author

[Mark-Simulacrum]

Could we just duplicate this across the actual default files? I would prefer to avoid adding even more layers of places where defaults are getting set.

[jyn514]

Yeah, in my mind setup.rs is for setting defaults that are not the same for everyone, while src/bootstrap/defaults are for things that can reasonably be shared.

[alarsyo]

Done! Added to all default configs. I just have a doubt for the config.codegen.toml, maybe people working close to codegen don't want the CI built one by default?

[alarsyo]

Ah, it seems moving these lines to the default profiles broke the bootstrap, because bootstrap.py doesn't know about the profile setting (I'm guessing it's only handled in the config.rs file)

[alarsyo]

Looking further into this, it seems bootstrap only has a basic regex mechanism to lookup toml keys, and returns the first key that matches, which means appending the contents of our default file should work as expected, allowing the main config.toml to override these defaults if needed. Definitely feels like a hack though :D I'll push some code soon

[jyn514]

Got it, thanks.

@Mark-Simulacrum how hard would it be to move that into rustbuild instead of bootstrap.py? Doesn't need to block this, but I'd like to do as little as possible in python anyway.

[Mark-Simulacrum]

That error comes from the Rust portion of rustbuild, I think? It looks like a Rust error...

Presumably LLVM was not downloaded or not unpacked which led to this problem.

[alarsyo]

FWIW, I'd be interested in tackling this to get more familiar with the bootstrap code, if you decide to move this to rustbuild :)

[alarsyo]

Presumably LLVM was not downloaded or not unpacked which led to this problem.

Yes, the rust code doesn't find the executable because it wasn't downloaded by bootstrap.py, because bootstrap.py didn't know about the default file, so the get_toml('download-ci-llvm') returned None

[jyn514]

Let's tackle this in a follow-up, it seems reasonable to have bootstrap.py read profiles anyway.

[jyn514]

I'm 90% sure this will break - this causes invalid syntax if there's already anything in the config.toml. #76628 (comment)

[alarsyo]

It would break if parsed by a real TOML parser, but that's not the case, the basic regex parser will just match the first relevant key

[alarsyo]

(and since the aggregation of the two files is only stored as a string in memory, not written to a toml file somewhere, I don't think it should break anything else?)

[alarsyo]

@rustbot modify labels: +S-waiting-on-review -S-waiting-on-author

[jyn514]

@bors r+

In a follow-up I'd like to move handling of LLVM from bootstrap.py to rustbuild, but that doesn't need to block this change.

[bors]

📌 Commit b8ae4c5 has been approved by jyn514

[bors]

🌲 The tree is currently closed for pull requests below priority 100, this pull request will be tested once the tree is reopened

[alarsyo]

@jyn514 should I open an issue about a follow-up change for rustbuild ?

[Mark-Simulacrum]

It would mean a curl or other "http talker" dep in rustbuild which I'm opposed to at this point I think, but I don't mind an issue.

[bors]

⌛ Testing commit b8ae4c5 with merge 19e1aac...

[bors]

☀️ Test successful - checks-actions, checks-azure
Approved by: jyn514
Pushing 19e1aac to master...