Updating target json support

[Firestar99]

cargo gpu: Rust-GPU/cargo-gpu#75

This is the first PR to update the target spec jsons, which causes cargo gpu to run into some trouble. The codegen backend verifies that these jsons match exactly what it expects, and there's no flag to skip that check, and adding one now wouldn't help since we want to keep compatible with at least 0.9.0. Cargo gpu was never designed to have multiple target-spec versions, it always just used the latest. [...]

#249 (comment)

Note: replace rustc_codegen_spirv-types with the new rustc_codegen_spirv-target-specs crate

Could we move the target jsons from spirv_builder to rustc_codegen_spirv-types? That would allow cargo-gpu to resolve the required target jsons from rustc_codegen_spirv depending on rustc_codegen_spirv-types (and spirv_builder will retain access to it). If we can't find any target jsons in rustc_codegen_spirv-types, we can just fall back on the current target jsons, which we'd have to ship with cargo-gpu for the forseeable future. But I think that's a fine compromise, at least we wouldn't need to ship all future target jsons as well.

#249 (comment)

[LegNeato]

Should this be its own crate? @eddyb and I discussed it in the past.

[Firestar99]

You mean just the target specs json files as a separate crate? And have cargo gpu to just permanently depend on the "legacy" version of the target specs? (legacy meaning before this PR)

Just one issue with that: spirv-builder wants the path to the json files so it can point rustc to them, which is ensured to exist by directly depending on that crate. But in cargo gpu the legacy target jsons are include_str! into the binary, so we don't depend on those legacy json files remaining within cargo's crate cache. Which may be reset independently of the cargo gpu installation.

[LegNeato]

Yeah the json as their own crate. Doing so lets us have multiple versions and doesn't bring in other deps if something just needs the json. As I said we discussed it in the past but I think just kicked the can down the road until problems cropped up. We may now be there ha

[Firestar99]

Created a new rustc_codegen_spirv_target_specs crate just containing the target specs, see the readme:

rust-gpu/crates/rustc_codegen_spirv_target_specs/README.md

Lines 1 to 7 in 0f6287b

	# `rustc_codegen_spirv_target_spec`
	The target spec json files of rust-gpu to hand to the rustc compiler, declaring various metadata about our codegen backend.
	## Features
	*`include_str`: include target specs as string constants, for bundling with `cargo-gpu`
	*`dir_path`: export a path to the target specs dir, for `spirv-builder` and `compiletest` in this repo

[LegNeato]

I'd like @eddyb to chime in before approving.

[Firestar99]

@LegNeato it would be nice to have rustc_codegen_spirv_target_spec published on crates.io after this PR merges. Cargo gpu needs a permanent version-fixed dependency on the result of this PR to support the "legacy" rust-gpu versions before this PR. And I'd like that to be a crates.io version, not a git rev.

[LegNeato]

Agreed!

[eddyb]

I personally don't like the proliferation of crates, and I've yet to review any of the cargo-gpu work, but:

• most of this seems to be straightforward
• the ugly rustc_codegen_spirv prefix actually works in its favor

I only have one requests, which is mostly aesthetic.

I'm tempted to suggest something more general that could emcompass both the -types crate and the new target specs, or at least not overly narrowing the new one to "target specs" - but being specific to rustc_codegen_spirv, not Rust-GPU or even the spirv-* crates, makes me once again not want to overthink it.

[eddyb]

Cargo gpu needs a permanent version-fixed dependency on the result of this PR to support the "legacy" rust-gpu versions before this PR. And I'd like that to be a crates.io version, not a git rev.

What's kind of nice with this setup is you can easily release older versions of this crate afterwards, even if starting at 0.9.

_{The only downside is the "dead zone" of git revs (e.g. if anyone wanted to bisect Rust-GPU versions), though, hmm, it's not like the files weren't there in the git repo - I'm maybe a bit surprised you couldn't pull it from the source of spirv-builder, which sounds like the kind of thing my old proof of concept (EmbarkStudios/rust-gpu#1103) might've done (I believe in its case it was grabbing rust-toolchain.toml from the source of rustc_codegen_spirv, to control rustup with? given a Cargo dep spec for it, that is, so version/git rev/path/etc.)}

[Firestar99]

I didn't want to "magically" pull them from spirv-builder, as that assumes using some version of spirv-std for spirv-builder will give me the correct target-specs. Which is cursed. Now rustc_codegen_spirv just has a dependency to rustc_codegen_spirv-target-specs, so using standard cargo dependency resolution, I always get the correct ones. Plus some hackery for all the previous "legacy" versions.