Skeleton for distributing run-task/fetch-content as rust binaries.

[bhearsum]

This is a tiny skeleton, but it gets us far enough that run-task and fetch-content are runnable after installing taskgraph.

I've also managed to mostly prove that cross compilation works. maturin build --target x86_64-pc-windows-msvc produced a Windows compatible wheel for me. maturin build --target x86_64-apple-darwin failed to produce a macOS wheel, but I believe this is fixable with these instructions about installing a compiler toolchain (I haven't had a chance to test it out yet - but I certainly know that cross compiling Rust from Linux to macOS is possible...so there's a high chance it can be made to work.)

[bhearsum]

(CI is failing because there's no Rust toolchain in the image...d'oh.)

[ahal]

This is awesome! Though something just occurred to me.. how will this play with the fact we vendor Taskgraph in Gecko?

I guess run-task is forked there anyway so we could just keep using the forked Python version. But compiled binaries will add a snag if we think merging them is important :/. We may also need to provide a way to use Taskgraph "without" these binaries for vendoring purposes.

[bhearsum]

This is awesome! Though something just occurred to me.. how will this play with the fact we vendor Taskgraph in Gecko?

I guess run-task is forked there anyway so we could just keep using the forked Python version. But compiled binaries will add a snag if we think merging them is important :/. We may also need to provide a way to use Taskgraph "without" these binaries for vendoring purposes.

How does this work for other binary dependencies (eg: cryptography)? Do they always come from a pypi mirror instead of the Gecko tree? If so, we could...theoretically do the same with taskgraph, although I'm not sure the loss in grepability is worth the trade off at that point.

Another option would be to split out any binary components into a separate package, and put only that elsewhere. (Although at that point...maybe we should just make these two things a crate that is vendored in Gecko, and built for usage?)

I don't have any brilliant ideas so far...

[ahal]

How does this work for other binary dependencies (eg: cryptography)?

Afaik, we don't vendor any binary packages for this very reason.

Another option is we could stop vendoring Taskgraph and just install it from pypi, and that should be fine (the bits of the build system that depend on Taskgraph require an internet connection anyway). As I recall, the main reason we vendor it was because Aki was concerned it would be difficult to land changes across both taskgraph and gecko_taskgraph in the event of an incident.

[bhearsum]

@jcristau and I had a discussion about this on Matrix which centered around how this would work for decision task - which is run through run-task and also publishesrun-task for other tasks to use.

This is yet another complication of having a compiled run-task. It means that the decision task either needs to build, or fetch and republish already built run-task binaries from elsewhere. It also means that the decision task itself can't use run-task which is really inconvenient.

I'm starting to think that including any compiled code in taskgraph is not going to be worth the effort...

[ahal]

Yeah, that could definitely be the case. It's probably a solvable problem.. but is it worth the headache