feat: add variant override

[wolfv]

Adds variant overrides that would add a variants.yaml file upon generation.

I think we might have to change some GHA template files in the repositories to make it work.

[traversaro]

Non blocking, but personally I would find more convenient if the variant override could be specified in pkg_additonal_info yaml file.

[traversaro]

ChatGPT made a good point:

3. One important “gotcha” when building
   rattler-build can auto-discover variants.yaml next to each recipe, but any variant filesq you pass explicitly with -m/--variant-config get higher priority than auto-discovered ones. �

[wolfv]

yes, that is indeed true. Meaning it would not work well in PR builds, unfortunately. But in the package builds we can add -m ... for an additional variant file.

[traversaro]

Meaning it would not work well in PR builds, unfortunately.

Exactly, PR builds and local builds done via pixi run build, that in my experience are relatively popular for people that want to build variants of ROS install that are not available in our channels.

[traversaro]

Ok, I see two ways forward:

• O1: We change pixi run build of robostack/ros-* repos to not just use a simple rattler-build invocation, like rattler-build build --recipe-dir ./recipes -m ./conda_build_config.yaml -c https://prefix.dev/robostack-jazzy -c https://prefix.dev/conda-forge --skip-existing" . To be honest, I am not sure how to do that.
• O2: We move the inclusion logic at the vinca recipe-generation time, and we copy/render the conda_build_config.yaml inside the recipe itself as variant.yaml (we will need to deal with v0 vs v1 style in that case). In this way, vinca can deal with any override or stuff like that. This will simplify all subsequent logic, as we could just drop any -m ./conda_build_config.yaml in pixi/PR or post-PR logic.

[traversaro]

I probably prefer O2, or at least I would know how to implement O2.

[Tobias-Fischer]

O2 sounds good - we don’t have any old style recipes anymore, do we?

[Tobias-Fischer]

Hi @wolfv - quick follow up here :)

[Tobias-Fischer]

Another ping @wolfv

[Tobias-Fischer]

One more ping @ruben-arts@wolfv

[wolfv]

Updated and rebased this PR. I implemented O2 from the discussion:

• conda_build_config.yaml is now copied into every generated recipe as variants.yaml
• package-specific variant_overrides from pkg_additional_info.yaml are merged on top
• legacy selector comments are preserved for rattler-build
• documented that downstream builds should stop passing the repository config with -m, since an explicit variant file has higher priority
• added tests for base-config merging and package overrides

Local tests/lint/format pass, and CI is green on Linux, macOS, and Windows.

[traversaro]

conda_build_config.yaml is now copied into every generated recipe as variants.yaml

Are the preprocessor converted recipe v0 style properly? I can't see that in the tests, and all the conda_build_config.yaml files in robostack/ros-* repos have preprocessor comments in that form, that should be converted to v1 preprocessor to be embedded in variants.yaml files.