gh-152384: pixi-packages: use `flags` to define variants by lucascolley · Pull Request #152385 · python/cpython

lucascolley · 2026-06-27T10:56:40Z

cc @wolfv @baszalmstra @jaimergp I'm very excited about this PR!

lucascolley · 2026-06-27T10:57:35Z

 - `freethreading`
 - `asan`: ASan-instrumented build
- `tsan-freethreading`: TSan-instrumented free-threading build
+- `tsan_freethreading`: TSan-instrumented free-threading build


I think it makes sense to remove the - from this variant name, given that it is invalid in the flags field: https://conda.org/learn/ceps/cep-0045#repodata-record-syntax

seems fine to me, I think you and @crusaderky are just about the only users of these builds so far for what its worth

lucascolley · 2026-06-27T10:59:04Z

 run_exports:
 noarch:
 - python
 weak:
 - python_abi ${{ version }}.* *_${{ abi_tag }}


we could also define flags on python_abi and perhaps not need to rely on abi_tag as much? But I suppose we still want to keep abi_tag around at least while conda-forge does?
EDIT: can probably leave this for a follow-up and open an issue

lucascolley · 2026-06-27T11:01:37Z

@StanFromIreland this goes some way towards making the version update situation better by reducing duplication in the variant definition

lucascolley · 2026-06-27T11:12:25Z

+openssl:
+ - '3.5'


ref. prefix-dev/pixi#6460 (comment)@h-vetinari, I wonder whether there is a better way to do this by pulling in (some of?) https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml

obviously you could download the current pinning file, yaml-load it and extract the openssl pin, but that's perhaps more complexity and dynamism than justified - an argument could be made that openssl is special enough to handle it separately... and although pins have a habit of going stale, staying on an LTS version probably makes sense here.
conda-forge will probably migrate to openssl 4 in the fall, but since AFAICT python has no other dependencies that are themselves openssl-dependent, this should not cause any conflicts (except in larger environments with the most recent builds, but that's not really relevant for the ASAN stuff).

obviously you could download the current pinning file, yaml-load it and extract the openssl pin, but that's perhaps more complexity and dynamism than justified
an idea was floated at some point that variant configs themselves could be packaged as conda packages and Pixi could consume them that way. Not sure if anyone has thought about that recently.

I mean, conda-forge-pinningis exactly that package. You'd still have to go parse the yaml though. 🤷

ngoldbaum · 2026-07-09T18:02:19Z

The CI failure looks unrelated but before this PR can be merged it's going to need a NEWS entry. Can you add one?

hugovk · 2026-07-10T11:39:45Z

I've updated this branch from main, which should pull in a skip for the failing unrelated macOS test.

lucascolley · 2026-07-10T13:40:04Z

news added

lucascolley · 2026-07-13T12:23:17Z

@ngoldbaum @kumaraditya303 any remaining concerns from your side? I think we can merge and patch up any more changes in a follow-up, this seems to be an improvement on the situation in main at least :)

Co-authored-by: Isuru Fernando <isuruf@gmail.com>

lucascolley · 2026-07-16T09:46:08Z

@ngoldbaum @hugovk I believe this is ready for a final look!

ngoldbaum

Other than the issue with the error message, LGTM!

Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>

kumaraditya303 · 2026-07-17T06:31:25Z

This looks good, I'll merge it now to unblock other stuff.

hugovk · 2026-07-17T06:44:49Z

How far do we want to backport this? Or just main for now?

I generally prefer to have less diverse config across branches, as it makes it harder to maintain.

rgommers · 2026-07-17T06:58:20Z

Either way works I think, it shouldn't break anything to backport. Both because it's still very new with only a few users, and because the main way of using it is to point at an exact commit hash in a downstream pixi.toml; in SciPy CI we're pointing at commits on main.

lucascolley · 2026-07-17T08:20:41Z

a backport to 3.15 would be nice, however https://github.com/python/cpython/blob/main/Tools/pixi-packages/variants.yaml#L1 will need a tweak

lucascolley · 2026-08-02T16:40:26Z

a backport to 3.15 would be nice, however main/Tools/pixi-packages/variants.yaml#L1 will need a tweak

hey @hugovk, any chance of a backport?

miss-islington-app · 2026-08-02T17:49:06Z

Thanks @lucascolley for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

bedevere-app · 2026-08-02T17:49:17Z

GH-155091 is a backport of this pull request to the 3.15 branch.

hugovk · 2026-08-02T17:51:41Z

Sure, how does #155091 look?

…52385) (#155091) * gh-152384: pixi-packages: use `flags` to define variants (GH-152385) (cherry picked from commit 2931f20) Co-authored-by: Lucas Colley <lucas.colley8@gmail.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Isuru Fernando <isuruf@gmail.com> Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com> * version: ["3.15"] --------- Co-authored-by: Lucas Colley <lucas.colley8@gmail.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Isuru Fernando <isuruf@gmail.com> Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>

bedevere-appBot mentioned this pull request Jun 27, 2026
pixi-packages: simplify to a single pixi.toml#152384
Closed

bedevere-appBot added the awaiting review label Jun 27, 2026

lucascolley commented Jun 27, 2026
View reviewed changes

lucascolley force-pushed the pixi-flags branch from 714d63f to 34cbfc3Compare June 27, 2026 11:10

lucascolley commented Jun 27, 2026
View reviewed changes

pythongh-152384: pixi-packages: useflagsto define variants
8b5b0c2

lucascolley force-pushed the pixi-flags branch from 34cbfc3 to 8b5b0c2Compare June 27, 2026 11:16

lucascolley commented Jun 27, 2026
View reviewed changes

Comment threadTools/pixi-packages/README.md Outdated

This was referenced Jun 27, 2026
ENH: pixi-packages: use skip-pyc-compilation for tsan-freethreadingnumpy/numpy#31558
Draft
ENH: pixi-packages: use flagsnumpy/numpy#31805
Merged

This was referenced Jul 4, 2026
BLD/DEV: ASAN at your fingertips! scipy/scipy#24066
Open
GH-49321: [C++][Python] Add ASAN / UBSAN pixi builds for Arrow and PyArrow apache/arrow#49849
Open

ngoldbaum added awaiting changes and removed awaiting review labels Jul 9, 2026

Merge branch 'main' into pixi-flags
81c8972

lucascolley added 2 commits July 10, 2026 14:31

tweaks
b226f35

add news
5cba45c

lucascolley requested a review from ngoldbaum July 10, 2026 13:39

ngoldbaum added awaiting core review and removed awaiting changes labels Jul 10, 2026

lucascolley commented Jul 12, 2026
View reviewed changes

Comment threadTools/pixi-packages/variants.yaml

pin to clang 21
f152ba7

isuruf reviewed Jul 13, 2026
View reviewed changes

Comment threadTools/pixi-packages/variants.yaml Outdated

remove abi_tag from variant
18c12e6
Co-authored-by: Isuru Fernando <isuruf@gmail.com>

isuruf reviewed Jul 13, 2026
View reviewed changes

Comment threadTools/pixi-packages/recipe.yaml Outdated

lucascolley requested review from AA-Turner, JacobCoffee, ezio-melotti, hugovk, itamaro and webknjaz as code owners July 16, 2026 09:45

ngoldbaum reviewed Jul 16, 2026
View reviewed changes

Comment threadTools/pixi-packages/build.sh Outdated

ngoldbaum approved these changes Jul 16, 2026
View reviewed changes

Update Tools/pixi-packages/build.sh
d72075b
Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>

isuruf reviewed Jul 16, 2026
View reviewed changes

Comment threadTools/pixi-packages/build.sh Outdated

Update Tools/pixi-packages/build.sh
de788e8

kumaraditya303 approved these changes Jul 17, 2026
View reviewed changes

bedevere-appBot added awaiting merge and removed awaiting core review labels Jul 17, 2026

kumaraditya303 merged commit 2931f20 into python:mainJul 17, 2026
55 checks passed

bedevere-appBot removed the awaiting merge label Jul 17, 2026

lucascolley deleted the pixi-flags branch July 17, 2026 08:19

hugovk added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Aug 2, 2026

bedevere-appBot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Aug 2, 2026

hugovk mentioned this pull request Aug 2, 2026
[3.15] gh-152384: pixi-packages: use flags to define variants (GH-152385) #155091
Merged

		openssl:
		- '3.5'

Uh oh!

Uh oh!

Conversation

lucascolley commented Jun 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucascolleyJun 27, 2026

Choose a reason for hiding this comment

Uh oh!

ngoldbaumJul 9, 2026

Choose a reason for hiding this comment

Uh oh!

lucascolleyJun 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucascolley commented Jun 27, 2026

Uh oh!

lucascolleyJun 27, 2026

Choose a reason for hiding this comment

Uh oh!

h-vetinariJun 28, 2026

Choose a reason for hiding this comment

Uh oh!

lucascolleyJun 28, 2026

Choose a reason for hiding this comment

Uh oh!

h-vetinariJun 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ngoldbaum commented Jul 9, 2026

Uh oh!

hugovk commented Jul 10, 2026

Uh oh!

lucascolley commented Jul 10, 2026

Uh oh!

Uh oh!

lucascolley commented Jul 13, 2026

Uh oh!

Uh oh!

Uh oh!

lucascolley commented Jul 16, 2026

Uh oh!

Uh oh!

ngoldbaum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kumaraditya303 commented Jul 17, 2026

Uh oh!

Uh oh!

hugovk commented Jul 17, 2026

Uh oh!

rgommers commented Jul 17, 2026

Uh oh!

lucascolley commented Jul 17, 2026

Uh oh!

lucascolley commented Aug 2, 2026

Uh oh!

miss-islington-appBot commented Aug 2, 2026

Uh oh!

bedevere-appBot commented Aug 2, 2026

Uh oh!

hugovk commented Aug 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

lucascolley commented Jun 27, 2026•
edited
Loading

lucascolleyJun 27, 2026•
edited
Loading