Skip to content

gh-125022: add support for simple SIMD features detection - #125011

Draft
picnixz wants to merge 91 commits into
python:mainfrom
picnixz:core/simd-helpers
Draft

gh-125022: add support for simple SIMD features detection#125011
picnixz wants to merge 91 commits into
python:mainfrom
picnixz:core/simd-helpers

Conversation

@picnixz

@picnixzpicnixz commented Oct 5, 2024

Copy link
Copy Markdown
Member

In #124951, there has been some initial discussion on improving the performances of base64 and possibly {bytearray,bytes,str}.translate using SIMD instructions.

More generally, if we want to use specific SIMD instructions, it'd be good if we at least know whether we can use them or not. This PR is a PoC, experimental and (hence the skip news).

Note that the detection is essentially based on what was done in the blake2 module (though the flags being detected are different in this case but we can always add all those that are needed). To prepare for a harder detection of AVX instructions support, I added all the flags that are known to CPUID and possibly relevant. Note that more work needs to be done for a production use, especially in autoconf I think.

Note

The detection of a wider large family of CPUs (here we just assume Intel for simplicity and because we don't want to be overcomplicated for now) is still an ongoing work.

cc @gpshead

@picnixzpicnixz added skip issue skip news build The build process and cross-build labels Oct 5, 2024
Comment threadPython/cpuinfo.c Outdated
Comment threadPython/cpuinfo.c Outdated
Comment threadInclude/internal/pycore_cpuinfo.h Outdated
Comment threadInclude/internal/pycore_cpuinfo.h Outdated
Comment threadInclude/internal/pycore_cpuinfo.h Outdated
Comment threadPython/cpuinfo.c Outdated
Comment threadPython/cpuinfo.c Outdated
Comment threadPython/cpuinfo.c Outdated
Comment threadPython/cpuinfo.c Outdated
@picnixz
picnixz marked this pull request as ready for review October 6, 2024 09:49
@picnixzpicnixz changed the title Add support for simple SIMD features detection in autoconf (PoC)gh-125022: add support for simple SIMD features detection (PoC)Oct 6, 2024
@picnixzpicnixz changed the title gh-125022: add support for simple SIMD features detection (PoC)gh-125022: add support for simple SIMD features detectionOct 7, 2024
@picnixz

Copy link
Copy Markdown
MemberAuthor

!buildbot AMD64 FreeBSD14 PR

@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 820d140 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F125011%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 FreeBSD14 PR

The builders matched are:

  • AMD64 FreeBSD14 PR

@picnixz

Copy link
Copy Markdown
MemberAuthor

!buildbot AMD64 FreeBSD14 PR

@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @picnixz for commit df85ce5 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F125011%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 FreeBSD14 PR

The builders matched are:

  • AMD64 FreeBSD14 PR

@picnixz

Copy link
Copy Markdown
MemberAuthor

!buildbot AMD64 FreeBSD14 PR

@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @picnixz for commit c6cf903 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F125011%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 FreeBSD14 PR

The builders matched are:

  • AMD64 FreeBSD14 PR

@picnixz

Copy link
Copy Markdown
MemberAuthor

!buildbot AMD64 FreeBSD14 PR

@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 498518f 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F125011%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 FreeBSD14 PR

The builders matched are:

  • AMD64 FreeBSD14 PR

@picnixz

Copy link
Copy Markdown
MemberAuthor

!buildbot AMD64 FreeBSD14 PR

@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 3d56d93 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F125011%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 FreeBSD14 PR

The builders matched are:

  • AMD64 FreeBSD14 PR

@picnixz

Copy link
Copy Markdown
MemberAuthor

!buildbot AMD64 FreeBSD14 PR

@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 08daa8a 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F125011%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 FreeBSD14 PR

The builders matched are:

  • AMD64 FreeBSD14 PR

@picnixz

Copy link
Copy Markdown
MemberAuthor

I suspect something actually was wrong on FreeBSD/AMD side as others had similar issues. So I just removed XGETBV support. We don't really need it and I think I'll also get rid of other fields that we don't need to use now (but I'll keep their macros around).

@picnixzpicnixz added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 15, 2025
@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 79eb72d 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F125011%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 15, 2025
@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

@github-actionsgithub-actionsBot added the stale Stale PR or inactive for long period of time. label May 13, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

buildThe build process and cross-buildskip newsstaleStale PR or inactive for long period of time.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@picnixz@rruuaanng@bedevere-bot@erlend-aasland