bpo-29882: Add _Py_popcount32() function

[vstinner]

• Rename pycore_byteswap.h to pycore_bitutils.h.
• Move popcount_digit() to pycore_bitutils.h as _Py_popcount32().
• _Py_popcount32() uses GCC and clang builtin function if available.
• Add unit tests to _Py_popcount32().

https://bugs.python.org/issue29882

[vstinner]

@mdickinson: here you have.

cc @pablogsal@1st1

[vstinner]

On my x86-64 Fedora 32 with GCC 10.1.1, __builtin_popcount() becomes a call to __popcountdi2() function:

(gdb) disassemble __popcountdi2 Dump of assembler code for function __popcountdi2:
0x00007fffeaa997c0 <+0>:	endbr64 0x00007fffeaa997c4 <+4>:	movabs rdx,0x5555555555555555
0x00007fffeaa997ce <+14>:	mov rax,rdi
0x00007fffeaa997d1 <+17>:	shr rax,1
0x00007fffeaa997d4 <+20>:	and rax,rdx
0x00007fffeaa997d7 <+23>:	movabs rdx,0x3333333333333333
0x00007fffeaa997e1 <+33>:	sub rdi,rax
0x00007fffeaa997e4 <+36>:	mov rax,rdi
0x00007fffeaa997e7 <+39>:	shr rdi,0x2
0x00007fffeaa997eb <+43>:	and rax,rdx
0x00007fffeaa997ee <+46>:	and rdi,rdx
0x00007fffeaa997f1 <+49>:	add rdi,rax
0x00007fffeaa997f4 <+52>:	mov rax,rdi
0x00007fffeaa997f7 <+55>:	shr rax,0x4
0x00007fffeaa997fb <+59>:	add rax,rdi
0x00007fffeaa997fe <+62>:	movabs rdi,0xf0f0f0f0f0f0f0f
0x00007fffeaa99808 <+72>:	and rax,rdi
0x00007fffeaa9980b <+75>:	movabs rdi,0x101010101010101
0x00007fffeaa99815 <+85>:	imul rax,rdi
0x00007fffeaa99819 <+89>:	shr rax,0x38
0x00007fffeaa9981d <+93>:	ret End of assembler dump.

It looks like the SWAR popcount.

Using -march=native, I get the expected native x86 POPCNT instruction:

popcnt ecx,ecx

[vstinner]

I wrote this PR to factorize duplicate function:

• hamt_bitcount() in hamt.c
• popcount_digit() in Objects/longobject.c

I also added __builtin_popcount() to address the first paragraph of hamt.c comment:

 /* We could use native popcount instruction but that would
require to either add configure flags to enable SSE4.2
support or to detect it dynamically. Otherwise, we have
a risk of CPython not working properly on older hardware.
In practice, there's no observable difference in
performance between using a popcount instruction or the
following fallback code.
The algorithm is copied from:
https://graphics.stanford.edu/~seander/bithacks.html
*/

[vstinner]

This PR is a follow-up of PR #771 which added int.bit_count() Python function and popcount_digit() C function.

[vstinner]

@mdickinson: I updated the PR. Would you mind to review it?

[mdickinson]

Would you mind to review it?

Will do, but not before the weekend.

[mdickinson]

I left a few comments; mostly nitpicks, but there are some bits I don't understand (like the SIZEOF_INT == 8 branch).

[vstinner]

Oh, 45af786 is good example of why I'm mentioning one used function on an include :-)

[vstinner]

@mdickinson: PR updated to address your latest review.

PR rebased to fix a merge conflict with 45af786

[mdickinson]

LGTM

[vstinner]

Thanks @mdickinson for the reviews!