gh-127298: Refactor test_hashlib for better usedforsecurity & openssl fips mode env support.

[gpshead]

• Issue: Ensure builtin hashlib implementations honor usedforsecurity=True when _hashlib is in FIPS mode #127298

[gpshead]

!buildbot FIPS

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @gpshead for commit 59d9a85 🤖

The command will test the builders whose names match following regular expression: FIPS

The builders matched are:

• AMD64 CentOS9 FIPS Only Blake2 Builtin Hash PR
• AMD64 RHEL8 FIPS No Builtin Hashes PR
• AMD64 RHEL8 FIPS Only Blake2 Builtin Hash PR
• AMD64 CentOS9 FIPS No Builtin Hashes PR

[gpshead]

!buildbot FIPS

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @gpshead for commit bd46651 🤖

The command will test the builders whose names match following regular expression: FIPS

The builders matched are:

• AMD64 CentOS9 FIPS Only Blake2 Builtin Hash PR
• AMD64 RHEL8 FIPS No Builtin Hashes PR
• AMD64 RHEL8 FIPS Only Blake2 Builtin Hash PR
• AMD64 CentOS9 FIPS No Builtin Hashes PR

[gpshead]

#127467 is follow-on work to this that combined gets the FIPS mode buildbots passing in main.

[vstinner]

cc @stratakis@encukou

[stratakis]

Thanks for the PR! I'll have a look at it and I can also provide ssh access to the FIPS buildbots if that would make things easier to debug.

[vstinner]

I tested manually the change on RHEL8. 3 MD5 tests of test_hashlib are failing:

======================================================================
ERROR: test_case_md5_0 (test.test_hashlib.HashLibTestCase.test_case_md5_0)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vstinner/cpython/Lib/test/test_hashlib.py", line 547, in test_case_md5_0
self.check(
~~~~~~~~~~^
'md5', b'', 'd41d8cd98f00b204e9800998ecf8427e',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
usedforsecurity=False
^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/vstinner/cpython/Lib/test/test_hashlib.py", line 427, in check
self.check_file_digest(name, data, hexdigest)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vstinner/cpython/Lib/test/test_hashlib.py", line 447, in check_file_digest
hashlib.file_digest(buf, digest).hexdigest(), hexdigest
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/home/vstinner/cpython/Lib/hashlib.py", line 211, in file_digest
digestobj = digest()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
======================================================================
ERROR: test_case_md5_1 (test.test_hashlib.HashLibTestCase.test_case_md5_1)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vstinner/cpython/Lib/test/test_hashlib.py", line 553, in test_case_md5_1
self.check(
~~~~~~~~~~^
'md5', b'abc', '900150983cd24fb0d6963f7d28e17f72',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
usedforsecurity=False
^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/vstinner/cpython/Lib/test/test_hashlib.py", line 427, in check
self.check_file_digest(name, data, hexdigest)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vstinner/cpython/Lib/test/test_hashlib.py", line 447, in check_file_digest
hashlib.file_digest(buf, digest).hexdigest(), hexdigest
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/home/vstinner/cpython/Lib/hashlib.py", line 211, in file_digest
digestobj = digest()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
======================================================================
ERROR: test_case_md5_2 (test.test_hashlib.HashLibTestCase.test_case_md5_2)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vstinner/cpython/Lib/test/test_hashlib.py", line 559, in test_case_md5_2
self.check(
~~~~~~~~~~^
'md5',
^^^^^^
...<2 lines>...
usedforsecurity=False
^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/vstinner/cpython/Lib/test/test_hashlib.py", line 427, in check
self.check_file_digest(name, data, hexdigest)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vstinner/cpython/Lib/test/test_hashlib.py", line 447, in check_file_digest
hashlib.file_digest(buf, digest).hexdigest(), hexdigest
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/home/vstinner/cpython/Lib/hashlib.py", line 211, in file_digest
digestobj = digest()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
----------------------------------------------------------------------

[vstinner]

I built Python with ./configure --with-pydebug && make. MD5 is available:

[vstinner@python-builder-rhel8-fips cpython]$ ./python Python 3.14.0a2+ (heads/pr/127492:3f9ec82db, Dec 5 2024, 06:10:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-22)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.md5(usedforsecurity=False).hexdigest()
'd41d8cd98f00b204e9800998ecf8427e'

[vstinner]

Suggested fix for file digest:

diff --git a/Lib/hashlib.py b/Lib/hashlib.py
index 44656c33a..93d602571 100644
--- a/Lib/hashlib.py+++ b/Lib/hashlib.py@@ -192,7 +192,7 @@ def __hash_new(name, data=b'', **kwargs):
pass
-def file_digest(fileobj, digest, /, *, _bufsize=2**18):+def file_digest(fileobj, digest, /, *, usedforsecurity=True, _bufsize=2**18):
"""Hash the contents of a file-like object. Returns a digest object.
*fileobj* must be a file-like object opened for reading in binary mode.
@@ -206,9 +206,9 @@ def file_digest(fileobj, digest, /, *, _bufsize=2**18):
# On Linux we could use AF_ALG sockets and sendfile() to archive zero-copy
# hashing with hardware acceleration.
if isinstance(digest, str):
- digestobj = new(digest)+ digestobj = new(digest, usedforsecurity=usedforsecurity)
else:
- digestobj = digest()+ digestobj = digest(usedforsecurity=usedforsecurity)
if hasattr(fileobj, "getbuffer"):
# io.BytesIO object, use zero-copy buffer
diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py
index 1c1a0396c..a7cca0ba5 100644
--- a/Lib/test/test_hashlib.py+++ b/Lib/test/test_hashlib.py@@ -443,11 +443,13 @@ def check_file_digest(self, name, data, hexdigest):
for digest in digests:
buf = io.BytesIO(data)
buf.seek(0)
- self.assertEqual(- hashlib.file_digest(buf, digest).hexdigest(), hexdigest- )+ digestobj = hashlib.file_digest(buf, digest,+ usedforsecurity=False)+ self.assertEqual(digestobj.hexdigest(), hexdigest)+
with open(os_helper.TESTFN, "rb") as f:
- digestobj = hashlib.file_digest(f, digest)+ digestobj = hashlib.file_digest(f, digest,+ usedforsecurity=False)
self.assertEqual(digestobj.hexdigest(), hexdigest)
finally:
os.unlink(os_helper.TESTFN)

[xnox]

@gpshead the suggestion from @vstinner looks reasonable in #127492 (comment) will you push that to this pr? Or should that change be done separately, with a separate blurb entry?

[xnox]

@gpshead hi, my PRs had comments that you are taking over with these changes instead, but they now seem to have stalled. Are you still actively working on this and related PRs?

[picnixz]

Note that changing how we materialize a digest object should done carefully (see PEP-452). For new(), it should be fine as we're using our own hsahlib.new but for digest(), we should be careful here because not all digest constructors necessarily need to accept usedforsecurity (see the examples in https://docs.python.org/3/library/hashlib.html#hashlib.file_digest):

buf=io.BytesIO(b"somedata")
mac1=hmac.HMAC(b"key", digestmod=hashlib.sha512)
digest=hashlib.file_digest(buf, lambda: mac1)

[xnox]

@gpshead Any updates on landing this?

[github-actions]

This PR is stale because it has been open for 30 days with no activity.

[gpshead]

I believe this PR is no longer relevant as is.