Add a file_digest() function in hashlib

[tarekziade]

BPO	45150
Nosy	@gpshead, @tiran, @tarekziade, @SonOfLilit, @miss-islington
PRs	bpo-45150: add a simple file_digest helper #28252 bpo-45150: draft implementation only for sha224,sha256 #31928 bpo-45150: Add hashlib.file_digest() for efficient file hashing #31930 bpo-45150: Fix testing under FIPS mode (GH-32046) #32046

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = None
created_at = <Date 2021-09-09.10:03:17.355>
labels = ['type-feature', 'library', '3.11']
title = 'Add a file_digest() function in hashlib'
updated_at = <Date 2022-03-22.15:41:04.324>
user = 'https://github.com/tarekziade'

bugs.python.org fields:

activity = <Date 2022-03-22.15:41:04.324>
actor = 'christian.heimes'
assignee = 'christian.heimes'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-09-09.10:03:17.355>
creator = 'tarek'
dependencies = []
files = []
hgrepos = []
issue_num = 45150
keywords = ['patch']
message_count = 14.0
messages = ['401457', '401461', '401462', '415138', '415139', '415141', '415320', '415321', '415324', '415326', '415336', '415356', '415754', '415793']
nosy_count = 6.0
nosy_names = ['gregory.p.smith', 'christian.heimes', 'tarek', 'python-dev', 'Aur.Saraf', 'miss-islington']
pr_nums = ['28252', '31928', '31930', '32046']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue45150'
versions = ['Python 3.11']

[tarekziade]

I am proposing the addition of a very simple helper to return the hash of a file.

[tiran]

Hey Tarek, long time no see!

• the _sha256 module is optional, can be disabled and is not available in some distributions.

• I also don't like to use sha256 has default. It's slow, even slower than sha512. Any default makes it also harder to upgrade to a better, more secure default in the future.

• like hmac.new() a file_digest() should accept PEP-452-compatible arguments and hash name as digstmod argument, not just a callable.

• a filename argument prevents users from passing in file-like objects like BytesIO.

• 4096 bytes chunk size is very conservative. The call overhead for read() and update() may dominate the performance of the function.

• The hex argument feels weird.

In a perfect world, the hash and hmac objects should get an "update_file" method. The OpenSSL-based hashes could even release the GIL and utilize OpenSSL's BIO layer to avoid any Python overhead.

[tarekziade]

Hey Christian, I hope things are well for you!
Thanks for all the precious feedback, I'll rework the patch accordingly

[SonOfLilit]

Tarek,

Are you still working on this? Would you like me to take over?

Aur

[tarekziade]

@aur, go for it, I started to implement it and got lost into the details for each backend..

[SonOfLilit]

OK, I'll give it a go.

[SonOfLilit]

PR contains a draft implementation, would appreciate some review before I implement the same interface on all builtin hashes as well as OpenSSL hashes.