Skip to content

gh-109653: Improve the import time of email.utils#109824

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hauntsaninja merged 12 commits into from
Oct 12, 2023
Merged

gh-109653: Improve the import time of email.utils#109824

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
0aca553
WIP
AlexWaygood Sep 25, 2023
f3c815d
news
AlexWaygood Sep 25, 2023
992274f
I can spell my own name
AlexWaygood Sep 25, 2023
493aa33
Apply suggestions from code review
AlexWaygood Sep 25, 2023
a697d9d
Merge branch 'main' into email-message-import
AlexWaygood Oct 3, 2023
03233f1
Abandon the separate submodule idea
AlexWaygood Oct 3, 2023
f11bc98
stats are slightly less impressive now because importing random is ch…
AlexWaygood Oct 3, 2023
c5e7a61
Merge branch 'email-message-import' of https://github.com/AlexWaygood…
AlexWaygood Oct 3, 2023
bcaeb4c
Merge branch 'main' into email-message-import
AlexWaygood Oct 3, 2023
b387dcd
Update Lib/email/utils.py
AlexWaygood Oct 3, 2023
c72ee5c
Merge branch 'main' of https://github.com/python/cpython into email-m…
AlexWaygood Oct 4, 2023
e201028
eagerly import os, it is free
AlexWaygood Oct 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions Lib/email/utils.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -25,8 +25,6 @@
import os
import re
import time
import random
import socket
import datetime
import urllib.parse

Expand All@@ -36,9 +34,6 @@

from email._parseaddr import parsedate, parsedate_tz, _parsedate_tz

# Intrapackage imports
from email.charset import Charset

COMMASPACE = ', '
EMPTYSTRING = ''
UEMPTYSTRING = ''
Expand DownExpand Up@@ -94,6 +89,8 @@ def formataddr(pair, charset='utf-8'):
name.encode('ascii')
except UnicodeEncodeError:
if isinstance(charset, str):
# lazy import to improve module import time
from email.charset import Charset

@hauntsaninjahauntsaninjaOct 4, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I like this one, only fine with it since formataddr doesn't look too widely used (a lot of the time it's nice to pay these costs upfront, predictable performance is important, e.g. don't want the first request your webserver serves to be randomly slow)

@AlexWaygoodAlexWaygoodOct 4, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(a lot of the time it's nice to pay these costs upfront, predictable performance is important

agreed. On the other hand, though, the email package goes in quite heavily for lazy imports in some other places, so this does seem in keeping with that general philosophy:

cpython/Lib/email/__init__.py

Lines 28 to 61 in f786029

# Some convenience routines. Don't import Parser and Message as side-effects
# of importing email since those cascadingly import most of the rest of the
# email package.
defmessage_from_string(s, *args, **kws):
"""Parse a string into a Message object model.
Optional _class and strict are passed to the Parser constructor.
"""
fromemail.parserimportParser
returnParser(*args, **kws).parsestr(s)
defmessage_from_bytes(s, *args, **kws):
"""Parse a bytes string into a Message object model.
Optional _class and strict are passed to the Parser constructor.
"""
fromemail.parserimportBytesParser
returnBytesParser(*args, **kws).parsebytes(s)
defmessage_from_file(fp, *args, **kws):
"""Read a file and parse its contents into a Message object model.
Optional _class and strict are passed to the Parser constructor.
"""
fromemail.parserimportParser
returnParser(*args, **kws).parse(fp)
defmessage_from_binary_file(fp, *args, **kws):
"""Read a binary file and parse its contents into a Message object model.
Optional _class and strict are passed to the Parser constructor.
"""
fromemail.parserimportBytesParser
returnBytesParser(*args, **kws).parse(fp)

@AlexWaygoodAlexWaygoodOct 4, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be happy to change it so it's imported at the top of the function if you think that'd be better?

@hauntsaninjahauntsaninjaOct 4, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah that'd be worse, module level or here. I'm fine with this as is!

charset = Charset(charset)
encoded_name = charset.header_encode(name)
return "%s <%s>" % (encoded_name, address)
Expand DownExpand Up@@ -181,6 +178,11 @@ def make_msgid(idstring=None, domain=None):
portion of the message id after the '@'. It defaults to the locally
defined hostname.
"""
# Lazy imports to speedup module import time
# (no other functions in email.utils need these modules)
import random
import socket
Comment thread
AlexWaygood marked this conversation as resolved.

timeval = int(time.time()*100)
pid = os.getpid()
randint = random.getrandbits(64)
Expand Down
4 changes: 4 additions & 0 deletions Misc/NEWS.d/next/Library/2023-09-25-10-47-22.gh-issue-109653.TUHrId.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
Reduce the import time of :mod:`email.utils` by around 43%. This results in
the import time of :mod:`email.message` falling by around 18%, which in turn
reduces the import time of :mod:`importlib.metadata` by around 6%. Patch by
Alex Waygood.