Skip to content

gh-106751: Optimize SelectSelector.select() for many iteration case - #106879

Merged
corona10 merged 2 commits into
python:mainfrom
corona10:gh-106754-select
Jul 19, 2023
Merged

gh-106751: Optimize SelectSelector.select() for many iteration case#106879
corona10 merged 2 commits into
python:mainfrom
corona10:gh-106754-select

Conversation

@corona10

@corona10corona10 commented Jul 19, 2023

Copy link
Copy Markdown
Member

closes: gh-106751

@corona10

corona10 commented Jul 19, 2023

Copy link
Copy Markdown
MemberAuthor

Similar technique can be adopted.

Benchmark

importtimeitimportosfromselectorsimportSelectSelector, EVENT_WRITE, EVENT_READselect=SelectSelector()
for_inrange(100):
r, w=os.pipe()
os.write(w, b"a")
select.register(r, EVENT_READ)
elapsed=timeit.timeit(
"selector.select()",
number=100000,
globals={"selector": select},
)
print(f"elapsed: {elapsed}")

Result

AS-IS: elapsed -> 2.051669834000222
TO-BE: elapsed -> 1.9723411249997298 (1.04x faster)

@corona10

Copy link
Copy Markdown
MemberAuthor

cc @eendebakpt

Comment threadLib/selectors.py
events |= EVENT_WRITE

key = self._fd_to_key.get(fd)
r = frozenset(r)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: on my system frozenset is slightly slower than set, but the difference is very small

Comment threadLib/selectors.py
key = self._fd_to_key.get(fd)
r = frozenset(r)
w = frozenset(w)
rw = r | w

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rw is only used once. You assigned r | w to a variable for readability?

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize XXXSelector for many iterations of the event loop

4 participants

@corona10@methane@eendebakpt@bedevere-bot