Skip to content

bpo-44019: Implement operator.call(). - #27888

Merged
mdickinson merged 3 commits into
python:mainfrom
anntzer:operator.call
Sep 24, 2021
Merged

bpo-44019: Implement operator.call().#27888
mdickinson merged 3 commits into
python:mainfrom
anntzer:operator.call

Conversation

@anntzer

@anntzeranntzer commented Aug 22, 2021

Copy link
Copy Markdown
Contributor

Having operator.call(obj, arg) mean type(obj).__call__(obj, arg) is
consistent with the other dunder operators. The semantics with *args, **kwargs then follow naturally from the single-arg semantics.


I realize that I proposed different semantics in the original bug report, but that didn't even actually match to the proposed use case; the second proposal in the thread is the correct one. The semantics here work for at least two different use cases that I have seen:

  • Using concurrent.futures.Executor.map on a list of callables (executor.map(operator.call, callables)). In that case the performance doesn't actually matter much and a Python helper would in fact be fine.

  • Loading a "typed" csv file (cf. numpy.loadtxt), where each column has a separate type, e.g.

converters= [int, float, int, float]
lines= [ # Assume that we already `str.split` them.
["1", "2.2", "3", "4.4"],
["5", "6.6", "7", "8.8"],
]
[[*map(operator.call, converters, words)] forwordsinlines]
# faster than the list comprehension
[[conv(word) forconv, wordinzip(converters, words)] forwordsinlines]

Here the performance actually matters (with numpy, one may easily be loading millions of rows). A simple benchmark gives

$ ./python -mtimeit -s 'from operator import call; funcs = [int, float, float, int]; inputs = ["1", "2", "3", "4"]' -- '[*map(call, funcs, inputs)]'
500000 loops, best of 5: 761 nsec per loop
$ ./python -mtimeit -s 'from operator import call; funcs = [int, float, float, int]; inputs = ["1", "2", "3", "4"]' -- '[func(inp) for func, inp in zip(funcs, inputs)]'
200000 loops, best of 5: 1.07 usec per loop

i.e. map(operator.call, ...) gives a significant speedup.

https://bugs.python.org/issue44019

Comment threadLib/operator.py Outdated
Comment threadLib/test/test_operator.py Outdated
@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

@github-actionsgithub-actionsBot added the stale Stale PR or inactive for long period of time. label Sep 23, 2021
@anntzer

Copy link
Copy Markdown
ContributorAuthor

From my side, this is ready to go, afaict.

Comment threadDoc/library/operator.rst Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add / to the prototype.

Suggested change
.. function:: call(obj, *args, **kwargs)
.. function:: call(obj, /, *args, **kwargs)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, thanks for noticing.

Having `operator.call(obj, arg)` mean `type(obj).__call__(obj, arg)` is
consistent with the other dunder operators. The semantics with `*args,
**kwargs` then follow naturally from the single-arg semantics.
@merwok

Copy link
Copy Markdown
Member

The user experience isn’t good for reviewers when history is rewritten in a PR, so please do regular commits and pushes for CPython (see https://devguide.python.org/pullrequest/#submitting)

@vstinnervstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The implementation is correct. I'm not convinced of the usefulness of this change, but the benchmark is interesting. I let other core dev merging the change :-)

@github-actionsgithub-actionsBot removed the stale Stale PR or inactive for long period of time. label Sep 24, 2021
@anntzer

anntzer commented Sep 24, 2021

Copy link
Copy Markdown
ContributorAuthor

Apologies for the squash commit, I did not realize this was the policy here.

@mdickinsonmdickinson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This seems like a fairly natural addition to the operator module for me.

@mdickinson

Copy link
Copy Markdown
Member

Do we need a "what's new" entry for this?

@anntzer

Copy link
Copy Markdown
ContributorAuthor

There's a NEWS.d entry, did you mean anything else?

@mdickinson

Copy link
Copy Markdown
Member

@vstinner

Copy link
Copy Markdown
Member

Ah yes please, document the addition in What's New in Python 3.11.

@anntzer

Copy link
Copy Markdown
ContributorAuthor

Done.

@vstinner

Copy link
Copy Markdown
Member

Done.

I don't see any change on Doc/whatsnew/3.11.rst.

@anntzer

Copy link
Copy Markdown
ContributorAuthor

Comment threadDoc/whatsnew/3.11.rst
Comment on lines +204 to +206
* A new function ``operator.call`` has been added, such that
``operator.call(obj, *args, **kwargs) == obj(*args, **kwargs)``.
(Contributed by Antony Lee in :issue:`44019`.)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* A new function ``operator.call`` has been added, such that
``operator.call(obj, *args, **kwargs) == obj(*args, **kwargs)``.
(Contributed by Antony Lee in :issue:`44019`.)
* A new function ``operator.call`` has been added, such that
``operator.call(obj, *args, **kwargs) == obj(*args, **kwargs)``.
(Contributed by Antony Lee in :issue:`44019`.)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

woopsie, thanks, fixed.

@vstinnervstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I let another core dev merge this PR.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@anntzer@merwok@mdickinson@vstinner@the-knights-who-say-ni@bedevere-bot