Skip to content

gh-117431: Optimize str.startswith - #117480

Closed
eendebakpt wants to merge 2 commits into
python:mainfrom
eendebakpt:tailmatch
Closed

gh-117431: Optimize str.startswith#117480
eendebakpt wants to merge 2 commits into
python:mainfrom
eendebakpt:tailmatch

Conversation

@eendebakpt

@eendebakpteendebakpt commented Apr 2, 2024

Copy link
Copy Markdown
Contributor

We apply two optimizations in tailmatch (which is used in both str.startswith and str.endswith).

  • For one and two character arguments we avoid a call to memcmp as all characters have already been checked
  • In the call to memcmp we can reduce the number of bytes compared since the first and last character have already been checked.

Notes:

Two possible optimizations not included in this PR:

  • For the single character case we still do some double work as PyUnicode_READ(kind_self, data_self, offset) == PyUnicode_READ(kind_sub, data_sub, 0) and PyUnicode_READ(kind_self, data_self, offset + end_sub) == PyUnicode_READ(kind_sub, data_sub, end_sub) are equal in that case. We can eliminate that by adding something like
int first_character_equal = PyUnicode_READ(kind_self, data_self, offset) == PyUnicode_READ(kind_sub, data_sub, 0)
if (PyUnicode_GET_LENGTH(substring)==1) {
return first_character_equal ;
...

This makes the code for the single character case a bit faster, but the code a bit more complex.

  • We can make the number of bytes compared even smaller, but we would have calculate a different offset which does not seem worth the effort.

Benchmark (on top of #117466): python -m timeit -s "s = 'abcdef'" "s.startswith('a')"

main: 10000000 loops, best of 5: 27.2 nsec per loop
PR: 10000000 loops, best of 5: 26.3 nsec per loop

@erlend-aasland

Copy link
Copy Markdown
Contributor

Could you add the other optimisations as separate commits?

@serhiy-storchaka

Copy link
Copy Markdown
Member

The difference between 27.2 and 26.3 ns is too small and can be the result of unrelated factors. I get a nanosecond variation when run the same command several times.

@erlend-aasland

Copy link
Copy Markdown
Contributor

The difference between 27.2 and 26.3 ns is too small and can be the result of unrelated factors. I get a nanosecond variation when run the same command several times.

Yes, so I'm curious about the other two mentioned optimisations that are not (yet) part of this PR. Perhaps they have a greater impact.

@eendebakpt

Copy link
Copy Markdown
ContributorAuthor

The difference between 27.2 and 26.3 ns is too small and can be the result of unrelated factors. I get a nanosecond variation when run the same command several times.

Yes, so I'm curious about the other two mentioned optimisations that are not (yet) part of this PR. Perhaps they have a greater impact.

I created a PR with the other approach: #117782.

@eendebakpt

Copy link
Copy Markdown
ContributorAuthor

Closing this in favor of the alternate PR.

@eendebakpt
eendebakpt deleted the tailmatch branch June 26, 2025 13:45
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@eendebakpt@erlend-aasland@serhiy-storchaka