Skip to content

Enable Regex to use SearchValues<string> in compiled / source generator for IgnoreCase multi-strings - #98791

Merged
stephentoub merged 5 commits into
dotnet:mainfrom
stephentoub:usesearchvaluesstringinregex
Feb 23, 2024
Merged

Enable Regex to use SearchValues<string> in compiled / source generator for IgnoreCase multi-strings#98791
stephentoub merged 5 commits into
dotnet:mainfrom
stephentoub:usesearchvaluesstringinregex

Conversation

@stephentoub

@stephentoubstephentoub commented Feb 22, 2024•
edited
Loading

Copy link
Copy Markdown
Member

The analyzer determines a set of prefixes that can start any match, and then uses SearchValues<string> with IndexOfAny to find the next one from that set. It's currently only enabled for case-insensitive; we need to do some more perf validation before enabling for case-sensitive.

MethodToolchainPatternOptionsMeanRatio
Count\main\corerun.exe(?i)Sher[a-z]+|Hol[a-z]+Compiled691.66 us1.00
Count\pr\corerun.exe(?i)Sher[a-z]+|Hol[a-z]+Compiled130.90 us0.19
Count\main\corerun.exe(?i)Sherlock|Holmes|WatsonCompiled873.37 us1.00
Count\pr\corerun.exe(?i)Sherlock|Holmes|WatsonCompiled132.12 us0.15

Contributes to #85693

@stephentoub
Enable Regex to use SearchValues<string> in compiled / source generat…
98ec70e
…or TryFindNextStartingPosition
The analyzer determines a set of prefixes that can start any match, and then uses SearchValues with IndexOfAny to find the next one from that set. It's currently only enabled for case-insensitive; we need to do some more perf validation before enabling for case-sensitive.
MihaZupan
MihaZupan approved these changes Feb 22, 2024
View reviewed changes

@MihaZupanMihaZupan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

Comment threadsrc/libraries/System.Text.RegularExpressions/gen/RegexGenerator.Emitter.cs Outdated
Comment thread...ies/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexPrefixAnalyzer.cs Outdated
Comment thread...ies/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexPrefixAnalyzer.cs
Comment on lines +51 to +52
// Arbitrary string length limit (with some wiggle room) to avoid creating strings that are longer than is useful and consuming too much memory.
const int MaxPrefixLength = 8;

@MihaZupanMihaZupanFeb 22, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

longer than is useful and consuming too much memory

This is mainly about not spending too many resources on the analysis part, not about the cost of the potential SearchValues itself, right?

@stephentoubstephentoubFeb 22, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct

@stephentoubstephentoubFeb 22, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, "the longer than is useful" part was about SearchValues itself. Is that not the case?

Comment thread...ies/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexPrefixAnalyzer.cs
Comment thread...ies/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexPrefixAnalyzer.cs
Comment threadsrc/libraries/System.Text.RegularExpressions/tests/UnitTests/RegexPrefixAnalyzerTests.cs
@danmoseley

danmoseley commented Feb 22, 2024

Copy link
Copy Markdown
Contributor

A lot of work to get to this point! I guess I should remeasure the rust benchmarks with all the alternations.

@ghost

ghost commented Feb 22, 2024

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

Issue Details

The analyzer determines a set of prefixes that can start any match, and then uses SearchValues<string> with IndexOfAny to find the next one from that set. It's currently only enabled for case-insensitive; we need to do some more perf validation before enabling for case-sensitive.

MethodToolchainPatternOptionsMeanRatio
Count\main\corerun.exe(?i)Sher[a-z]+|Hol[a-z]+Compiled691.66 us1.00
Count\pr\corerun.exe(?i)Sher[a-z]+|Hol[a-z]+Compiled130.90 us0.19
Count\main\corerun.exe(?i)Sherlock|Holmes|WatsonCompiled873.37 us1.00
Count\pr\corerun.exe(?i)Sherlock|Holmes|WatsonCompiled132.12 us0.15

Contributes to #85693

Author:stephentoub
Assignees:stephentoub
Labels:

area-System.Text.RegularExpressions

Milestone:-

@stephentoub
stephentoub merged commit 99b7601 into dotnet:mainFeb 23, 2024
@stephentoub
stephentoub deleted the usesearchvaluesstringinregex branch February 23, 2024 11:28
@build-analysisbuild-analysisBot mentioned this pull request Feb 24, 2024
Closed
@MihaZupanMihaZupan mentioned this pull request Feb 25, 2024
Merged
@DrewScoggins

DrewScoggins commented Feb 27, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Possible improvements:
dotnet/perf-autofiling-issues#29847
dotnet/perf-autofiling-issues#29821
dotnet/perf-autofiling-issues#29805

This was referenced Feb 28, 2024
Closed
Closed
@stephentoubstephentoub mentioned this pull request Mar 11, 2024
Closed
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Mar 29, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@MihaZupanMihaZupanMihaZupan approved these changes
@danmoseleydanmoseleyAwaiting requested review from danmoseley
@joperezrjoperezrAwaiting requested review from joperezr

Assignees

@stephentoubstephentoub

Labels

area-System.Text.RegularExpressions

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@stephentoub@danmoseley@DrewScoggins@MihaZupan