Skip to content

Simplify UTF-16 validation Vector128 codepath - #121981

Merged
tannergooding merged 1 commit into
dotnet:mainfrom
ylpoonlg:github-utf16-validation
Dec 2, 2025
Merged

Simplify UTF-16 validation Vector128 codepath#121981
tannergooding merged 1 commit into
dotnet:mainfrom
ylpoonlg:github-utf16-validation

Conversation

@ylpoonlg

Copy link
Copy Markdown
Contributor

Re-attempt at #121383.

Refactor the vectorized code path by combining the SSE2 "intrinsified" path with the original Vector128 algorithm. There are still some platform specific code (for AdvSimd), as it is difficult to fully rely on Vector128 APIs without sacrificing performance too much. The main issue is the lack of an instruction for Vector128.ExtractMostSignificantBits on Arm, so it is significantly slower when trying to force it to use the same mask format as the SSE2 algorithm. I have looked into the possibility of using IndexOf and Count etc, but they also use ExtractMostSignificantBits so it poses the same problem.
This PR tries to encapsulate this difference in a few helper methods so they can share the same code path for the main algorithm.

Performance wise, there is not as much improvements, but hopefully the code will be easier to maintain.

Arm Neoverse-V2:

MethodInputVersionMeanErrorRatio
GetByteCountEnglishAllAsciiBefore4.437 us0.0437 us1.000
GetByteCountEnglishAllAsciiAfter4.475 us0.1618 us1.009
GetByteCountEnglishMostlyAsciiBefore20.387 us0.1744 us1.000
GetByteCountEnglishMostlyAsciiAfter19.941 us0.1079 us0.978
GetByteCountChineseBefore9.145 us0.0072 us1.000
GetByteCountChineseAfter8.992 us0.0069 us0.983
GetByteCountCyrillicBefore7.936 us0.0095 us1.000
GetByteCountCyrillicAfter7.812 us0.0056 us0.984
GetByteCountGreekBefore10.077 us0.0106 us1.000
GetByteCountGreekAfter9.952 us0.0120 us0.988

Intel Sapphire Rapids:

MethodInputVersionMeanErrorRatio
GetByteCountEnglishAllAsciiBefore8.144 us0.3398 us1.000
GetByteCountEnglishAllAsciiAfter8.126 us0.2759 us0.998
GetByteCountEnglishMostlyAsciiBefore22.971 us0.4046 us1.000
GetByteCountEnglishMostlyAsciiAfter22.155 us0.9902 us0.964
GetByteCountChineseBefore10.582 us0.3425 us1.000
GetByteCountChineseAfter10.048 us0.2135 us0.950
GetByteCountCyrillicBefore9.222 us0.1874 us1.000
GetByteCountCyrillicAfter9.100 us0.2704 us0.987
GetByteCountGreekBefore11.802 us0.3551 us1.000
GetByteCountGreekAfter11.224 us0.3505 us0.951

Combine the SSE2 codepath with a more generic Vector128 algorithm.
AdvSimd is handled slightly differently to avoid using Vector128
ExtractMostSignificantBits, because there is no such equivalent
instruction on Arm so the performance would be very slow otherwise.
@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Nov 26, 2025
@github-actionsgithub-actionsBot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Nov 26, 2025
@ylpoonlg

Copy link
Copy Markdown
ContributorAuthor

cc @dotnet/arm64-contrib @a74nh@SwapnilGaikwad@tannergooding@EgorBo

@ylpoonlg
ylpoonlg marked this pull request as ready for review November 26, 2025 10:15
@tannergooding
tannergooding merged commit ac7db14 into dotnet:mainDec 2, 2025
144 checks passed
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jan 2, 2026
@EgorBo

Copy link
Copy Markdown
Member

Improvements: dotnet/perf-autofiling-issues#67360

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

community-contributionIndicates that the PR has been added by a community memberneeds-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@ylpoonlg@EgorBo@a74nh@tannergooding