Optimize HashBytes for 64-bit and remove dead code by AaronRobinsonMSFT · Pull Request #129640 · dotnet/runtime

AaronRobinsonMSFT · 2026-06-19T21:47:44Z

Optimize HashBytes (src/coreclr/inc/utilcode.h) to use xxHash32 primitives (XXHash32_QueueRound/XXHash32_MixFinal) processing 4 bytes at a time, replacing the original byte-at-a-time DJB2 hash.

HashBytes is used by the string literal interning path (EEUnicodeStringLiteralHashTableHelper::Hash) which hashes every string literal resolved during JIT compilation (ldstr token resolution). This makes it part of the steady-state cost for all managed applications.

The xxHash32 primitives are extracted from typehashingalgorithms.h into a new shared header src/coreclr/inc/dn_xxhash.h so they can be reused by both the type hashing and HashBytes paths.

Also removes unused dead code: EEUnicodeHashTableHelper, EEUnicodeStringHashTable, EEStringData::IsOnlyLowChars, HashStringN, HashiStringA, HashiStringN, and CaseInsensitiveStringCompareHash.

Measurement

A microbenchmark calling string.IsInterned() 200M times against a pool of 1000 non-interned strings was used to isolate the hashing cost. Profiled on macOS ARM64 (Apple M5) with a full Release build (clr+libs -c Release). Each row is the median of 5 runs.

Version	Median	vs Baseline
Upstream baseline (byte-at-a-time DJB2)	8.53s	—
xxHash32 primitives (QueueRound loop)	7.65s	-10.3%
xxHash32 class (uint32_t)	8.30s	-2.7%
xxHash32 class (uint64_t)	9.06s	+6.2%

The direct primitives approach was chosen because the xxHash class queue machinery (position tracking, branching, accumulator state) adds overhead that erases the gain when used for bulk byte-stream hashing. The primitives inline cleanly into a tight loop.

Note

This PR was generated with the assistance of GitHub Copilot.

On 64-bit hosts, process 8 bytes at a time using a multiply-xorshift mixing step instead of byte-at-a-time DJB2. The tail bytes (0-7) fall through to the original byte loop. 32-bit hosts are unchanged. Remove unused EEUnicodeHashTableHelper and EEUnicodeStringHashTable along with their method implementations. Fix stale comment in dynamicmethod.cpp that referenced the removed type. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

dotnet-policy-service · 2026-06-19T21:48:54Z

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

Copilot

Pull request overview

This PR updates CoreCLR’s internal hashing helpers by optimizing HashBytes on 64-bit hosts and removing unused Unicode/string hashing helper types, primarily impacting hash-table usage in the VM and shared hash utilities.

Changes:

Add a 64-bit HashBytes fast-path in utilcode.h that consumes data in 8-byte chunks before hashing the remaining tail bytes.
Remove the unused EEUnicodeHashTableHelper / EEUnicodeStringHashTable implementation and related dead code.
Remove unused string hashing helpers / traits and update a comment that referenced removed code.

Show a summary per file

File	Description
src/coreclr/vm/eehash.h	Removes unused Unicode hash table helper declaration/typedef.
src/coreclr/vm/eehash.cpp	Removes unused Unicode hash table helper implementation; keeps string literal helper.
src/coreclr/vm/dynamicmethod.cpp	Updates an in-code comment to no longer reference removed helper.
src/coreclr/inc/utilcode.h	Adds 64-bit chunking fast-path to `HashBytes`; removes unused string-hash helpers.
src/coreclr/inc/shash.h	Removes unused case-insensitive string compare/hash traits wrapper.

Copilot's findings

Files reviewed: 5/5 changed files
Comments generated: 1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rewrite HashBytes to use xxHash32 QueueRound/MixFinal primitives instead of byte-at-a-time DJB2. Extract xxHash32 code from typehashingalgorithms.h into a shared inc/dn_xxhash.h header. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot's findings

Files reviewed: 9/9 changed files
Comments generated: 3

- Include clrtypes.h for UINT32/DWORD/UINT_PTR/_rotl/HOST_64BIT - Use UINT_PTR cast in MixPointerIntoHash for consistency - Fix typo: mixin -> mix in Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Restructure xxHash32 as a template class parameterized by a traits type that provides GenerateGlobalSeed(). Move XXHash32_* free functions into static methods on xxHash<T> (MixEmptyState, MixState, QueueRound, Round, MixFinal). Combining helpers become function templates defaulting to xxHashDefaultTraits. Introduce three traits types: - xxHashDefaultTraits: random seed via minipal (general use) - xxHashBytesTraits: fixed seed 5381 for HashBytes - xxHashVersionResilientTraits: fixed seed 0 for stable type hashing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot's findings

Files reviewed: 12/12 changed files
Comments generated: 1

…ocess Use a function-local static to ensure the random seed is generated only once and reused for the lifetime of the process. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot's findings

Files reviewed: 12/12 changed files
Comments generated: 1

Ensures a single process-global seed by defining GenerateGlobalSeed in a translation unit rather than inline in the header. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Optimize `HashBytes` (`src/coreclr/inc/utilcode.h`) to use xxHash32 primitives (`XXHash32_QueueRound`/`XXHash32_MixFinal`) processing 4 bytes at a time, replacing the original byte-at-a-time DJB2 hash. `HashBytes` is used by the string literal interning path (`EEUnicodeStringLiteralHashTableHelper::Hash`) which hashes every string literal resolved during JIT compilation (`ldstr` token resolution). This makes it part of the steady-state cost for all managed applications. The xxHash32 primitives are extracted from `typehashingalgorithms.h` into a new shared header `src/coreclr/inc/dn_xxhash.h` so they can be reused by both the type hashing and `HashBytes` paths. Also removes unused dead code: `EEUnicodeHashTableHelper`, `EEUnicodeStringHashTable`, `EEStringData::IsOnlyLowChars`, `HashStringN`, `HashiStringA`, `HashiStringN`, and `CaseInsensitiveStringCompareHash`. ### Measurement A microbenchmark calling `string.IsInterned()` 200M times against a pool of 1000 non-interned strings was used to isolate the hashing cost. Profiled on macOS ARM64 (Apple M5) with a full Release build (`clr+libs -c Release`). Each row is the median of 5 runs. | Version | Median | vs Baseline | |---------|--------|-------------| | Upstream baseline (byte-at-a-time DJB2) | 8.53s | — | | xxHash32 primitives (QueueRound loop) | 7.65s | **-10.3%** | | xxHash32 class (uint32_t) | 8.30s | -2.7% | | xxHash32 class (uint64_t) | 9.06s | +6.2% | The direct primitives approach was chosen because the `xxHash` class queue machinery (position tracking, branching, accumulator state) adds overhead that erases the gain when used for bulk byte-stream hashing. The primitives inline cleanly into a tight loop. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

AaronRobinsonMSFT requested review from Copilot and jkotas June 19, 2026 21:47

github-actionsBot added the area-VM-coreclr label Jun 19, 2026

dotnet-policy-serviceBot assigned AaronRobinsonMSFT Jun 19, 2026

github-project-automationBot added this to AppModel Jun 19, 2026

Copilotstarted reviewing on behalf of AaronRobinsonMSFT June 19, 2026 21:48 View session

CopilotAI reviewed Jun 19, 2026
View reviewed changes

Comment threadsrc/coreclr/inc/utilcode.h Outdated

am11 reviewed Jun 19, 2026
View reviewed changes

Comment threadsrc/coreclr/vm/eehash.cpp

AaronRobinsonMSFTand others added 2 commits June 19, 2026 15:21

Fix build: remove last caller of 3-arg EEStringData constructor
8bff549
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CopilotAI review requested due to automatic review settings June 19, 2026 23:45

Copilotstarted reviewing on behalf of AaronRobinsonMSFT June 19, 2026 23:45 View session

CopilotAI reviewed Jun 19, 2026
View reviewed changes

Comment threadsrc/coreclr/inc/dn_xxhash.h
Comment threadsrc/coreclr/inc/dn_xxhash.h Outdated
Comment threadsrc/coreclr/inc/dn_xxhash.h Outdated

Address PR feedback: make dn_xxhash.h self-contained
f597777
- Include clrtypes.h for UINT32/DWORD/UINT_PTR/_rotl/HOST_64BIT - Use UINT_PTR cast in MixPointerIntoHash for consistency - Fix typo: mixin -> mix in Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

jkotas reviewed Jun 20, 2026
View reviewed changes

Comment threadsrc/coreclr/inc/dn_xxhash.h Outdated

CopilotAI review requested due to automatic review settings June 22, 2026 23:10

Copilotstarted reviewing on behalf of AaronRobinsonMSFT June 22, 2026 23:10 View session

CopilotAI reviewed Jun 22, 2026
View reviewed changes

Comment threadsrc/coreclr/inc/dn_xxhash.h Outdated

Make xxHashDefaultTraits::GenerateGlobalSeed() initialize once per pr…
408042b
…ocess Use a function-local static to ensure the random seed is generated only once and reused for the lifetime of the process. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

build-analysisBot mentioned this pull request Jun 23, 2026
Intermittent build failure in AfterSourceBuild: "Could not write state file" #76488
Closed

Merge remote-tracking branch 'upstream/main' into perf1
94c8993

CopilotAI review requested due to automatic review settings June 23, 2026 02:26

Copilotstarted reviewing on behalf of AaronRobinsonMSFT June 23, 2026 02:26 View session

AaronRobinsonMSFT requested a review from jkotas June 23, 2026 02:29

CopilotAI reviewed Jun 23, 2026
View reviewed changes

Comment threadsrc/coreclr/inc/dn_xxhash.h

Move xxHashDefaultTraits::GenerateGlobalSeed to dn_xxhash.cpp
3f08464
Ensures a single process-global seed by defining GenerateGlobalSeed in a translation unit rather than inline in the header. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

build-analysisBot mentioned this pull request Jun 23, 2026
"We stopped hearing from agent Azure Pipelines 32. Verify the agent machine is running and has a healthy network connection" dotnet/dnceng#1886
Open
3 tasks

AaronRobinsonMSFT requested a review from davidwrighton June 23, 2026 16:49

davidwrighton approved these changes Jun 23, 2026
View reviewed changes

Comment threadsrc/coreclr/vm/ceeload.cpp

AaronRobinsonMSFT merged commit 8e67632 into dotnet:mainJun 23, 2026
109 of 112 checks passed

github-project-automationBot moved this to Done in AppModel Jun 23, 2026

AaronRobinsonMSFT deleted the perf1 branch June 23, 2026 18:23

dotnet-maestroBot mentioned this pull request Jun 24, 2026
[main] Source code updates from dotnet/runtime dotnet/dotnet#7344
Merged

dotnet-milestone-botBot added this to the 11.0-preview6 milestone Jun 24, 2026

github-actionsBot locked and limited conversation to collaborators Jul 24, 2026

Navigation Menu

Uh oh!

Optimize HashBytes for 64-bit and remove dead code - #129640

Optimize HashBytes for 64-bit and remove dead code#129640
AaronRobinsonMSFT merged 8 commits into
dotnet:mainfrom
AaronRobinsonMSFT:perf1

AaronRobinsonMSFT commented Jun 19, 2026•
edited
Loading

Uh oh!

dotnet-policy-serviceBot commented Jun 19, 2026

Uh oh!

CopilotAI left a comment

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Uh oh!

Uh oh!

CopilotAI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

AaronRobinsonMSFT commented Jun 19, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Measurement

Uh oh!

dotnet-policy-serviceBot commented Jun 19, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

AaronRobinsonMSFT commented Jun 19, 2026•
edited
Loading