Skip to content
This repository was archived by the owner on Jul 13, 2026. It is now read-only.

Replace Crc64 with XxHash64 in Files class - #335

Merged
jonathanpeppers merged 3 commits into
mainfrom
jonathanpeppers/replace-crc64-with-xxhash64
Apr 23, 2026
Merged

Replace Crc64 with XxHash64 in Files class#335
jonathanpeppers merged 3 commits into
mainfrom
jonathanpeppers/replace-crc64-with-xxhash64

Conversation

@jonathanpeppers

@jonathanpeppersjonathanpeppers commented Apr 22, 2026

Copy link
Copy Markdown
Member

Replaces the software Crc64 implementation with hardware-accelerated XxHash64 from System.IO.Hashing 10.0.7 for all hashing in the Files class (HashBytes, HashFile, HashStream).

The Crc64 class is retained — only its usage in Files is replaced.

Changes

  • Files.cs: HashBytes, HashFile(string), and HashStream now use XxHash64 instead of Crc64. Uses stackalloc with Span-based APIs for zero-allocation hashing. HashFile(string, HashAlgorithm) overload is unchanged for backward compat.
  • Files.cs: ToHexString(ReadOnlySpan<byte>) is now public. Uses stackalloc for small hashes (≤64 bytes) to avoid char[] allocation. The byte[] overload delegates to it.
  • Files.cs: HashFile overloads simplified to use File.OpenRead().
  • Microsoft.Android.Build.BaseTasks.csproj: Added System.IO.Hashing package reference.
  • Directory.Build.targets: Pinned System.IO.Hashing version to 10.0.7 (centralized, matching other packages).
  • FilesTests.cs: Fixed pre-existing bug where tests reassigned tempDir from a directory to a file path, causing TearDown to skip cleanup. All tests now use tempDir as a directory and write to local tempFile variables inside it.

Benchmark results (1 MB data, Intel i9-14900KF)

MethodBefore (Crc64)AllocAfter (XxHash64)AllocSpeedup
HashBytes468.2 μs232 B46.00 μs56 B~10x
HashStream482.9 μs232 B44.98 μs184 B~11x
HashFile764.9 μs474 B75.83 μs424 B~10x
HasFileChanged1,496.6 μs945 B163.23 μs848 B~9x

Depends on #334

@simonrozsival

Copy link
Copy Markdown
Member

That's very interesting build perf improvement. I will switch to xxhash64 in the trimmable type map.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jonathanpeppers
jonathanpeppersforce-pushed the jonathanpeppers/replace-crc64-with-xxhash64 branch 2 times, most recently from e2dcffb to 255c8eaCompareApril 23, 2026 15:02
Use hardware-accelerated XxHash64 from System.IO.Hashing 10.0.7 instead
of the software Crc64 implementation for file/stream/byte hashing in the
Files class. This provides 5-10x faster hashing for incremental build
change detection.
The Crc64 class is retained for other consumers.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jonathanpeppers
jonathanpeppersforce-pushed the jonathanpeppers/replace-crc64-with-xxhash64 branch 7 times, most recently from 20234cb to a3c0421CompareApril 23, 2026 15:24
@jonathanpeppers
jonathanpeppers marked this pull request as ready for review April 23, 2026 15:37
CopilotAI review requested due to automatic review settings April 23, 2026 15:37
@jonathanpeppersjonathanpeppers added the ready-to-review This PR is ready to review/merge. label Apr 23, 2026

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Replaces the Files hashing implementation from a software Crc64 to System.IO.Hashing.XxHash64 (pinned to 10.0.7) to improve performance in incremental-build hot paths, along with associated test and benchmark updates.

Changes:

  • Switch Files.HashBytes, HashFile(string), and HashStream to XxHash64 with Span/stackalloc usage.
  • Add System.IO.Hashing package reference + central version pin.
  • Fix FilesTests temp directory handling and update benchmark README numbers.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
src/Microsoft.Android.Build.BaseTasks/Files.csMoves hashing to XxHash64, adds Span-based ToHexString overload, and adjusts file opening for hashing.
src/Microsoft.Android.Build.BaseTasks/Microsoft.Android.Build.BaseTasks.csprojAdds System.IO.Hashing package reference.
Directory.Build.targetsPins System.IO.Hashing to version 10.0.7.
tests/Microsoft.Android.Build.BaseTasks-Tests/FilesTests.csFixes temp dir/file path mix-up so teardown consistently cleans up and tests use dedicated temp files.
tests/Xamarin.Android.Tools.Benchmarks/README.mdUpdates published benchmark results to reflect the new hashing implementation.

Comment threadsrc/Microsoft.Android.Build.BaseTasks/Files.cs Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jonathanpeppers
jonathanpeppersforce-pushed the jonathanpeppers/replace-crc64-with-xxhash64 branch from a3c0421 to 249ab0cCompareApril 23, 2026 15:45
@jonathanpeppers
jonathanpeppers merged commit e873716 into mainApr 23, 2026
2 checks passed
@jonathanpeppers
jonathanpeppers deleted the jonathanpeppers/replace-crc64-with-xxhash64 branch April 23, 2026 15:57
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

ready-to-reviewThis PR is ready to review/merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jonathanpeppers@simonrozsival