Skip to content

src: add Latin1 fast path in StringBytes::Encode utf8 - #63385

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
mertcanaltin:mert/buffer-tostring-utf8-latin1
Jun 1, 2026
Merged

src: add Latin1 fast path in StringBytes::Encode utf8#63385
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
mertcanaltin:mert/buffer-tostring-utf8-latin1

Conversation

@mertcanaltin

@mertcanaltinmertcanaltin commented May 17, 2026

Copy link
Copy Markdown
Member

In StringBytes::Encode utf8, latin1-fits content was going through the UTF-16 path. I added a latin1 fast path that converts via simdutf and returns a one-byte V8 string. Benefits every Buffer.toString('utf8'), including fs.readFile/promises.readFile when they delegate to it.

@nodejs/performance @mcollina@anonrig@lemire@addaleax

Benchmark results:

➜ node git:(mert/buffer-tostring-utf8-latin1) ✗ node-benchmark-compare ./buffer-result.csv
confidence improvement accuracy (*) (**) (***)
buffers/buffer-tostring-utf8-latin1.js n=10000 content='ascii' size=1024 0.34 % ±1.92% ±2.56% ±3.35%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='ascii' size=16384 -0.57 % ±1.01% ±1.34% ±1.74%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='ascii' size=262144 0.80 % ±1.13% ±1.51% ±1.96%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='ascii' size=4194304 * 0.52 % ±0.46% ±0.61% ±0.80%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='ascii' size=64 9.85 % ±30.50% ±40.59% ±52.85%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1_then_cjk' size=1024 *** 5.25 % ±1.60% ±2.13% ±2.78%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1_then_cjk' size=16384 *** 24.32 % ±0.71% ±0.95% ±1.24%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1_then_cjk' size=262144 *** 24.33 % ±0.86% ±1.15% ±1.51%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1_then_cjk' size=4194304 -0.06 % ±0.53% ±0.71% ±0.92%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1_then_cjk' size=64 -2.30 % ±2.95% ±3.93% ±5.13%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1' size=1024 *** 9.02 % ±3.85% ±5.17% ±6.82%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1' size=16384 *** 34.52 % ±1.42% ±1.90% ±2.50%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1' size=262144 *** 30.06 % ±1.48% ±1.98% ±2.62%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1' size=4194304 0.09 % ±0.42% ±0.56% ±0.73%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='latin1' size=64 * 6.60 % ±5.00% ±6.70% ±8.82%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='utf8_mixed' size=1024 *** 6.47 % ±1.88% ±2.50% ±3.27%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='utf8_mixed' size=16384 *** 33.69 % ±1.62% ±2.16% ±2.82%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='utf8_mixed' size=262144 *** 47.29 % ±1.09% ±1.45% ±1.89%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='utf8_mixed' size=4194304 -0.31 % ±0.61% ±0.82% ±1.07%
buffers/buffer-tostring-utf8-latin1.js n=10000 content='utf8_mixed' size=64 * -2.47 % ±2.25% ±2.99% ±3.89%
Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 20 comparisons, you can thus expect the following amount of false-positive results:
1.00 false positives, when considering a 5% risk acceptance (*, **, ***),
0.20 false positives, when considering a 1% risk acceptance (**, ***),
0.02 false positives, when considering a 0.1% risk acceptance (***)
➜ node git:(mert/buffer-tostring-utf8-latin1) ✗

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels May 17, 2026
@codecov

codecovBot commented May 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 68.25397% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.06%. Comparing base (265679b) to head (55f6ae9).
⚠️ Report is 189 commits behind head on main.

Files with missing linesPatch %Lines
src/string_bytes.cc68.25%14 Missing and 6 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #63385 +/- ##
==========================================
+ Coverage 90.05% 90.06% +0.01% 
==========================================
Files 714 714 Lines 225628 225973 +345 Branches 42673 42761 +88 ==========================================
+ Hits 203198 203533 +335 + Misses 14225 14212 -13 - Partials 8205 8228 +23 
Files with missing linesCoverage Δ
src/string_bytes.cc73.21% <68.25%> (-1.42%)⬇️

... and 57 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mertcanaltin

Copy link
Copy Markdown
MemberAuthor

Also covers @addaleax's review on #63370, fs picks it up through the shared StringBytes path

Comment threadsrc/string_bytes.cc Outdated
Signed-off-by: Mert Can Altin <mertgold60@gmail.com>
@mertcanaltin
mertcanaltinforce-pushed the mert/buffer-tostring-utf8-latin1 branch from 73d9df0 to 55f6ae9CompareMay 19, 2026 18:07

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollinamcollina added the request-ci Add this label to start a Jenkins CI on a PR. label May 29, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 29, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@gurgundaygurgunday left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@gurgundaygurgunday added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 31, 2026
@mcollinamcollina added the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 1, 2026
@nodejs-github-botnodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 1, 2026
@nodejs-github-bot
nodejs-github-bot merged commit 6813080 into nodejs:mainJun 1, 2026
77 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in 6813080

aduh95 pushed a commit that referenced this pull request Jun 18, 2026
Signed-off-by: Mert Can Altin <mertgold60@gmail.com>
PR-URL: #63385
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
aduh95 pushed a commit that referenced this pull request Jun 25, 2026
Signed-off-by: Mert Can Altin <mertgold60@gmail.com>
PR-URL: #63385
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
aduh95 pushed a commit that referenced this pull request Jul 30, 2026
Signed-off-by: Mert Can Altin <mertgold60@gmail.com>
PR-URL: #63385
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs that have at least one approval, no pending requests for changes, and a CI started.bufferIssues and PRs related to the buffer subsystem.c++Issues and PRs that require attention from people who are familiar with C++.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@mertcanaltin@nodejs-github-bot@mcollina@addaleax@gurgunday