deps: update simdutf to 9.1.0 and transcode UTF-8 writes in a single pass

[lemire]

Follow-up to #65324.

Update deps/v8/third_party/simdutf from 7.7.0 to 9.1.0 (upstream
2d4e249d, single-header release). StringBytes::Write() then uses
convert_utf16_to_utf8_with_replacement() so two-byte UTF-8 writes no
longer validate and copy through to_well_formed_utf16(). Unpaired
surrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.

benchmark/buffers/buffer-write-string-utf8.js on Linux x64 (Xeon Gold
6548N, 20 runs, significance as in compare.R):

len=65536 chars='two-byte-astral' *** +46.14% ±0.14%
len=65536 chars='two-byte-lone-surrogate' *** +31.33% ±0.13%
len=2048 chars='two-byte-astral' *** +39.85% ±0.14%
len=2048 chars='two-byte-lone-surrogate' *** +19.99% ±0.34%
len=65536 chars='two-byte' *** +9.79% ±0.12%
len=2048 chars='two-byte' *** +4.41% ±0.24%
one-byte strings / length <= 32 ~0%

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324

[nodejs-github-bot]

Review requested:

• @nodejs/security-wg
• @nodejs/v8-update

[github-actions]

Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721

[lemire]

@anonrig was first to push this feature to simdutf (convert_utf16_to_utf8_with_replacement), but it took some time to get it published and fast. By using the fast convert_utf16_to_utf8_with_replacement in simdutf 9.1, we can simplify the recent change by @codebytere while improving the performance in some cases.

[anonrig]

Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75971/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.19%. Comparing base (55e4ca3) to head (3da3299).

Additional details and impacted files

@@ Coverage Diff @@## main #65390 +/- ##
==========================================
+ Coverage 90.14% 90.19% +0.05% 
==========================================
Files 752 752 Lines 251870 251866 -4 Branches 47365 47407 +42 ==========================================
+ Hits 227037 227180 +143 + Misses 16177 16031 -146 + Partials 8656 8655 -1 

Files with missing lines	Coverage Δ
src/string_bytes.cc	75.00% <100.00%> (-0.24%)	⬇️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lemire]

@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update.

[lemire]

I'm closing this, but let us remember to come back to it.

For now, what @codebytere did works fine.

[lemire]

Follow-up to #65324.

Update deps/v8/third_party/simdutf from 7.7.0 to 9.1.0 (upstream
2d4e249d, single-header release). StringBytes::Write() then uses
convert_utf16_to_utf8_with_replacement() so two-byte UTF-8 writes no
longer validate and copy through to_well_formed_utf16(). Unpaired
surrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.

benchmark/buffers/buffer-write-string-utf8.js on Linux x64 (Xeon Gold
6548N, 20 runs, significance as in compare.R):

len=65536 chars='two-byte-astral' *** +46.14% ±0.14%
len=65536 chars='two-byte-lone-surrogate' *** +31.33% ±0.13%
len=2048 chars='two-byte-astral' *** +39.85% ±0.14%
len=2048 chars='two-byte-lone-surrogate' *** +19.99% ±0.34%
len=65536 chars='two-byte' *** +9.79% ±0.12%
len=2048 chars='two-byte' *** +4.41% ±0.24%
one-byte strings / length <= 32 ~0%

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324

[nodejs-github-bot]

Review requested:

• @nodejs/security-wg
• @nodejs/v8-update

[github-actions]

Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721

[lemire]

@anonrig was first to push this feature to simdutf (convert_utf16_to_utf8_with_replacement), but it took some time to get it published and fast. By using the fast convert_utf16_to_utf8_with_replacement in simdutf 9.1, we can simplify the recent change by @codebytere while improving the performance in some cases.

[anonrig]

Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75971/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.19%. Comparing base (55e4ca3) to head (3da3299).

Additional details and impacted files

@@ Coverage Diff @@## main #65390 +/- ##
==========================================
+ Coverage 90.14% 90.19% +0.05% 
==========================================
Files 752 752 Lines 251870 251866 -4 Branches 47365 47407 +42 ==========================================
+ Hits 227037 227180 +143 + Misses 16177 16031 -146 + Partials 8656 8655 -1 

Files with missing lines	Coverage Δ
src/string_bytes.cc	75.00% <100.00%> (-0.24%)	⬇️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lemire]

@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update.

[lemire]

I'm closing this, but let us remember to come back to it.

For now, what @codebytere did works fine.

[lemire]

Follow-up to #65324.

Update deps/v8/third_party/simdutf from 7.7.0 to 9.1.0 (upstream
2d4e249d, single-header release). StringBytes::Write() then uses
convert_utf16_to_utf8_with_replacement() so two-byte UTF-8 writes no
longer validate and copy through to_well_formed_utf16(). Unpaired
surrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.

benchmark/buffers/buffer-write-string-utf8.js on Linux x64 (Xeon Gold
6548N, 20 runs, significance as in compare.R):

len=65536 chars='two-byte-astral' *** +46.14% ±0.14%
len=65536 chars='two-byte-lone-surrogate' *** +31.33% ±0.13%
len=2048 chars='two-byte-astral' *** +39.85% ±0.14%
len=2048 chars='two-byte-lone-surrogate' *** +19.99% ±0.34%
len=65536 chars='two-byte' *** +9.79% ±0.12%
len=2048 chars='two-byte' *** +4.41% ±0.24%
one-byte strings / length <= 32 ~0%

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324

[nodejs-github-bot]

Review requested:

• @nodejs/security-wg
• @nodejs/v8-update

[github-actions]

Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721

[lemire]

@anonrig was first to push this feature to simdutf (convert_utf16_to_utf8_with_replacement), but it took some time to get it published and fast. By using the fast convert_utf16_to_utf8_with_replacement in simdutf 9.1, we can simplify the recent change by @codebytere while improving the performance in some cases.

[anonrig]

Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75971/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.19%. Comparing base (55e4ca3) to head (3da3299).

Additional details and impacted files

@@ Coverage Diff @@## main #65390 +/- ##
==========================================
+ Coverage 90.14% 90.19% +0.05% 
==========================================
Files 752 752 Lines 251870 251866 -4 Branches 47365 47407 +42 ==========================================
+ Hits 227037 227180 +143 + Misses 16177 16031 -146 + Partials 8656 8655 -1 

Files with missing lines	Coverage Δ
src/string_bytes.cc	75.00% <100.00%> (-0.24%)	⬇️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lemire]

@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update.

[lemire]

I'm closing this, but let us remember to come back to it.

For now, what @codebytere did works fine.

[lemire]

Follow-up to #65324.

Update deps/v8/third_party/simdutf from 7.7.0 to 9.1.0 (upstream
2d4e249d, single-header release). StringBytes::Write() then uses
convert_utf16_to_utf8_with_replacement() so two-byte UTF-8 writes no
longer validate and copy through to_well_formed_utf16(). Unpaired
surrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.

benchmark/buffers/buffer-write-string-utf8.js on Linux x64 (Xeon Gold
6548N, 20 runs, significance as in compare.R):

len=65536 chars='two-byte-astral' *** +46.14% ±0.14%
len=65536 chars='two-byte-lone-surrogate' *** +31.33% ±0.13%
len=2048 chars='two-byte-astral' *** +39.85% ±0.14%
len=2048 chars='two-byte-lone-surrogate' *** +19.99% ±0.34%
len=65536 chars='two-byte' *** +9.79% ±0.12%
len=2048 chars='two-byte' *** +4.41% ±0.24%
one-byte strings / length <= 32 ~0%

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324

[nodejs-github-bot]

Review requested:

• @nodejs/security-wg
• @nodejs/v8-update

[github-actions]

Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721

[lemire]

@anonrig was first to push this feature to simdutf (convert_utf16_to_utf8_with_replacement), but it took some time to get it published and fast. By using the fast convert_utf16_to_utf8_with_replacement in simdutf 9.1, we can simplify the recent change by @codebytere while improving the performance in some cases.

[anonrig]

Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75971/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.19%. Comparing base (55e4ca3) to head (3da3299).

Additional details and impacted files

@@ Coverage Diff @@## main #65390 +/- ##
==========================================
+ Coverage 90.14% 90.19% +0.05% 
==========================================
Files 752 752 Lines 251870 251866 -4 Branches 47365 47407 +42 ==========================================
+ Hits 227037 227180 +143 + Misses 16177 16031 -146 + Partials 8656 8655 -1 

Files with missing lines	Coverage Δ
src/string_bytes.cc	75.00% <100.00%> (-0.24%)	⬇️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lemire]

@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update.

[lemire]

I'm closing this, but let us remember to come back to it.

For now, what @codebytere did works fine.

[lemire]

Follow-up to #65324.

Update deps/v8/third_party/simdutf from 7.7.0 to 9.1.0 (upstream
2d4e249d, single-header release). StringBytes::Write() then uses
convert_utf16_to_utf8_with_replacement() so two-byte UTF-8 writes no
longer validate and copy through to_well_formed_utf16(). Unpaired
surrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.

benchmark/buffers/buffer-write-string-utf8.js on Linux x64 (Xeon Gold
6548N, 20 runs, significance as in compare.R):

len=65536 chars='two-byte-astral' *** +46.14% ±0.14%
len=65536 chars='two-byte-lone-surrogate' *** +31.33% ±0.13%
len=2048 chars='two-byte-astral' *** +39.85% ±0.14%
len=2048 chars='two-byte-lone-surrogate' *** +19.99% ±0.34%
len=65536 chars='two-byte' *** +9.79% ±0.12%
len=2048 chars='two-byte' *** +4.41% ±0.24%
one-byte strings / length <= 32 ~0%

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324

[nodejs-github-bot]

Review requested:

• @nodejs/security-wg
• @nodejs/v8-update

[github-actions]

Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721

[lemire]

@anonrig was first to push this feature to simdutf (convert_utf16_to_utf8_with_replacement), but it took some time to get it published and fast. By using the fast convert_utf16_to_utf8_with_replacement in simdutf 9.1, we can simplify the recent change by @codebytere while improving the performance in some cases.

[anonrig]

Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75971/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.19%. Comparing base (55e4ca3) to head (3da3299).

Additional details and impacted files

@@ Coverage Diff @@## main #65390 +/- ##
==========================================
+ Coverage 90.14% 90.19% +0.05% 
==========================================
Files 752 752 Lines 251870 251866 -4 Branches 47365 47407 +42 ==========================================
+ Hits 227037 227180 +143 + Misses 16177 16031 -146 + Partials 8656 8655 -1 

Files with missing lines	Coverage Δ
src/string_bytes.cc	75.00% <100.00%> (-0.24%)	⬇️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lemire]

@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update.

[lemire]

I'm closing this, but let us remember to come back to it.

For now, what @codebytere did works fine.

[lemire]

Follow-up to #65324.

Update deps/v8/third_party/simdutf from 7.7.0 to 9.1.0 (upstream
2d4e249d, single-header release). StringBytes::Write() then uses
convert_utf16_to_utf8_with_replacement() so two-byte UTF-8 writes no
longer validate and copy through to_well_formed_utf16(). Unpaired
surrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.

benchmark/buffers/buffer-write-string-utf8.js on Linux x64 (Xeon Gold
6548N, 20 runs, significance as in compare.R):

len=65536 chars='two-byte-astral' *** +46.14% ±0.14%
len=65536 chars='two-byte-lone-surrogate' *** +31.33% ±0.13%
len=2048 chars='two-byte-astral' *** +39.85% ±0.14%
len=2048 chars='two-byte-lone-surrogate' *** +19.99% ±0.34%
len=65536 chars='two-byte' *** +9.79% ±0.12%
len=2048 chars='two-byte' *** +4.41% ±0.24%
one-byte strings / length <= 32 ~0%

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324

[nodejs-github-bot]

Review requested:

• @nodejs/security-wg
• @nodejs/v8-update

[github-actions]

Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721

[lemire]

@anonrig was first to push this feature to simdutf (convert_utf16_to_utf8_with_replacement), but it took some time to get it published and fast. By using the fast convert_utf16_to_utf8_with_replacement in simdutf 9.1, we can simplify the recent change by @codebytere while improving the performance in some cases.

[anonrig]

Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75971/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.19%. Comparing base (55e4ca3) to head (3da3299).

Additional details and impacted files

@@ Coverage Diff @@## main #65390 +/- ##
==========================================
+ Coverage 90.14% 90.19% +0.05% 
==========================================
Files 752 752 Lines 251870 251866 -4 Branches 47365 47407 +42 ==========================================
+ Hits 227037 227180 +143 + Misses 16177 16031 -146 + Partials 8656 8655 -1 

Files with missing lines	Coverage Δ
src/string_bytes.cc	75.00% <100.00%> (-0.24%)	⬇️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lemire]

@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update.

[lemire]

I'm closing this, but let us remember to come back to it.

For now, what @codebytere did works fine.

[lemire]

Follow-up to #65324.

Update deps/v8/third_party/simdutf from 7.7.0 to 9.1.0 (upstream
2d4e249d, single-header release). StringBytes::Write() then uses
convert_utf16_to_utf8_with_replacement() so two-byte UTF-8 writes no
longer validate and copy through to_well_formed_utf16(). Unpaired
surrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.

benchmark/buffers/buffer-write-string-utf8.js on Linux x64 (Xeon Gold
6548N, 20 runs, significance as in compare.R):

len=65536 chars='two-byte-astral' *** +46.14% ±0.14%
len=65536 chars='two-byte-lone-surrogate' *** +31.33% ±0.13%
len=2048 chars='two-byte-astral' *** +39.85% ±0.14%
len=2048 chars='two-byte-lone-surrogate' *** +19.99% ±0.34%
len=65536 chars='two-byte' *** +9.79% ±0.12%
len=2048 chars='two-byte' *** +4.41% ±0.24%
one-byte strings / length <= 32 ~0%

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324

[nodejs-github-bot]

Review requested:

• @nodejs/security-wg
• @nodejs/v8-update

[github-actions]

Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721

[lemire]

@anonrig was first to push this feature to simdutf (convert_utf16_to_utf8_with_replacement), but it took some time to get it published and fast. By using the fast convert_utf16_to_utf8_with_replacement in simdutf 9.1, we can simplify the recent change by @codebytere while improving the performance in some cases.

[anonrig]

Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75971/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.19%. Comparing base (55e4ca3) to head (3da3299).

Additional details and impacted files

@@ Coverage Diff @@## main #65390 +/- ##
==========================================
+ Coverage 90.14% 90.19% +0.05% 
==========================================
Files 752 752 Lines 251870 251866 -4 Branches 47365 47407 +42 ==========================================
+ Hits 227037 227180 +143 + Misses 16177 16031 -146 + Partials 8656 8655 -1 

Files with missing lines	Coverage Δ
src/string_bytes.cc	75.00% <100.00%> (-0.24%)	⬇️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lemire]

@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update.

[lemire]

I'm closing this, but let us remember to come back to it.

For now, what @codebytere did works fine.

[lemire]

Follow-up to #65324.

Update deps/v8/third_party/simdutf from 7.7.0 to 9.1.0 (upstream
2d4e249d, single-header release). StringBytes::Write() then uses
convert_utf16_to_utf8_with_replacement() so two-byte UTF-8 writes no
longer validate and copy through to_well_formed_utf16(). Unpaired
surrogates still become U+FFFD. Truncating writes and strings of 32
code units or fewer still go through V8.

benchmark/buffers/buffer-write-string-utf8.js on Linux x64 (Xeon Gold
6548N, 20 runs, significance as in compare.R):

len=65536 chars='two-byte-astral' *** +46.14% ±0.14%
len=65536 chars='two-byte-lone-surrogate' *** +31.33% ±0.13%
len=2048 chars='two-byte-astral' *** +39.85% ±0.14%
len=2048 chars='two-byte-lone-surrogate' *** +19.99% ±0.34%
len=65536 chars='two-byte' *** +9.79% ±0.12%
len=2048 chars='two-byte' *** +4.41% ±0.24%
one-byte strings / length <= 32 ~0%

Refs: https://github.com/simdutf/simdutf/releases/tag/v9.1.0
Refs: #65324

[nodejs-github-bot]

Review requested:

• @nodejs/security-wg
• @nodejs/v8-update

[github-actions]

Benchmark GHA (buffers / buffer-write-string-utf8): https://github.com/nodejs/node/actions/runs/32170665721

[lemire]

@anonrig was first to push this feature to simdutf (convert_utf16_to_utf8_with_replacement), but it took some time to get it published and fast. By using the fast convert_utf16_to_utf8_with_replacement in simdutf 9.1, we can simplify the recent change by @codebytere while improving the performance in some cases.

[anonrig]

Unfortunately we need to wait for v8 to update its simdutf version or use our own simdutf as a dependency and make sure v8 uses our own simdutf

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/75971/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.19%. Comparing base (55e4ca3) to head (3da3299).

Additional details and impacted files

@@ Coverage Diff @@## main #65390 +/- ##
==========================================
+ Coverage 90.14% 90.19% +0.05% 
==========================================
Files 752 752 Lines 251870 251866 -4 Branches 47365 47407 +42 ==========================================
+ Hits 227037 227180 +143 + Misses 16177 16031 -146 + Partials 8656 8655 -1 

Files with missing lines	Coverage Δ
src/string_bytes.cc	75.00% <100.00%> (-0.24%)	⬇️

... and 43 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[lemire]

@anonrig Chromium is at 7.7.0. Let us see if we can't get them to update.

[lemire]

I'm closing this, but let us remember to come back to it.

For now, what @codebytere did works fine.