lib: unify ICU and no-ICU TextDecoder by ChALkeR · Pull Request #61409 · nodejs/node

ChALkeR · 2026-01-17T14:14:58Z

Better reviewed with whitespace ignored

FixTextDecoder defaults to stream: true instead of false with null options #61411
Add support for fatal UTF-8 decoder in no-ICU version unless streaming is used
add utf-8 fast path to no-ICU version
reduce duplication of TextDecoder source
normalize how this[kHandle] was not set in no-ICU version
The refactor changes the encoding reported in the no-ICU version when encoding is recognized but unsupported: now the actual resolved encoding name is reported as unsupported when previously the original input string was reported.
I don't think this is major, and reporting the actual encoding name is better in that case.

This will allow further fixes and common fast path for UTF-16 / fixed impl for UTF-16 as string_decoder doesn't implement UTF-16 per spec.

UTF-16 is still broken in no-ICU version here, this PR does not address that.
Tracking: #61041

nodejs-github-bot · 2026-01-17T14:15:03Z

Review requested:

@nodejs/startup

codecov · 2026-01-18T01:25:53Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.78%. Comparing base (77e8d44) to head (54e7ba2).
⚠️ Report is 47 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #61409 +/- ##
==========================================
- Coverage 89.87% 89.78% -0.09% 
==========================================
Files 671 672 +1 Lines 203178 203876 +698 Branches 39062 39186 +124 ==========================================
+ Hits 182599 183051 +452 - Misses 12926 13142 +216 - Partials 7653 7683 +30

Files with missing lines	Coverage Δ
lib/internal/encoding.js	`100.00% <100.00%> (ø)`

... and 70 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

avivkeller · 2026-01-19T23:12:28Z

+let StringDecoder;
+function lazyStringDecoder() {
+ if (StringDecoder === undefined)
+ ({ StringDecoder } = require('string_decoder'));
+ return StringDecoder;
+}


There's a lazy utility in the internal utils.

@avivkeller this is not new code, it's just moved to an outer scope
Ideally all that should go away with follow-up fixes, string_decoder path is invalid anyway
I don't think it's worth refactoring it further

I still think we should use the new lazy module. Regardless of this being new code or not, git sees it as a new code.

@anonrig Next two prs will remove this anyway. This is going in iterative steps, the intention of this PR is to make rewrite easier, not to be that rewrite. I don't think it should be blocked on rewriting a code block that wasn't introduced here and is gonna be removed.

#61549 removes string_decoder usage for UTF-8
For UTF-16 it's broken currently and will be refactored in a similar way in once #61549 lands, then string_decoder import will be just removed here

ChALkeR · 2026-01-21T19:25:46Z

Rebased, no actual changes

gurgunday

lgtm

nodejs-github-bot · 2026-01-25T01:19:35Z

CI: https://ci.nodejs.org/job/node-test-pull-request/71011/

nodejs-github-bot · 2026-01-25T18:04:44Z

CI: https://ci.nodejs.org/job/node-test-pull-request/71031/

richardlau · 2026-01-25T22:33:14Z

The withoutintl test failure looks related.
https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu2404_sharedlibs_withoutintl_x64/54468/console

19:12:10 not ok 3707 parallel/test-whatwg-encoding-custom-textdecoder
19:12:10 ---
19:12:10 duration_ms: 70.66700
19:12:10 severity: fail
19:12:10 exitcode: 1
19:12:10 stack: |-
19:12:10 node:internal/assert/utils:146
19:12:10 throw error;
19:12:10 ^
19:12:10 19:12:10 AssertionError [ERR_ASSERTION]: Missing expected exception (TypeError).
19:12:10 at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-whatwg-encoding-custom-textdecoder.js:88:10)
19:12:10 at Module._compile (node:internal/modules/cjs/loader:1803:14)
19:12:10 at Object..js (node:internal/modules/cjs/loader:1934:10)
19:12:10 at Module.load (node:internal/modules/cjs/loader:1524:32)
19:12:10 at Module._load (node:internal/modules/cjs/loader:1326:12)
19:12:10 at TracingChannel.traceSync (node:diagnostics_channel:328:14)
19:12:10 at wrapModuleLoad (node:internal/modules/cjs/loader:245:24)
19:12:10 at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:154:5)
19:12:10 at node:internal/main/run_main_module:33:47 {
19:12:10 generatedMessage: false,
19:12:10 code: 'ERR_ASSERTION',
19:12:10 actual: undefined,
19:12:10 expected: {
19:12:10 code: 'ERR_NO_ICU',
19:12:10 name: 'TypeError',
19:12:10 message: '"fatal" option is not supported on Node.js compiled without ICU'
19:12:10 },
19:12:10 operator: 'throws',
19:12:10 diff: 'simple'
19:12:10 }
19:12:10 19:12:10 Node.js v26.0.0-pre
19:12:10 ...

ChALkeR · 2026-01-26T03:26:06Z

@richardlau indeed, I missed that, thanks! I'll update the test

nodejs-github-bot · 2026-01-26T12:56:14Z

CI: https://ci.nodejs.org/job/node-test-pull-request/71041/

nodejs-github-bot · 2026-01-26T21:00:33Z

CI: https://ci.nodejs.org/job/node-test-pull-request/71050/

nodejs-github-bot · 2026-01-27T14:34:24Z

Landed in e155415

PR-URL: #61409 Reviewed-By: Aviv Keller <me@aviv.sh> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day> Reviewed-By: Richard Lau <richard.lau@ibm.com>

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Jan 17, 2026

ChALkeR changed the title ~~Chalker/decoder/unify/1~~lib: unify ICU and no-ICU TextDecoderJan 17, 2026

ChALkeR force-pushed the chalker/decoder/unify/1 branch 2 times, most recently from 367908c to 5a2317fCompare January 17, 2026 14:24

ChALkeR commented Jan 17, 2026
View reviewed changes

Comment threadlib/internal/encoding.js Outdated

ChALkeR mentioned this pull request Jan 17, 2026
TextDecoder defaults to stream: true instead of false with null options #61411
Closed

ChALkeR force-pushed the chalker/decoder/unify/1 branch 5 times, most recently from 4d6ff8f to b2e1eceCompare January 17, 2026 23:54

ChALkeR marked this pull request as ready for review January 17, 2026 23:55

ChALkeR force-pushed the chalker/decoder/unify/1 branch 7 times, most recently from adb2410 to fb0ca32Compare January 18, 2026 00:29

avivkeller reviewed Jan 19, 2026
View reviewed changes

ChALkeR requested a review from avivkeller January 20, 2026 00:04

avivkeller approved these changes Jan 20, 2026
View reviewed changes

ChALkeR force-pushed the chalker/decoder/unify/1 branch from fb0ca32 to 1204484Compare January 21, 2026 13:55

lib: unify ICU and no-ICU TextDecoder
e74c6fe

ChALkeR force-pushed the chalker/decoder/unify/1 branch from 1204484 to e74c6feCompare January 21, 2026 19:25

anonrig approved these changes Jan 23, 2026
View reviewed changes

gurgunday approved these changes Jan 25, 2026
View reviewed changes

gurgunday added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 25, 2026

github-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 25, 2026

test: fix test expectations
54e7ba2

ChALkeR force-pushed the chalker/decoder/unify/1 branch from 701604e to 54e7ba2Compare January 26, 2026 08:19

anonrig approved these changes Jan 26, 2026
View reviewed changes

richardlau approved these changes Jan 26, 2026
View reviewed changes

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jan 27, 2026

nodejs-github-bot merged commit e155415 into nodejs:mainJan 27, 2026
70 checks passed

gurgunday mentioned this pull request Jan 27, 2026
src: move all 1-byte encodings to native #61118
Open

This was referenced Jan 27, 2026
lib: use utf8 fast path for streaming TextDecoder #61549
Closed
TextDecoder is wrong and very slow #61041
Open

github-actionsBot mentioned this pull request Feb 2, 2026
2026-02-03, Version 25.6.0 (Current) #61635
Merged

github-actionsBot mentioned this pull request Feb 22, 2026
2026-02-24, Version 24.14.0 'Krypton' (LTS) #61924
Merged

Uh oh!

Uh oh!

Conversation

ChALkeR commented Jan 17, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nodejs-github-bot commented Jan 17, 2026

Uh oh!

Uh oh!

codecovBot commented Jan 18, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

avivkellerJan 19, 2026

Choose a reason for hiding this comment

Uh oh!

ChALkeRJan 20, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anonrigJan 23, 2026

Choose a reason for hiding this comment

Uh oh!

ChALkeRJan 23, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChALkeRJan 27, 2026

Choose a reason for hiding this comment

Uh oh!

ChALkeR commented Jan 21, 2026

Uh oh!

gurgunday left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Jan 25, 2026

Uh oh!

nodejs-github-bot commented Jan 25, 2026

Uh oh!

richardlau commented Jan 25, 2026

Uh oh!

ChALkeR commented Jan 26, 2026

Uh oh!

nodejs-github-bot commented Jan 26, 2026

Uh oh!

nodejs-github-bot commented Jan 26, 2026

Uh oh!

Uh oh!

nodejs-github-bot commented Jan 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ChALkeR commented Jan 17, 2026•
edited
Loading

codecovBot commented Jan 18, 2026•
edited
Loading

ChALkeRJan 20, 2026•
edited
Loading

ChALkeRJan 23, 2026•
edited
Loading