Skip to content

child_process: serialize advanced IPC messages natively - #63933

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
anonrig:ipc-serdes-native
Jun 18, 2026
Merged

child_process: serialize advanced IPC messages natively#63933
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
anonrig:ipc-serdes-native

Conversation

@anonrig

Copy link
Copy Markdown
Member

The advanced child_process IPC serialization codec was implemented in
JavaScript (ChildProcessSerializer / ChildProcessDeserializer in
lib/internal/child_process/serialization.js). It allocated a wrapper
serializer/deserializer per message and crossed the JS/C++ boundary several
times for every message (writeHeader, writeValue, releaseBuffer,
readHeader, readValue, …).

This moves the codec into a native ipc_serdes binding that drives the V8
ValueSerializer/ValueDeserializer with a C++ delegate. The wire format is
preserved byte-for-byte: a big-endian uint32 length prefix followed by the
V8 payload, with ArrayBufferViews tagged as host objects so that Node
Buffers round-trip as Buffers rather than plain Uint8Arrays. The json
codec is intentionally left unchanged (its hot path, JSON.stringify/parse,
is already native).

Performance

Measured A/B on identical built arm64 release binaries (only this change
differs), benchmark/child_process/child-process-ipc-roundtrip.js,
round-trips/sec, average of 3 runs:

PayloadBefore (JS)After (native)Speedup
64 B~300,000/s~800,000/s2.7× (+166%)
1 KiB~272,000/s~616,000/s2.3× (+126%)
16 KiB~91,000/s~120,000/s1.3× (+32%)
64 KiB~30,000/s~35,000/s1.16× (+16%)

json mode is unchanged within noise (~543k → ~554k at 1 KiB).

The gain is largest for small messages, where the fixed per-message JavaScript
overhead (per-message serializer/deserializer allocation and the JS/C++
boundary crossings) dominated. It tapers for large messages, where the actual
serialization — already native in both versions — dominates. These are
codec/IPC-throughput numbers from a saturated round-trip; a real fork()
workload also pays for pipe I/O, the event loop and the user message handler,
so application-level gains will be smaller.

Verification

  • test/parallel/test-child-process-* and test/parallel/test-cluster-*
    (190+ tests) pass, including advanced-serialization, -largebuffer,
    -splitted-length-field and fork-advanced-header-serialization.
  • New cctest test/cctest/test_node_ipc_serdes.cc covers serialize/deserialize
    round-trips for primitives, objects, typed arrays and Buffers (including the
    Buffer-vs-Uint8Array distinction) and asserts the length-prefix framing; the
    full cctest suite passes.
  • cpplint, git-clang-format, eslint and tsc --strict (typings) are clean.

There is no observable behavior change and the IPC wire format is unchanged.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/gyp

@nodejs-github-botnodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Jun 15, 2026
@anonrig

Copy link
Copy Markdown
MemberAuthor

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can’t we use V8 serdes?

@panvapanva left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no observable behavior change and the IPC wire format is unchanged.

There are observable changes tho:

repro.js
// repro.js// Run: node repro.js'use strict';const{ fork }=require('node:child_process');constfs=require('node:fs');constv8=require('node:v8');const{ MessageChannel }=require('node:worker_threads');if(process.argv[2]==='inspect'){process.on('message',(value)=>{process.send({isBuffer: Buffer.isBuffer(value),constructorName: value?.constructor?.name,keys: Object.keys(value),visible: value?.visible,bytes: ArrayBuffer.isView(value) ?
Buffer.from(value.buffer,value.byteOffset,value.byteLength).toString('hex') :
undefined,});});}elseif(process.argv[2]==='bad-tag'){classBadTagSerializerextendsv8.DefaultSerializer{_writeHostObject(value){this.writeUint32(2);// old child_process codec only accepts 0 or 1returnsuper._writeHostObject(value);}}constser=newBadTagSerializer();ser.writeHeader();ser.writeValue(Buffer.from('x'));constpayload=ser.releaseBuffer();constframed=Buffer.allocUnsafe(4+payload.length);framed.writeUInt32BE(payload.length,0);payload.copy(framed,4);fs.writeSync(process.channel.fd,framed);}else{main();}functioninspect(value){returnnewPromise((resolve)=>{constchild=fork(__filename,['inspect'],{serialization: 'advanced',stdio: ['ignore','ignore','inherit','ipc'],});child.once('message',(message)=>{child.disconnect();resolve({ok: true, message });});child.once('exit',(code,signal)=>{resolve({ok: false, code, signal });});try{child.send(value);}catch(err){child.disconnect();resolve({ok: false,threw: err.message});}});}functionsendBadTag(){returnnewPromise((resolve)=>{constchild=fork(__filename,['bad-tag'],{serialization: 'advanced',stdio: ['ignore','ignore','inherit','ipc'],});functiononUncaughtException(err){process.removeListener('uncaughtException',onUncaughtException);resolve({rejected: true,error: err.code||err.message});}process.once('uncaughtException',onUncaughtException);child.once('message',(value)=>{process.removeListener('uncaughtException',onUncaughtException);resolve({accepted: true,isBuffer: Buffer.isBuffer(value),value: Buffer.isBuffer(value) ? value.toString() : value,});});});}functioncheck(name,passed,details){console.log(`${passed ? 'PASS' : 'FAIL'}: ${name}`);console.log(details);console.log();if(!passed)process.exitCode=1;}asyncfunctionmain(){console.log('Expected on release/current Node: all PASS');console.log('Regression on PR build: one or more FAIL\n');const{ port1, port2 }=newMessageChannel();port1.visible=1;Object.defineProperty(Object.prototype,'visible',{configurable: true,set(){thrownewError('setter called');},});constsetter=awaitinspect(port1);deleteObject.prototype.visible;port1.close();port2.close();check('host-object spread must not invoke inherited setters',setter.ok&&setter.message.visible===1&&setter.message.keys[0]==='visible',setter.threw ? `regression: child.send() threw "${setter.threw}"` :
`result: ${JSON.stringify(setter)}`,);constbuf=Buffer.from('abc');buf.constructor=Uint8Array;constbufResult=awaitinspect(buf);check('Buffer with constructor = Uint8Array keeps old classification',bufResult.ok&&bufResult.message.isBuffer===false,`expected Buffer.isBuffer(received) === false\nresult: ${JSON.stringify(bufResult)}`,);constuint8=newUint8Array([1,2,3]);uint8.constructor=Buffer;constuint8Result=awaitinspect(uint8);check('Uint8Array with constructor = Buffer keeps old classification',uint8Result.ok&&uint8Result.message.isBuffer===true,`expected Buffer.isBuffer(received) === true\nresult: ${JSON.stringify(uint8Result)}`,);constbadTag=awaitsendBadTag();check('invalid child_process host-object tag is rejected',badTag.rejected===true,badTag.rejected ? `rejected with: ${badTag.error}` :
`regression: malformed tag accepted: ${JSON.stringify(badTag)}`,);}

@anonrig

Copy link
Copy Markdown
MemberAuthor

@panva thanks for the thorough repro — you're right, those were real regressions, and they're fixed in ef73d8f:

  1. Inherited setters — non-view host objects are now copied with CreateDataProperty instead of Set, so inherited setters are no longer invoked (matching the old { ...object } spread).
  2. Buffer classification — a Node Buffer is now identified by value.constructor === Buffer (matching v8.DefaultSerializer._writeHostObject) instead of by its prototype, so reassigning .constructor classifies exactly as before: a Buffer with constructor = Uint8Array round-trips as a Uint8Array, and a Uint8Array with constructor = Buffer round-trips as a Buffer.
  3. Invalid tag — the deserializer now rejects any host-object tag other than 0/1 with ERR_INVALID_STATE, matching the previous assert(tag === kNotArrayBufferViewTag).

I added test/parallel/test-child-process-advanced-serialization-host-objects.js covering all four cases from your repro; it fails on the previous build and passes now.

@anonrig

Copy link
Copy Markdown
MemberAuthor

@mcollina it does use V8 serdes — the binding wraps v8::ValueSerializer/v8::ValueDeserializer, the same way the previous code used v8.DefaultSerializer/DefaultDeserializer. What's custom (and all this PR moves from JS into C++) is the thin child_process layer around them: the kArrayBufferViewTag/kNotArrayBufferViewTag host-object tagging, the JSON-like shallow copy for non-view host objects, and the 4-byte big-endian length framing — all needed to keep the wire format byte-identical for cross-version IPC. The motivation is removing the per-message JS allocations and JS↔C++ boundary crossings (benchmark numbers in the description), not replacing V8's serializer.

@anonrig
anonrig requested a review from lemireJune 16, 2026 01:02
@panva

Copy link
Copy Markdown
Member

@anonrig The fix should compare against a stable original Buffer constructor reference, not read it back from Buffer.prototype. This is still a regression, I'll let you be the judge of how big one.

'use strict';const{ fork }=require('node:child_process');if(process.argv[2]==='child'){process.on('message',(value)=>{process.send({isBuffer: Buffer.isBuffer(value),constructorName: value.constructor.name,keys: Object.keys(value),text: ArrayBuffer.isView(value) ?
Buffer.from(value.buffer,value.byteOffset,value.byteLength).toString() :
undefined,});});}else{console.log('Expected behavior: Buffer.prototype.constructor tampering affects advanced IPC classification.');console.log('On Node v26.3.0, the received value is a Uint8Array, not a Buffer.');console.log('Regression on the PR: the received value is still a Buffer.\n');constoriginal=Buffer.prototype.constructor;Buffer.prototype.constructor=Uint8Array;constchild=fork(__filename,['child'],{serialization: 'advanced',stdio: ['ignore','ignore','inherit','ipc'],});child.once('message',(message)=>{Buffer.prototype.constructor=original;constpassed=message.isBuffer===false&&message.constructorName==='Uint8Array'&&message.text==='abc';console.log(`${passed ? 'PASS' : 'FAIL'}: received classification`);console.log(message);if(!passed)process.exitCode=1;child.disconnect();});child.once('exit',()=>{Buffer.prototype.constructor=original;});child.send(Buffer.from('abc'));}

The `advanced` IPC serialization codec was implemented in JavaScript
(ChildProcessSerializer / ChildProcessDeserializer in
lib/internal/child_process/serialization.js). It allocated a wrapper
serializer/deserializer per message and crossed the JS/C++ boundary
several times for every message (writeHeader, writeValue, releaseBuffer,
readHeader, readValue and friends).
Move the codec into a native `ipc_serdes` binding that drives the V8
ValueSerializer/ValueDeserializer with a C++ delegate. The wire format
is preserved byte-for-byte: a big-endian uint32 length prefix followed
by the V8 payload, with ArrayBufferViews tagged as host objects so that
Node Buffers round-trip as Buffers rather than plain Uint8Arrays. The
JSON codec is left unchanged.
A cctest (test/cctest/test_node_ipc_serdes.cc) exercises the binding
directly, covering round-trips of primitives, objects, typed arrays and
Buffers (including the Buffer-vs-Uint8Array distinction) and asserting
the big-endian length-prefix framing.
Round-trip throughput
(benchmark/child_process/child-process-ipc-roundtrip):
payload before after change
64 B ~300k/s ~800k/s +166%
1 KiB ~272k/s ~616k/s +126%
16 KiB ~91k/s ~120k/s +32%
64 KiB ~30k/s ~35k/s +16%
The gain is largest for small messages, where per-message JavaScript
overhead dominated, and tapers for large messages, where the actual
serialization (already native) dominates.
Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
@anonrig

Copy link
Copy Markdown
MemberAuthor

@panva good catch — fixed in 557a596. The serializer now compares the view's constructor against the stable require('buffer').Buffer reference captured in serialization.js (passed into the native serialize()), instead of reading it back from Buffer.prototype.constructor. So tampering Buffer.prototype.constructor no longer fools the classifier: a genuine Buffer is then sent as a Uint8Array, matching v8.DefaultSerializer (and Node v26.x).

Your repro now passes (received Uint8Array, not Buffer), and I added that case to test/parallel/test-child-process-advanced-serialization-host-objects.js.

@anonrig
anonrig requested review from mcollina and panvaJune 17, 2026 15:25

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@anonriganonrig added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Jun 17, 2026
@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 17, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@anonriganonrig added the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 18, 2026
@nodejs-github-botnodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 18, 2026
@nodejs-github-bot
nodejs-github-bot merged commit a1074b8 into nodejs:mainJun 18, 2026
91 of 96 checks passed
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Landed in a1074b8

aduh95 pushed a commit that referenced this pull request Jun 20, 2026
The `advanced` IPC serialization codec was implemented in JavaScript
(ChildProcessSerializer / ChildProcessDeserializer in
lib/internal/child_process/serialization.js). It allocated a wrapper
serializer/deserializer per message and crossed the JS/C++ boundary
several times for every message (writeHeader, writeValue, releaseBuffer,
readHeader, readValue and friends).
Move the codec into a native `ipc_serdes` binding that drives the V8
ValueSerializer/ValueDeserializer with a C++ delegate. The wire format
is preserved byte-for-byte: a big-endian uint32 length prefix followed
by the V8 payload, with ArrayBufferViews tagged as host objects so that
Node Buffers round-trip as Buffers rather than plain Uint8Arrays. The
JSON codec is left unchanged.
A cctest (test/cctest/test_node_ipc_serdes.cc) exercises the binding
directly, covering round-trips of primitives, objects, typed arrays and
Buffers (including the Buffer-vs-Uint8Array distinction) and asserting
the big-endian length-prefix framing.
Round-trip throughput
(benchmark/child_process/child-process-ipc-roundtrip):
payload before after change
64 B ~300k/s ~800k/s +166%
1 KiB ~272k/s ~616k/s +126%
16 KiB ~91k/s ~120k/s +32%
64 KiB ~30k/s ~35k/s +16%
The gain is largest for small messages, where per-message JavaScript
overhead dominated, and tapers for large messages, where the actual
serialization (already native) dominates.
Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
PR-URL: #63933
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Jun 25, 2026
The `advanced` IPC serialization codec was implemented in JavaScript
(ChildProcessSerializer / ChildProcessDeserializer in
lib/internal/child_process/serialization.js). It allocated a wrapper
serializer/deserializer per message and crossed the JS/C++ boundary
several times for every message (writeHeader, writeValue, releaseBuffer,
readHeader, readValue and friends).
Move the codec into a native `ipc_serdes` binding that drives the V8
ValueSerializer/ValueDeserializer with a C++ delegate. The wire format
is preserved byte-for-byte: a big-endian uint32 length prefix followed
by the V8 payload, with ArrayBufferViews tagged as host objects so that
Node Buffers round-trip as Buffers rather than plain Uint8Arrays. The
JSON codec is left unchanged.
A cctest (test/cctest/test_node_ipc_serdes.cc) exercises the binding
directly, covering round-trips of primitives, objects, typed arrays and
Buffers (including the Buffer-vs-Uint8Array distinction) and asserting
the big-endian length-prefix framing.
Round-trip throughput
(benchmark/child_process/child-process-ipc-roundtrip):
payload before after change
64 B ~300k/s ~800k/s +166%
1 KiB ~272k/s ~616k/s +126%
16 KiB ~91k/s ~120k/s +32%
64 KiB ~30k/s ~35k/s +16%
The gain is largest for small messages, where per-message JavaScript
overhead dominated, and tapers for large messages, where the actual
serialization (already native) dominates.
Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
PR-URL: #63933
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Jul 22, 2026
The `advanced` IPC serialization codec was implemented in JavaScript
(ChildProcessSerializer / ChildProcessDeserializer in
lib/internal/child_process/serialization.js). It allocated a wrapper
serializer/deserializer per message and crossed the JS/C++ boundary
several times for every message (writeHeader, writeValue, releaseBuffer,
readHeader, readValue and friends).
Move the codec into a native `ipc_serdes` binding that drives the V8
ValueSerializer/ValueDeserializer with a C++ delegate. The wire format
is preserved byte-for-byte: a big-endian uint32 length prefix followed
by the V8 payload, with ArrayBufferViews tagged as host objects so that
Node Buffers round-trip as Buffers rather than plain Uint8Arrays. The
JSON codec is left unchanged.
A cctest (test/cctest/test_node_ipc_serdes.cc) exercises the binding
directly, covering round-trips of primitives, objects, typed arrays and
Buffers (including the Buffer-vs-Uint8Array distinction) and asserting
the big-endian length-prefix framing.
Round-trip throughput
(benchmark/child_process/child-process-ipc-roundtrip):
payload before after change
64 B ~300k/s ~800k/s +166%
1 KiB ~272k/s ~616k/s +126%
16 KiB ~91k/s ~120k/s +32%
64 KiB ~30k/s ~35k/s +16%
The gain is largest for small messages, where per-message JavaScript
overhead dominated, and tapers for large messages, where the actual
serialization (already native) dominates.
Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
PR-URL: #63933
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
aduh95 pushed a commit that referenced this pull request Jul 30, 2026
The `advanced` IPC serialization codec was implemented in JavaScript
(ChildProcessSerializer / ChildProcessDeserializer in
lib/internal/child_process/serialization.js). It allocated a wrapper
serializer/deserializer per message and crossed the JS/C++ boundary
several times for every message (writeHeader, writeValue, releaseBuffer,
readHeader, readValue and friends).
Move the codec into a native `ipc_serdes` binding that drives the V8
ValueSerializer/ValueDeserializer with a C++ delegate. The wire format
is preserved byte-for-byte: a big-endian uint32 length prefix followed
by the V8 payload, with ArrayBufferViews tagged as host objects so that
Node Buffers round-trip as Buffers rather than plain Uint8Arrays. The
JSON codec is left unchanged.
A cctest (test/cctest/test_node_ipc_serdes.cc) exercises the binding
directly, covering round-trips of primitives, objects, typed arrays and
Buffers (including the Buffer-vs-Uint8Array distinction) and asserting
the big-endian length-prefix framing.
Round-trip throughput
(benchmark/child_process/child-process-ipc-roundtrip):
payload before after change
64 B ~300k/s ~800k/s +166%
1 KiB ~272k/s ~616k/s +126%
16 KiB ~91k/s ~120k/s +32%
64 KiB ~30k/s ~35k/s +16%
The gain is largest for small messages, where per-message JavaScript
overhead dominated, and tapers for large messages, where the actual
serialization (already native) dominates.
Signed-off-by: Yagiz Nizipli <yagiz@nizipli.com>
PR-URL: #63933
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
ckerr added a commit to electron/electron that referenced this pull request Aug 4, 2026
Ref: nodejs/node#63933
Co-Authored-By: GitHub Copilot <copilot@github.com>
ckerr added a commit to electron/electron that referenced this pull request Aug 5, 2026
Ref: nodejs/node#63933
Co-Authored-By: GitHub Copilot <copilot@github.com>
ckerr added a commit to electron/electron that referenced this pull request Aug 5, 2026
Ref: nodejs/node#63933
Co-Authored-By: GitHub Copilot <copilot@github.com>
ckerr added a commit to electron/electron that referenced this pull request Aug 5, 2026
Ref: nodejs/node#63933
Co-Authored-By: GitHub Copilot <copilot@github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs that have at least one approval, no pending requests for changes, and a CI started.c++Issues and PRs that require attention from people who are familiar with C++.lib / srcIssues and PRs related to general changes in the lib or src directory.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@anonrig@nodejs-github-bot@panva@mcollina