feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket - #41091

Merged
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes
Jun 3, 2026
Merged

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket#41091
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes

Conversation

@dcrousso

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso
Devin Rousso (dcrousso)force-pushed the feat-har-WebSocket-sizes branch 2 times, most recently from 2f828b3 to f0088f6CompareJune 2, 2026 15:58
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Comment threadtests/library/har-websocket.spec.ts Outdated
Comment threadtests/library/har-websocket.spec.ts Outdated
data,
});
updateTime(timestamp);
updateTransferSize(opcode, data);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting added to the response transfer size whereas it should go to the request.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh lol good point

eventsHelper.addEventListener(webSocket, network.WebSocket.Events.Request, ({ headers }: { headers: HeadersArray }) => {
this._recordRequestHeadersAndCookies(harEntry, headers);
if (!this._options.omitSizes)
harEntry.request.headersSize = network.requestHeadersSize(headers, webSocket.url(), method);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't request handshake headersSize be added to the request transfer size?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i originally thought that, but when looking at Request.prototype.async _sizes i saw that it's (manually) calculated as just the sum of the response headers and response body

MDN corroborates this as well <https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/transferSize>

The size includes the response header fields plus the response payload body (as defined by RFC7230).

@yury-sYury Semikhatsky (yury-s)Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand this correctly, har file contains _transferSize only for responses, so anything request related is irrelevant, right? I first thought that we'd have _transferSize for request as well, but I don't see such field in har.ts

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's my understanding as well

i think the closest equivalent for the request would be something like entry.request.headersSize + entry.request.bodySize

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically _transferSize is a non-standard extension

in WebKit, it's equivalent to entry.response.headersSize + entry.response.bodySize so unless Chromium is using it to expose some special info then it might just be a "shortcut" to avoid having to do that math 😅

Comment threadtests/library/har-websocket.spec.ts Outdated
{ type: 'receive', opcode: 2, data: incoming },
]);
for (const m of messages) {
expect(m.time).toBeGreaterThanOrEqual(beforeMs / 1000 - 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.time comes straight from CDP Network.webSocketFrameSent.timestamp in Chromium, which is a MonotonicTime (seconds since an arbitrary process origin), I don't think we can safely compare it with the wall time here. Do we use walltime in other places in har? If so, websockets should be aligned to, otherwise compare time delta here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

Network.webSocketWillSendHandshakeRequest.wallTime + (Network.webSocketFrame{Sent,Received}.timestamp - Network.webSocketWillSendHandshakeRequest.timestamp)

in Playwright we do this via WebSocket.prototype._toWallTime

Do we use walltime in other places in har?

there are a bunch of different time values within HAR

  • startDateTime: when the request first began (which is being changed to use a wallTime in feat(har): use engine timestamps instead of server Date #41082)
  • time: the total time taken from intial request to final response
  • timings: an object showing how much time was taken for each portion of the request
  • _monotonicTime: how long since the playwright process was started

i can further dig into whether the time for each WebSocket frame should be a WallTime or timestamp/MonotonicTime

at the very least, Firefox seems to send it as a WallTime (see the PR_Now() call inside WebSocketFrame)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

This may well be this case, but har file doesn't have to inherit that behavior. As a user of har I'd expect all wall time timestamps to be on the same timeline, so that I could compare them. If we need an extra step in Chromium to convert from time since webSocketWillSendHandshakeRequest to wall time, let's do it in the browser-specific code?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have this conversion code in WebSocket.prototype._toWallTime but yeah i can move it to Chromium and WebKit code instead

if (this._options.omitSizes)
return;

harEntry.response._transferSize = Math.max(0, harEntry.response._transferSize!) + ((opcode === 1) ? Buffer.byteLength(data, 'utf8') : base64ByteLength(data));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_transferSize counts payload bytes only, not frame overhead.
WebSocket framing adds a 2–14 byte header per frame (plus a 4-byte mask for client -> server frames). The computed value is payload-only, so it'll under-report vs. an actual on-wire transfer size. If we we can't compute it more accurately, it may be acceptable as an approximation, but worth a comment.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great point! ill look into this a bit more and see if i can get it more accurate (or as you suggest add a comment if i cant)

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.


// If the payload is more than 125 bytes then add either 2 or 8 more bytes depending on how long the payload is.
if (length > 2 ** 16)
headerSize += 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comment on where this number comes from?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is from the same URL as above but yeah i can repeat it here too

else if (length > 125)
headerSize += 2;

// The number of bytes needed for the mask are not included as that information is not exposed via any protocol.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chromium exposes mask: boolean field. Can mask size vary or it's always 4-bytes if present? Perhaps we can easily add similar information to wk and ff? I mean if we are anyway try to provide more accurate size estimate, we should probably support that too.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh neat! i didnt notice that. great catch! ill plumb that through and see what i can do to get any similar info from WebKit and Firefox

yes it's either 0 or 4 bytes depending on whether mask is false or true

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think we can always assume this as it seems to be required for the client to send a masked frame to the server <https://www.rfc-editor.org/info/rfc6455/#section-5.1>:

a client MUST mask all frames that it sends to the server

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7230 passed, 1103 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 flaky⚠️ [firefox-library] › library/har-websocket.spec.ts:313 › should record websocket connection failure `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:224 › cli codegen › should generate frame locators (4) `@firefox-ubuntu-22.04-node20`

39545 passed, 775 skipped


Merge workflow run.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@dcrousso@yury-s
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket - #41091

Merged
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes
Jun 3, 2026
Merged

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket#41091
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes

Conversation

@dcrousso

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso
Devin Rousso (dcrousso)force-pushed the feat-har-WebSocket-sizes branch 2 times, most recently from 2f828b3 to f0088f6CompareJune 2, 2026 15:58
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Comment threadtests/library/har-websocket.spec.ts Outdated
Comment threadtests/library/har-websocket.spec.ts Outdated
data,
});
updateTime(timestamp);
updateTransferSize(opcode, data);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting added to the response transfer size whereas it should go to the request.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh lol good point

eventsHelper.addEventListener(webSocket, network.WebSocket.Events.Request, ({ headers }: { headers: HeadersArray }) => {
this._recordRequestHeadersAndCookies(harEntry, headers);
if (!this._options.omitSizes)
harEntry.request.headersSize = network.requestHeadersSize(headers, webSocket.url(), method);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't request handshake headersSize be added to the request transfer size?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i originally thought that, but when looking at Request.prototype.async _sizes i saw that it's (manually) calculated as just the sum of the response headers and response body

MDN corroborates this as well <https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/transferSize>

The size includes the response header fields plus the response payload body (as defined by RFC7230).

@yury-sYury Semikhatsky (yury-s)Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand this correctly, har file contains _transferSize only for responses, so anything request related is irrelevant, right? I first thought that we'd have _transferSize for request as well, but I don't see such field in har.ts

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's my understanding as well

i think the closest equivalent for the request would be something like entry.request.headersSize + entry.request.bodySize

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically _transferSize is a non-standard extension

in WebKit, it's equivalent to entry.response.headersSize + entry.response.bodySize so unless Chromium is using it to expose some special info then it might just be a "shortcut" to avoid having to do that math 😅

Comment threadtests/library/har-websocket.spec.ts Outdated
{ type: 'receive', opcode: 2, data: incoming },
]);
for (const m of messages) {
expect(m.time).toBeGreaterThanOrEqual(beforeMs / 1000 - 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.time comes straight from CDP Network.webSocketFrameSent.timestamp in Chromium, which is a MonotonicTime (seconds since an arbitrary process origin), I don't think we can safely compare it with the wall time here. Do we use walltime in other places in har? If so, websockets should be aligned to, otherwise compare time delta here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

Network.webSocketWillSendHandshakeRequest.wallTime + (Network.webSocketFrame{Sent,Received}.timestamp - Network.webSocketWillSendHandshakeRequest.timestamp)

in Playwright we do this via WebSocket.prototype._toWallTime

Do we use walltime in other places in har?

there are a bunch of different time values within HAR

  • startDateTime: when the request first began (which is being changed to use a wallTime in feat(har): use engine timestamps instead of server Date #41082)
  • time: the total time taken from intial request to final response
  • timings: an object showing how much time was taken for each portion of the request
  • _monotonicTime: how long since the playwright process was started

i can further dig into whether the time for each WebSocket frame should be a WallTime or timestamp/MonotonicTime

at the very least, Firefox seems to send it as a WallTime (see the PR_Now() call inside WebSocketFrame)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

This may well be this case, but har file doesn't have to inherit that behavior. As a user of har I'd expect all wall time timestamps to be on the same timeline, so that I could compare them. If we need an extra step in Chromium to convert from time since webSocketWillSendHandshakeRequest to wall time, let's do it in the browser-specific code?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have this conversion code in WebSocket.prototype._toWallTime but yeah i can move it to Chromium and WebKit code instead

if (this._options.omitSizes)
return;

harEntry.response._transferSize = Math.max(0, harEntry.response._transferSize!) + ((opcode === 1) ? Buffer.byteLength(data, 'utf8') : base64ByteLength(data));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_transferSize counts payload bytes only, not frame overhead.
WebSocket framing adds a 2–14 byte header per frame (plus a 4-byte mask for client -> server frames). The computed value is payload-only, so it'll under-report vs. an actual on-wire transfer size. If we we can't compute it more accurately, it may be acceptable as an approximation, but worth a comment.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great point! ill look into this a bit more and see if i can get it more accurate (or as you suggest add a comment if i cant)

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.


// If the payload is more than 125 bytes then add either 2 or 8 more bytes depending on how long the payload is.
if (length > 2 ** 16)
headerSize += 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comment on where this number comes from?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is from the same URL as above but yeah i can repeat it here too

else if (length > 125)
headerSize += 2;

// The number of bytes needed for the mask are not included as that information is not exposed via any protocol.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chromium exposes mask: boolean field. Can mask size vary or it's always 4-bytes if present? Perhaps we can easily add similar information to wk and ff? I mean if we are anyway try to provide more accurate size estimate, we should probably support that too.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh neat! i didnt notice that. great catch! ill plumb that through and see what i can do to get any similar info from WebKit and Firefox

yes it's either 0 or 4 bytes depending on whether mask is false or true

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think we can always assume this as it seems to be required for the client to send a masked frame to the server <https://www.rfc-editor.org/info/rfc6455/#section-5.1>:

a client MUST mask all frames that it sends to the server

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7230 passed, 1103 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 flaky⚠️ [firefox-library] › library/har-websocket.spec.ts:313 › should record websocket connection failure `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:224 › cli codegen › should generate frame locators (4) `@firefox-ubuntu-22.04-node20`

39545 passed, 775 skipped


Merge workflow run.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@dcrousso@yury-s
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket - #41091

Merged
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes
Jun 3, 2026
Merged

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket#41091
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes

Conversation

@dcrousso

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso
Devin Rousso (dcrousso)force-pushed the feat-har-WebSocket-sizes branch 2 times, most recently from 2f828b3 to f0088f6CompareJune 2, 2026 15:58
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Comment threadtests/library/har-websocket.spec.ts Outdated
Comment threadtests/library/har-websocket.spec.ts Outdated
data,
});
updateTime(timestamp);
updateTransferSize(opcode, data);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting added to the response transfer size whereas it should go to the request.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh lol good point

eventsHelper.addEventListener(webSocket, network.WebSocket.Events.Request, ({ headers }: { headers: HeadersArray }) => {
this._recordRequestHeadersAndCookies(harEntry, headers);
if (!this._options.omitSizes)
harEntry.request.headersSize = network.requestHeadersSize(headers, webSocket.url(), method);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't request handshake headersSize be added to the request transfer size?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i originally thought that, but when looking at Request.prototype.async _sizes i saw that it's (manually) calculated as just the sum of the response headers and response body

MDN corroborates this as well <https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/transferSize>

The size includes the response header fields plus the response payload body (as defined by RFC7230).

@yury-sYury Semikhatsky (yury-s)Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand this correctly, har file contains _transferSize only for responses, so anything request related is irrelevant, right? I first thought that we'd have _transferSize for request as well, but I don't see such field in har.ts

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's my understanding as well

i think the closest equivalent for the request would be something like entry.request.headersSize + entry.request.bodySize

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically _transferSize is a non-standard extension

in WebKit, it's equivalent to entry.response.headersSize + entry.response.bodySize so unless Chromium is using it to expose some special info then it might just be a "shortcut" to avoid having to do that math 😅

Comment threadtests/library/har-websocket.spec.ts Outdated
{ type: 'receive', opcode: 2, data: incoming },
]);
for (const m of messages) {
expect(m.time).toBeGreaterThanOrEqual(beforeMs / 1000 - 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.time comes straight from CDP Network.webSocketFrameSent.timestamp in Chromium, which is a MonotonicTime (seconds since an arbitrary process origin), I don't think we can safely compare it with the wall time here. Do we use walltime in other places in har? If so, websockets should be aligned to, otherwise compare time delta here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

Network.webSocketWillSendHandshakeRequest.wallTime + (Network.webSocketFrame{Sent,Received}.timestamp - Network.webSocketWillSendHandshakeRequest.timestamp)

in Playwright we do this via WebSocket.prototype._toWallTime

Do we use walltime in other places in har?

there are a bunch of different time values within HAR

  • startDateTime: when the request first began (which is being changed to use a wallTime in feat(har): use engine timestamps instead of server Date #41082)
  • time: the total time taken from intial request to final response
  • timings: an object showing how much time was taken for each portion of the request
  • _monotonicTime: how long since the playwright process was started

i can further dig into whether the time for each WebSocket frame should be a WallTime or timestamp/MonotonicTime

at the very least, Firefox seems to send it as a WallTime (see the PR_Now() call inside WebSocketFrame)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

This may well be this case, but har file doesn't have to inherit that behavior. As a user of har I'd expect all wall time timestamps to be on the same timeline, so that I could compare them. If we need an extra step in Chromium to convert from time since webSocketWillSendHandshakeRequest to wall time, let's do it in the browser-specific code?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have this conversion code in WebSocket.prototype._toWallTime but yeah i can move it to Chromium and WebKit code instead

if (this._options.omitSizes)
return;

harEntry.response._transferSize = Math.max(0, harEntry.response._transferSize!) + ((opcode === 1) ? Buffer.byteLength(data, 'utf8') : base64ByteLength(data));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_transferSize counts payload bytes only, not frame overhead.
WebSocket framing adds a 2–14 byte header per frame (plus a 4-byte mask for client -> server frames). The computed value is payload-only, so it'll under-report vs. an actual on-wire transfer size. If we we can't compute it more accurately, it may be acceptable as an approximation, but worth a comment.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great point! ill look into this a bit more and see if i can get it more accurate (or as you suggest add a comment if i cant)

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.


// If the payload is more than 125 bytes then add either 2 or 8 more bytes depending on how long the payload is.
if (length > 2 ** 16)
headerSize += 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comment on where this number comes from?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is from the same URL as above but yeah i can repeat it here too

else if (length > 125)
headerSize += 2;

// The number of bytes needed for the mask are not included as that information is not exposed via any protocol.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chromium exposes mask: boolean field. Can mask size vary or it's always 4-bytes if present? Perhaps we can easily add similar information to wk and ff? I mean if we are anyway try to provide more accurate size estimate, we should probably support that too.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh neat! i didnt notice that. great catch! ill plumb that through and see what i can do to get any similar info from WebKit and Firefox

yes it's either 0 or 4 bytes depending on whether mask is false or true

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think we can always assume this as it seems to be required for the client to send a masked frame to the server <https://www.rfc-editor.org/info/rfc6455/#section-5.1>:

a client MUST mask all frames that it sends to the server

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7230 passed, 1103 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 flaky⚠️ [firefox-library] › library/har-websocket.spec.ts:313 › should record websocket connection failure `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:224 › cli codegen › should generate frame locators (4) `@firefox-ubuntu-22.04-node20`

39545 passed, 775 skipped


Merge workflow run.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@dcrousso@yury-s
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket - #41091

Merged
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes
Jun 3, 2026
Merged

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket#41091
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes

Conversation

@dcrousso

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso
Devin Rousso (dcrousso)force-pushed the feat-har-WebSocket-sizes branch 2 times, most recently from 2f828b3 to f0088f6CompareJune 2, 2026 15:58
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Comment threadtests/library/har-websocket.spec.ts Outdated
Comment threadtests/library/har-websocket.spec.ts Outdated
data,
});
updateTime(timestamp);
updateTransferSize(opcode, data);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting added to the response transfer size whereas it should go to the request.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh lol good point

eventsHelper.addEventListener(webSocket, network.WebSocket.Events.Request, ({ headers }: { headers: HeadersArray }) => {
this._recordRequestHeadersAndCookies(harEntry, headers);
if (!this._options.omitSizes)
harEntry.request.headersSize = network.requestHeadersSize(headers, webSocket.url(), method);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't request handshake headersSize be added to the request transfer size?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i originally thought that, but when looking at Request.prototype.async _sizes i saw that it's (manually) calculated as just the sum of the response headers and response body

MDN corroborates this as well <https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/transferSize>

The size includes the response header fields plus the response payload body (as defined by RFC7230).

@yury-sYury Semikhatsky (yury-s)Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand this correctly, har file contains _transferSize only for responses, so anything request related is irrelevant, right? I first thought that we'd have _transferSize for request as well, but I don't see such field in har.ts

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's my understanding as well

i think the closest equivalent for the request would be something like entry.request.headersSize + entry.request.bodySize

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically _transferSize is a non-standard extension

in WebKit, it's equivalent to entry.response.headersSize + entry.response.bodySize so unless Chromium is using it to expose some special info then it might just be a "shortcut" to avoid having to do that math 😅

Comment threadtests/library/har-websocket.spec.ts Outdated
{ type: 'receive', opcode: 2, data: incoming },
]);
for (const m of messages) {
expect(m.time).toBeGreaterThanOrEqual(beforeMs / 1000 - 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.time comes straight from CDP Network.webSocketFrameSent.timestamp in Chromium, which is a MonotonicTime (seconds since an arbitrary process origin), I don't think we can safely compare it with the wall time here. Do we use walltime in other places in har? If so, websockets should be aligned to, otherwise compare time delta here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

Network.webSocketWillSendHandshakeRequest.wallTime + (Network.webSocketFrame{Sent,Received}.timestamp - Network.webSocketWillSendHandshakeRequest.timestamp)

in Playwright we do this via WebSocket.prototype._toWallTime

Do we use walltime in other places in har?

there are a bunch of different time values within HAR

  • startDateTime: when the request first began (which is being changed to use a wallTime in feat(har): use engine timestamps instead of server Date #41082)
  • time: the total time taken from intial request to final response
  • timings: an object showing how much time was taken for each portion of the request
  • _monotonicTime: how long since the playwright process was started

i can further dig into whether the time for each WebSocket frame should be a WallTime or timestamp/MonotonicTime

at the very least, Firefox seems to send it as a WallTime (see the PR_Now() call inside WebSocketFrame)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

This may well be this case, but har file doesn't have to inherit that behavior. As a user of har I'd expect all wall time timestamps to be on the same timeline, so that I could compare them. If we need an extra step in Chromium to convert from time since webSocketWillSendHandshakeRequest to wall time, let's do it in the browser-specific code?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have this conversion code in WebSocket.prototype._toWallTime but yeah i can move it to Chromium and WebKit code instead

if (this._options.omitSizes)
return;

harEntry.response._transferSize = Math.max(0, harEntry.response._transferSize!) + ((opcode === 1) ? Buffer.byteLength(data, 'utf8') : base64ByteLength(data));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_transferSize counts payload bytes only, not frame overhead.
WebSocket framing adds a 2–14 byte header per frame (plus a 4-byte mask for client -> server frames). The computed value is payload-only, so it'll under-report vs. an actual on-wire transfer size. If we we can't compute it more accurately, it may be acceptable as an approximation, but worth a comment.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great point! ill look into this a bit more and see if i can get it more accurate (or as you suggest add a comment if i cant)

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.


// If the payload is more than 125 bytes then add either 2 or 8 more bytes depending on how long the payload is.
if (length > 2 ** 16)
headerSize += 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comment on where this number comes from?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is from the same URL as above but yeah i can repeat it here too

else if (length > 125)
headerSize += 2;

// The number of bytes needed for the mask are not included as that information is not exposed via any protocol.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chromium exposes mask: boolean field. Can mask size vary or it's always 4-bytes if present? Perhaps we can easily add similar information to wk and ff? I mean if we are anyway try to provide more accurate size estimate, we should probably support that too.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh neat! i didnt notice that. great catch! ill plumb that through and see what i can do to get any similar info from WebKit and Firefox

yes it's either 0 or 4 bytes depending on whether mask is false or true

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think we can always assume this as it seems to be required for the client to send a masked frame to the server <https://www.rfc-editor.org/info/rfc6455/#section-5.1>:

a client MUST mask all frames that it sends to the server

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7230 passed, 1103 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 flaky⚠️ [firefox-library] › library/har-websocket.spec.ts:313 › should record websocket connection failure `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:224 › cli codegen › should generate frame locators (4) `@firefox-ubuntu-22.04-node20`

39545 passed, 775 skipped


Merge workflow run.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@dcrousso@yury-s
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket - #41091

Merged
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes
Jun 3, 2026
Merged

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket#41091
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes

Conversation

@dcrousso

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso
Devin Rousso (dcrousso)force-pushed the feat-har-WebSocket-sizes branch 2 times, most recently from 2f828b3 to f0088f6CompareJune 2, 2026 15:58
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Comment threadtests/library/har-websocket.spec.ts Outdated
Comment threadtests/library/har-websocket.spec.ts Outdated
data,
});
updateTime(timestamp);
updateTransferSize(opcode, data);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting added to the response transfer size whereas it should go to the request.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh lol good point

eventsHelper.addEventListener(webSocket, network.WebSocket.Events.Request, ({ headers }: { headers: HeadersArray }) => {
this._recordRequestHeadersAndCookies(harEntry, headers);
if (!this._options.omitSizes)
harEntry.request.headersSize = network.requestHeadersSize(headers, webSocket.url(), method);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't request handshake headersSize be added to the request transfer size?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i originally thought that, but when looking at Request.prototype.async _sizes i saw that it's (manually) calculated as just the sum of the response headers and response body

MDN corroborates this as well <https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/transferSize>

The size includes the response header fields plus the response payload body (as defined by RFC7230).

@yury-sYury Semikhatsky (yury-s)Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand this correctly, har file contains _transferSize only for responses, so anything request related is irrelevant, right? I first thought that we'd have _transferSize for request as well, but I don't see such field in har.ts

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's my understanding as well

i think the closest equivalent for the request would be something like entry.request.headersSize + entry.request.bodySize

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically _transferSize is a non-standard extension

in WebKit, it's equivalent to entry.response.headersSize + entry.response.bodySize so unless Chromium is using it to expose some special info then it might just be a "shortcut" to avoid having to do that math 😅

Comment threadtests/library/har-websocket.spec.ts Outdated
{ type: 'receive', opcode: 2, data: incoming },
]);
for (const m of messages) {
expect(m.time).toBeGreaterThanOrEqual(beforeMs / 1000 - 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.time comes straight from CDP Network.webSocketFrameSent.timestamp in Chromium, which is a MonotonicTime (seconds since an arbitrary process origin), I don't think we can safely compare it with the wall time here. Do we use walltime in other places in har? If so, websockets should be aligned to, otherwise compare time delta here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

Network.webSocketWillSendHandshakeRequest.wallTime + (Network.webSocketFrame{Sent,Received}.timestamp - Network.webSocketWillSendHandshakeRequest.timestamp)

in Playwright we do this via WebSocket.prototype._toWallTime

Do we use walltime in other places in har?

there are a bunch of different time values within HAR

  • startDateTime: when the request first began (which is being changed to use a wallTime in feat(har): use engine timestamps instead of server Date #41082)
  • time: the total time taken from intial request to final response
  • timings: an object showing how much time was taken for each portion of the request
  • _monotonicTime: how long since the playwright process was started

i can further dig into whether the time for each WebSocket frame should be a WallTime or timestamp/MonotonicTime

at the very least, Firefox seems to send it as a WallTime (see the PR_Now() call inside WebSocketFrame)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

This may well be this case, but har file doesn't have to inherit that behavior. As a user of har I'd expect all wall time timestamps to be on the same timeline, so that I could compare them. If we need an extra step in Chromium to convert from time since webSocketWillSendHandshakeRequest to wall time, let's do it in the browser-specific code?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have this conversion code in WebSocket.prototype._toWallTime but yeah i can move it to Chromium and WebKit code instead

if (this._options.omitSizes)
return;

harEntry.response._transferSize = Math.max(0, harEntry.response._transferSize!) + ((opcode === 1) ? Buffer.byteLength(data, 'utf8') : base64ByteLength(data));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_transferSize counts payload bytes only, not frame overhead.
WebSocket framing adds a 2–14 byte header per frame (plus a 4-byte mask for client -> server frames). The computed value is payload-only, so it'll under-report vs. an actual on-wire transfer size. If we we can't compute it more accurately, it may be acceptable as an approximation, but worth a comment.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great point! ill look into this a bit more and see if i can get it more accurate (or as you suggest add a comment if i cant)

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.


// If the payload is more than 125 bytes then add either 2 or 8 more bytes depending on how long the payload is.
if (length > 2 ** 16)
headerSize += 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comment on where this number comes from?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is from the same URL as above but yeah i can repeat it here too

else if (length > 125)
headerSize += 2;

// The number of bytes needed for the mask are not included as that information is not exposed via any protocol.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chromium exposes mask: boolean field. Can mask size vary or it's always 4-bytes if present? Perhaps we can easily add similar information to wk and ff? I mean if we are anyway try to provide more accurate size estimate, we should probably support that too.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh neat! i didnt notice that. great catch! ill plumb that through and see what i can do to get any similar info from WebKit and Firefox

yes it's either 0 or 4 bytes depending on whether mask is false or true

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think we can always assume this as it seems to be required for the client to send a masked frame to the server <https://www.rfc-editor.org/info/rfc6455/#section-5.1>:

a client MUST mask all frames that it sends to the server

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7230 passed, 1103 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 flaky⚠️ [firefox-library] › library/har-websocket.spec.ts:313 › should record websocket connection failure `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:224 › cli codegen › should generate frame locators (4) `@firefox-ubuntu-22.04-node20`

39545 passed, 775 skipped


Merge workflow run.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@dcrousso@yury-s
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket - #41091

Merged
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes
Jun 3, 2026
Merged

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket#41091
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes

Conversation

@dcrousso

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso
Devin Rousso (dcrousso)force-pushed the feat-har-WebSocket-sizes branch 2 times, most recently from 2f828b3 to f0088f6CompareJune 2, 2026 15:58
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Comment threadtests/library/har-websocket.spec.ts Outdated
Comment threadtests/library/har-websocket.spec.ts Outdated
data,
});
updateTime(timestamp);
updateTransferSize(opcode, data);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting added to the response transfer size whereas it should go to the request.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh lol good point

eventsHelper.addEventListener(webSocket, network.WebSocket.Events.Request, ({ headers }: { headers: HeadersArray }) => {
this._recordRequestHeadersAndCookies(harEntry, headers);
if (!this._options.omitSizes)
harEntry.request.headersSize = network.requestHeadersSize(headers, webSocket.url(), method);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't request handshake headersSize be added to the request transfer size?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i originally thought that, but when looking at Request.prototype.async _sizes i saw that it's (manually) calculated as just the sum of the response headers and response body

MDN corroborates this as well <https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/transferSize>

The size includes the response header fields plus the response payload body (as defined by RFC7230).

@yury-sYury Semikhatsky (yury-s)Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand this correctly, har file contains _transferSize only for responses, so anything request related is irrelevant, right? I first thought that we'd have _transferSize for request as well, but I don't see such field in har.ts

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's my understanding as well

i think the closest equivalent for the request would be something like entry.request.headersSize + entry.request.bodySize

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically _transferSize is a non-standard extension

in WebKit, it's equivalent to entry.response.headersSize + entry.response.bodySize so unless Chromium is using it to expose some special info then it might just be a "shortcut" to avoid having to do that math 😅

Comment threadtests/library/har-websocket.spec.ts Outdated
{ type: 'receive', opcode: 2, data: incoming },
]);
for (const m of messages) {
expect(m.time).toBeGreaterThanOrEqual(beforeMs / 1000 - 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.time comes straight from CDP Network.webSocketFrameSent.timestamp in Chromium, which is a MonotonicTime (seconds since an arbitrary process origin), I don't think we can safely compare it with the wall time here. Do we use walltime in other places in har? If so, websockets should be aligned to, otherwise compare time delta here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

Network.webSocketWillSendHandshakeRequest.wallTime + (Network.webSocketFrame{Sent,Received}.timestamp - Network.webSocketWillSendHandshakeRequest.timestamp)

in Playwright we do this via WebSocket.prototype._toWallTime

Do we use walltime in other places in har?

there are a bunch of different time values within HAR

  • startDateTime: when the request first began (which is being changed to use a wallTime in feat(har): use engine timestamps instead of server Date #41082)
  • time: the total time taken from intial request to final response
  • timings: an object showing how much time was taken for each portion of the request
  • _monotonicTime: how long since the playwright process was started

i can further dig into whether the time for each WebSocket frame should be a WallTime or timestamp/MonotonicTime

at the very least, Firefox seems to send it as a WallTime (see the PR_Now() call inside WebSocketFrame)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

This may well be this case, but har file doesn't have to inherit that behavior. As a user of har I'd expect all wall time timestamps to be on the same timeline, so that I could compare them. If we need an extra step in Chromium to convert from time since webSocketWillSendHandshakeRequest to wall time, let's do it in the browser-specific code?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have this conversion code in WebSocket.prototype._toWallTime but yeah i can move it to Chromium and WebKit code instead

if (this._options.omitSizes)
return;

harEntry.response._transferSize = Math.max(0, harEntry.response._transferSize!) + ((opcode === 1) ? Buffer.byteLength(data, 'utf8') : base64ByteLength(data));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_transferSize counts payload bytes only, not frame overhead.
WebSocket framing adds a 2–14 byte header per frame (plus a 4-byte mask for client -> server frames). The computed value is payload-only, so it'll under-report vs. an actual on-wire transfer size. If we we can't compute it more accurately, it may be acceptable as an approximation, but worth a comment.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great point! ill look into this a bit more and see if i can get it more accurate (or as you suggest add a comment if i cant)

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.


// If the payload is more than 125 bytes then add either 2 or 8 more bytes depending on how long the payload is.
if (length > 2 ** 16)
headerSize += 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comment on where this number comes from?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is from the same URL as above but yeah i can repeat it here too

else if (length > 125)
headerSize += 2;

// The number of bytes needed for the mask are not included as that information is not exposed via any protocol.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chromium exposes mask: boolean field. Can mask size vary or it's always 4-bytes if present? Perhaps we can easily add similar information to wk and ff? I mean if we are anyway try to provide more accurate size estimate, we should probably support that too.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh neat! i didnt notice that. great catch! ill plumb that through and see what i can do to get any similar info from WebKit and Firefox

yes it's either 0 or 4 bytes depending on whether mask is false or true

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think we can always assume this as it seems to be required for the client to send a masked frame to the server <https://www.rfc-editor.org/info/rfc6455/#section-5.1>:

a client MUST mask all frames that it sends to the server

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7230 passed, 1103 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 flaky⚠️ [firefox-library] › library/har-websocket.spec.ts:313 › should record websocket connection failure `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:224 › cli codegen › should generate frame locators (4) `@firefox-ubuntu-22.04-node20`

39545 passed, 775 skipped


Merge workflow run.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@dcrousso@yury-s
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket - #41091

Merged
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes
Jun 3, 2026
Merged

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket#41091
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes

Conversation

@dcrousso

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso
Devin Rousso (dcrousso)force-pushed the feat-har-WebSocket-sizes branch 2 times, most recently from 2f828b3 to f0088f6CompareJune 2, 2026 15:58
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Comment threadtests/library/har-websocket.spec.ts Outdated
Comment threadtests/library/har-websocket.spec.ts Outdated
data,
});
updateTime(timestamp);
updateTransferSize(opcode, data);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting added to the response transfer size whereas it should go to the request.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh lol good point

eventsHelper.addEventListener(webSocket, network.WebSocket.Events.Request, ({ headers }: { headers: HeadersArray }) => {
this._recordRequestHeadersAndCookies(harEntry, headers);
if (!this._options.omitSizes)
harEntry.request.headersSize = network.requestHeadersSize(headers, webSocket.url(), method);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't request handshake headersSize be added to the request transfer size?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i originally thought that, but when looking at Request.prototype.async _sizes i saw that it's (manually) calculated as just the sum of the response headers and response body

MDN corroborates this as well <https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/transferSize>

The size includes the response header fields plus the response payload body (as defined by RFC7230).

@yury-sYury Semikhatsky (yury-s)Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand this correctly, har file contains _transferSize only for responses, so anything request related is irrelevant, right? I first thought that we'd have _transferSize for request as well, but I don't see such field in har.ts

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's my understanding as well

i think the closest equivalent for the request would be something like entry.request.headersSize + entry.request.bodySize

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically _transferSize is a non-standard extension

in WebKit, it's equivalent to entry.response.headersSize + entry.response.bodySize so unless Chromium is using it to expose some special info then it might just be a "shortcut" to avoid having to do that math 😅

Comment threadtests/library/har-websocket.spec.ts Outdated
{ type: 'receive', opcode: 2, data: incoming },
]);
for (const m of messages) {
expect(m.time).toBeGreaterThanOrEqual(beforeMs / 1000 - 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.time comes straight from CDP Network.webSocketFrameSent.timestamp in Chromium, which is a MonotonicTime (seconds since an arbitrary process origin), I don't think we can safely compare it with the wall time here. Do we use walltime in other places in har? If so, websockets should be aligned to, otherwise compare time delta here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

Network.webSocketWillSendHandshakeRequest.wallTime + (Network.webSocketFrame{Sent,Received}.timestamp - Network.webSocketWillSendHandshakeRequest.timestamp)

in Playwright we do this via WebSocket.prototype._toWallTime

Do we use walltime in other places in har?

there are a bunch of different time values within HAR

  • startDateTime: when the request first began (which is being changed to use a wallTime in feat(har): use engine timestamps instead of server Date #41082)
  • time: the total time taken from intial request to final response
  • timings: an object showing how much time was taken for each portion of the request
  • _monotonicTime: how long since the playwright process was started

i can further dig into whether the time for each WebSocket frame should be a WallTime or timestamp/MonotonicTime

at the very least, Firefox seems to send it as a WallTime (see the PR_Now() call inside WebSocketFrame)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

This may well be this case, but har file doesn't have to inherit that behavior. As a user of har I'd expect all wall time timestamps to be on the same timeline, so that I could compare them. If we need an extra step in Chromium to convert from time since webSocketWillSendHandshakeRequest to wall time, let's do it in the browser-specific code?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have this conversion code in WebSocket.prototype._toWallTime but yeah i can move it to Chromium and WebKit code instead

if (this._options.omitSizes)
return;

harEntry.response._transferSize = Math.max(0, harEntry.response._transferSize!) + ((opcode === 1) ? Buffer.byteLength(data, 'utf8') : base64ByteLength(data));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_transferSize counts payload bytes only, not frame overhead.
WebSocket framing adds a 2–14 byte header per frame (plus a 4-byte mask for client -> server frames). The computed value is payload-only, so it'll under-report vs. an actual on-wire transfer size. If we we can't compute it more accurately, it may be acceptable as an approximation, but worth a comment.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great point! ill look into this a bit more and see if i can get it more accurate (or as you suggest add a comment if i cant)

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.


// If the payload is more than 125 bytes then add either 2 or 8 more bytes depending on how long the payload is.
if (length > 2 ** 16)
headerSize += 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comment on where this number comes from?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is from the same URL as above but yeah i can repeat it here too

else if (length > 125)
headerSize += 2;

// The number of bytes needed for the mask are not included as that information is not exposed via any protocol.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chromium exposes mask: boolean field. Can mask size vary or it's always 4-bytes if present? Perhaps we can easily add similar information to wk and ff? I mean if we are anyway try to provide more accurate size estimate, we should probably support that too.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh neat! i didnt notice that. great catch! ill plumb that through and see what i can do to get any similar info from WebKit and Firefox

yes it's either 0 or 4 bytes depending on whether mask is false or true

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think we can always assume this as it seems to be required for the client to send a masked frame to the server <https://www.rfc-editor.org/info/rfc6455/#section-5.1>:

a client MUST mask all frames that it sends to the server

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7230 passed, 1103 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 flaky⚠️ [firefox-library] › library/har-websocket.spec.ts:313 › should record websocket connection failure `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:224 › cli codegen › should generate frame locators (4) `@firefox-ubuntu-22.04-node20`

39545 passed, 775 skipped


Merge workflow run.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@dcrousso@yury-s
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket - #41091

Merged
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes
Jun 3, 2026
Merged

feat(har): calculate request.headerSize, response.headerSize, and response._transferSize for WebSocket#41091
Devin Rousso (dcrousso) merged 1 commit into
microsoft:mainfrom
dcrousso:feat-har-WebSocket-sizes

Conversation

@dcrousso

Copy link
Copy Markdown
Contributor

No description provided.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso
Devin Rousso (dcrousso)force-pushed the feat-har-WebSocket-sizes branch 2 times, most recently from 2f828b3 to f0088f6CompareJune 2, 2026 15:58
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Comment threadtests/library/har-websocket.spec.ts Outdated
Comment threadtests/library/har-websocket.spec.ts Outdated
data,
});
updateTime(timestamp);
updateTransferSize(opcode, data);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting added to the response transfer size whereas it should go to the request.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh lol good point

eventsHelper.addEventListener(webSocket, network.WebSocket.Events.Request, ({ headers }: { headers: HeadersArray }) => {
this._recordRequestHeadersAndCookies(harEntry, headers);
if (!this._options.omitSizes)
harEntry.request.headersSize = network.requestHeadersSize(headers, webSocket.url(), method);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't request handshake headersSize be added to the request transfer size?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i originally thought that, but when looking at Request.prototype.async _sizes i saw that it's (manually) calculated as just the sum of the response headers and response body

MDN corroborates this as well <https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/transferSize>

The size includes the response header fields plus the response payload body (as defined by RFC7230).

@yury-sYury Semikhatsky (yury-s)Jun 2, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand this correctly, har file contains _transferSize only for responses, so anything request related is irrelevant, right? I first thought that we'd have _transferSize for request as well, but I don't see such field in har.ts

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that's my understanding as well

i think the closest equivalent for the request would be something like entry.request.headersSize + entry.request.bodySize

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically _transferSize is a non-standard extension

in WebKit, it's equivalent to entry.response.headersSize + entry.response.bodySize so unless Chromium is using it to expose some special info then it might just be a "shortcut" to avoid having to do that math 😅

Comment threadtests/library/har-websocket.spec.ts Outdated
{ type: 'receive', opcode: 2, data: incoming },
]);
for (const m of messages) {
expect(m.time).toBeGreaterThanOrEqual(beforeMs / 1000 - 1);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m.time comes straight from CDP Network.webSocketFrameSent.timestamp in Chromium, which is a MonotonicTime (seconds since an arbitrary process origin), I don't think we can safely compare it with the wall time here. Do we use walltime in other places in har? If so, websockets should be aligned to, otherwise compare time delta here.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

Network.webSocketWillSendHandshakeRequest.wallTime + (Network.webSocketFrame{Sent,Received}.timestamp - Network.webSocketWillSendHandshakeRequest.timestamp)

in Playwright we do this via WebSocket.prototype._toWallTime

Do we use walltime in other places in har?

there are a bunch of different time values within HAR

  • startDateTime: when the request first began (which is being changed to use a wallTime in feat(har): use engine timestamps instead of server Date #41082)
  • time: the total time taken from intial request to final response
  • timings: an object showing how much time was taken for each portion of the request
  • _monotonicTime: how long since the playwright process was started

i can further dig into whether the time for each WebSocket frame should be a WallTime or timestamp/MonotonicTime

at the very least, Firefox seems to send it as a WallTime (see the PR_Now() call inside WebSocketFrame)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of the Chromium (and WebKit) protocols is that it's actually meant to be used as a delta from the timestamp (and wallTime) of Network.webSocketWillSendHandshakeRequest

This may well be this case, but har file doesn't have to inherit that behavior. As a user of har I'd expect all wall time timestamps to be on the same timeline, so that I could compare them. If we need an extra step in Chromium to convert from time since webSocketWillSendHandshakeRequest to wall time, let's do it in the browser-specific code?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have this conversion code in WebSocket.prototype._toWallTime but yeah i can move it to Chromium and WebKit code instead

if (this._options.omitSizes)
return;

harEntry.response._transferSize = Math.max(0, harEntry.response._transferSize!) + ((opcode === 1) ? Buffer.byteLength(data, 'utf8') : base64ByteLength(data));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_transferSize counts payload bytes only, not frame overhead.
WebSocket framing adds a 2–14 byte header per frame (plus a 4-byte mask for client -> server frames). The computed value is payload-only, so it'll under-report vs. an actual on-wire transfer size. If we we can't compute it more accurately, it may be acceptable as an approximation, but worth a comment.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great point! ill look into this a bit more and see if i can get it more accurate (or as you suggest add a comment if i cant)

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.


// If the payload is more than 125 bytes then add either 2 or 8 more bytes depending on how long the payload is.
if (length > 2 ** 16)
headerSize += 8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comment on where this number comes from?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is from the same URL as above but yeah i can repeat it here too

else if (length > 125)
headerSize += 2;

// The number of bytes needed for the mask are not included as that information is not exposed via any protocol.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chromium exposes mask: boolean field. Can mask size vary or it's always 4-bytes if present? Perhaps we can easily add similar information to wk and ff? I mean if we are anyway try to provide more accurate size estimate, we should probably support that too.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh neat! i didnt notice that. great catch! ill plumb that through and see what i can do to get any similar info from WebKit and Firefox

yes it's either 0 or 4 bytes depending on whether mask is false or true

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think we can always assume this as it seems to be required for the client to send a masked frame to the server <https://www.rfc-editor.org/info/rfc6455/#section-5.1>:

a client MUST mask all frames that it sends to the server

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7230 passed, 1103 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 flaky⚠️ [firefox-library] › library/har-websocket.spec.ts:313 › should record websocket connection failure `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:224 › cli codegen › should generate frame locators (4) `@firefox-ubuntu-22.04-node20`

39545 passed, 775 skipped


Merge workflow run.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@dcrousso@yury-s