Skip to content

c: use proper type coercion in consume - #44

Closed
indutny wants to merge 1 commit into
masterfrom
fix/type-coercion-in-consume
Closed

c: use proper type coercion in consume#44
indutny wants to merge 1 commit into
masterfrom
fix/type-coercion-in-consume

Conversation

@indutny

@indutnyindutny commented May 12, 2021

Copy link
Copy Markdown
Member

On 32bit platforms size_t is essentially uint32_t (or at times
even meager uint16_t). Loading uint64_t field value into size_t on
these platforms would truncate the high bits and leave only the low 32
(16) bits in place. This leads to various interesting errors in
downstream modules. See:

This patch makes all field loads go into their respective types.
Truncation doesn't happen in this case because C coercion rules will
cast both types to the largest necessary datatype to hold either of
them.

cc @Trott@mcollina @nodejs/http

I'd appreciate if we could fast-track this. We've been avoiding issues with this for awhile, but both wasm build in unidici and some arm users from the llhttp#110 hit this consistently on large uploads/downloads.

@indutny
indutny requested review from Trott and mcollinaMay 12, 2021 06:29
@indutny

Copy link
Copy Markdown
MemberAuthor

cc @addaleax as well since you are so good at C/C++!

On 32bit platforms `size_t` is essentially `uint32_t` (or at times
even meager `uint16_t`). Loading `uint64_t` field value into `size_t` on
these platforms would truncate the high bits and leave only the low 32
(16) bits in place. This leads to various interesting errors in
downstream modules. See:
- nodejs/llhttp#110
- nodejs/undici#803
This patch makes all field loads go into their respective types.
Truncation doesn't happen in this case because C coercion rules will
cast both types to the largest necessary datatype to hold either of
them.
@indutny
indutnyforce-pushed the fix/type-coercion-in-consume branch from d09bc9e to 0ea76edCompareMay 12, 2021 06:35
@targos

Copy link
Copy Markdown
Member

Does it affect 32-bit builds of Node.js too?

@indutny

Copy link
Copy Markdown
MemberAuthor

Does it affect 32-bit builds of Node.js too?

I believe it does.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina

Copy link
Copy Markdown
Member

+1 for fast-tracking this.

@indutny

Copy link
Copy Markdown
MemberAuthor

FWIW, here's how the generated code looks with this patch:

size_tavail;
uint64_tneed;
avail=endp-p;
need=state->content_length;
if (avail >= need) {
p+=need;
state->content_length=0;
goto s_n_llhttp__internal__n_span_end_llhttp__on_body;
}
state->content_length-=avail;
returns_n_llhttp__internal__n_consume_content_length;
/* UNREACHABLE */;
abort();

Note the uint64_t above. It was size_t for both variables before the patch.

indutny added a commit that referenced this pull request May 12, 2021
On 32bit platforms `size_t` is essentially `uint32_t` (or at times
even meager `uint16_t`). Loading `uint64_t` field value into `size_t` on
these platforms would truncate the high bits and leave only the low 32
(16) bits in place. This leads to various interesting errors in
downstream modules. See:
- nodejs/llhttp#110
- nodejs/undici#803
This patch makes all field loads go into their respective types.
Truncation doesn't happen in this case because C coercion rules will
cast both types to the largest necessary datatype to hold either of
them.
PR-URL: #44
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Daniele Belardi <dwon.dnl@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@indutny

Copy link
Copy Markdown
MemberAuthor

Landed in 037dd2f. Thanks y'all!

@indutnyindutny closed this May 12, 2021
@indutny
indutny deleted the fix/type-coercion-in-consume branch May 12, 2021 18:32
@Chaphasilor

Copy link
Copy Markdown

Landed in 037dd2f. Thanks y'all!

Hey @indutny thanks for this fix! Could you share any info on how soon this will trickle down into actual Node, and on which versions it will work?
Or is it just a matter of rebuilding Node and it will use the latest version of llparse? :)

@indutny

Copy link
Copy Markdown
MemberAuthor

I'm still waiting for llhttp's CI run to complete, and will publish patch version releases after that. Will submit PRs to Node.js right after that. We should be able to have a discussion on the release timing there!

indutny added a commit to nodejs/node that referenced this pull request May 13, 2021
indutny added a commit to nodejs/node that referenced this pull request May 13, 2021
indutny added a commit to nodejs/node that referenced this pull request May 20, 2021
Fix: #37053
See: nodejs/llparse#44
PR-URL: #38665
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Daniele Belardi <dwon.dnl@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
danielleadams pushed a commit to nodejs/node that referenced this pull request May 31, 2021
Fix: #37053
See: nodejs/llparse#44
PR-URL: #38665
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Daniele Belardi <dwon.dnl@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@indutny@targos@mcollina@Chaphasilor@Trott@addaleax@ronag@dnlup