Skip to content

Enforce a size limit in getSpreadType - #40755

Merged
Andrew Casey (amcasey) merged 2 commits into
microsoft:masterfrom
amcasey:SpreadLimit
Sep 30, 2020
Merged

Enforce a size limit in getSpreadType#40755
Andrew Casey (amcasey) merged 2 commits into
microsoft:masterfrom
amcasey:SpreadLimit

Conversation

@amcasey

Copy link
Copy Markdown
Member

When a union is spread into a union, the sizes are multiplied,
potentially resulting in an enormous union (especially if there are
repeated spreads). This check detects cases that used to run out of
memory.

Fixes#40754

When a union is spread into a union, the sizes are multiplied,
potentially resulting in an enormous union (especially if there are
repeated spreads). This check detects cases that used to run out of
memory.
Fixesmicrosoft#40754
@amcasey

Copy link
Copy Markdown
MemberAuthor

TypeScript Bot (@typescript-bot) user test this

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 24, 2020

Copy link
Copy Markdown
Contributor

Heya Andrew Casey (@amcasey), I've started to run the parallelized community code test suite on this PR at 6650496. You can monitor the build here.

Comment threadsrc/compiler/checker.ts Outdated
const resultSize = (left as UnionType).types.length * (right as UnionType).types.length;
if (resultSize > 100000) {
tracing.instant(tracing.Phase.Check, "getSpreadType_DepthLimit", { leftId: left.id, rightId: right.id });
error(currentNode, Diagnostics.Spread_expression_produces_a_union_type_that_is_too_complex_to_represent);

@weswighamWesley Wigham (weswigham)Sep 24, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this error is going to be frustrating to encounter with it also providing a suggestion for a workaround. What is the general-form workaround? Casting something somewhere to any? Should we add a related span/quickfix for that?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's a fix I'm sufficiently confident in to make it a quick fix. I gave this a new error number so that it would be easier to document/google.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did wonder about looking inside the union to figure out whether there's a smaller span we could squiggle.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the real code where this arose, the mitigation I suggested was adding a cast to each spread with a pre-existing type that listed all possible properties (optionally). I imagine any would also work, possible at the expense of completions and such.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is definitely fine, and, even if we broaden #34853 to fix the reported issue without an error (which I think we should consider), we'll still fundamentally need this as a fallback.

@amcasey

Copy link
Copy Markdown
MemberAuthor

If we broaden #34853, I'd probably drop the limit substantially. I based it on what would still compile quickly in my toy example.

@amcasey

Copy link
Copy Markdown
MemberAuthor

TypeScript Bot (@typescript-bot) test this

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 24, 2020

Copy link
Copy Markdown
Contributor

Heya Andrew Casey (@amcasey), I've started to run the extended test suite on this PR at 6650496. You can monitor the build here.

@typescript-bot

Copy link
Copy Markdown
Contributor

The user suite test run you requested has finished and failed. I've opened a PR with the baseline diff from master.

@amcasey

Copy link
Copy Markdown
MemberAuthor

Since there are no occurrences of my new error code in the user test baseline diff, I'm assuming I can ignore the failure.

@DanielRosenwasser

Copy link
Copy Markdown
Member

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 25, 2020

Copy link
Copy Markdown
Contributor

Heya Daniel Rosenwasser (@DanielRosenwasser), I've started to run the tarball bundle task on this PR at 6650496. You can monitor the build here.

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 25, 2020

Copy link
Copy Markdown
Contributor

Heya Daniel Rosenwasser (@DanielRosenwasser), I've started to run the perf test suite on this PR at 6650496. You can monitor the build here.

Update: The results are in!

@DanielRosenwasser

Copy link
Copy Markdown
Member

TypeScript Bot (@typescript-bot) test this

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 25, 2020

Copy link
Copy Markdown
Contributor

Heya Daniel Rosenwasser (@DanielRosenwasser), I've started to run the extended test suite on this PR at 6650496. You can monitor the build here.

Comment threadsrc/compiler/checker.ts Outdated
if (resultSize > 100000) {
tracing.instant(tracing.Phase.Check, "getSpreadType_DepthLimit", { leftId: left.id, rightId: right.id });
error(currentNode, Diagnostics.Spread_expression_produces_a_union_type_that_is_too_complex_to_represent);
return errorType;

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mine Starks (@minestarks) raised an interesting point: should this be anyType so that users have the option of ts-ignoring this error?

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 25, 2020

Copy link
Copy Markdown
Contributor

Hey Daniel Rosenwasser (@DanielRosenwasser), I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
"devDependencies": {
"typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/86514/artifacts?artifactName=tgz&fileId=D9585AAA2ECA42B07D99CAE4C35EB7EBDFBD0966E447E25C919235B4C846210402&fileName=/typescript-4.1.0-insiders.20200925.tgz"
}
}

and then running npm install.


There is also a playground for this build.

@typescript-bot

Copy link
Copy Markdown
Contributor

Daniel Rosenwasser (@DanielRosenwasser)
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..40755

Metricmaster40755DeltaBestWorst
Angular - node (v10.16.3, x64)
Memory used349,631k (± 0.02%)349,621k (± 0.02%)-9k (- 0.00%)349,480k349,713k
Parse Time2.00s (± 0.50%)2.01s (± 0.53%)+0.01s (+ 0.45%)1.99s2.03s
Bind Time0.83s (± 0.80%)0.83s (± 0.63%)+0.00s (+ 0.12%)0.82s0.84s
Check Time4.91s (± 0.54%)4.92s (± 0.60%)+0.01s (+ 0.18%)4.86s4.97s
Emit Time5.23s (± 0.90%)5.21s (± 0.51%)-0.02s (- 0.36%)5.16s5.27s
Total Time12.97s (± 0.40%)12.96s (± 0.31%)-0.00s (- 0.02%)12.88s13.06s
Monaco - node (v10.16.3, x64)
Memory used354,336k (± 0.02%)354,356k (± 0.02%)+20k (+ 0.01%)354,171k354,534k
Parse Time1.56s (± 0.31%)1.56s (± 0.38%)+0.01s (+ 0.45%)1.55s1.58s
Bind Time0.71s (± 0.69%)0.72s (± 1.06%)+0.00s (+ 0.28%)0.70s0.73s
Check Time5.05s (± 0.47%)5.07s (± 0.73%)+0.02s (+ 0.38%)4.99s5.16s
Emit Time2.75s (± 0.55%)2.79s (± 0.97%)+0.05s (+ 1.64%)2.74s2.85s
Total Time10.07s (± 0.30%)10.14s (± 0.61%)+0.07s (+ 0.67%)9.99s10.28s
TFS - node (v10.16.3, x64)
Memory used307,564k (± 0.02%)307,574k (± 0.02%)+10k (+ 0.00%)307,433k307,704k
Parse Time1.22s (± 0.62%)1.22s (± 0.39%)+0.00s (+ 0.33%)1.21s1.23s
Bind Time0.67s (± 0.74%)0.67s (± 1.08%)-0.00s (- 0.44%)0.65s0.68s
Check Time4.53s (± 0.48%)4.54s (± 0.51%)+0.00s (+ 0.11%)4.48s4.59s
Emit Time2.89s (± 0.81%)2.91s (± 1.17%)+0.02s (+ 0.62%)2.84s2.99s
Total Time9.31s (± 0.45%)9.33s (± 0.52%)+0.03s (+ 0.29%)9.24s9.43s
material-ui - node (v10.16.3, x64)
Memory used488,962k (± 0.01%)488,937k (± 0.02%)-25k (- 0.01%)488,709k489,106k
Parse Time1.99s (± 0.53%)2.00s (± 0.68%)+0.00s (+ 0.20%)1.98s2.04s
Bind Time0.65s (± 0.91%)0.65s (± 0.61%)-0.00s (- 0.46%)0.64s0.66s
Check Time13.61s (± 0.91%)13.53s (± 1.06%)-0.08s (- 0.57%)13.29s13.88s
Emit Time0.00s (± 0.00%)0.00s (± 0.00%)0.00s ( NaN%)0.00s0.00s
Total Time16.26s (± 0.81%)16.18s (± 0.92%)-0.08s (- 0.47%)15.92s16.55s
Angular - node (v12.1.0, x64)
Memory used326,702k (± 0.09%)326,810k (± 0.02%)+107k (+ 0.03%)326,682k326,975k
Parse Time1.99s (± 0.40%)2.00s (± 0.62%)+0.01s (+ 0.40%)1.97s2.03s
Bind Time0.81s (± 0.58%)0.81s (± 0.84%)-0.00s (- 0.49%)0.79s0.82s
Check Time4.82s (± 0.99%)4.84s (± 0.96%)+0.02s (+ 0.44%)4.76s4.97s
Emit Time5.41s (± 1.06%)5.37s (± 0.60%)-0.04s (- 0.78%)5.30s5.43s
Total Time13.03s (± 0.59%)13.01s (± 0.57%)-0.02s (- 0.15%)12.87s13.15s
Monaco - node (v12.1.0, x64)
Memory used336,564k (± 0.02%)336,492k (± 0.02%)-72k (- 0.02%)336,346k336,718k
Parse Time1.55s (± 0.81%)1.54s (± 0.84%)-0.01s (- 0.45%)1.51s1.57s
Bind Time0.69s (± 0.64%)0.69s (± 0.72%)+0.00s (+ 0.43%)0.69s0.71s
Check Time4.86s (± 0.53%)4.87s (± 0.43%)+0.01s (+ 0.12%)4.81s4.90s
Emit Time2.82s (± 0.70%)2.82s (± 0.66%)-0.00s (- 0.04%)2.78s2.87s
Total Time9.92s (± 0.46%)9.92s (± 0.34%)+0.00s (+ 0.01%)9.84s10.00s
TFS - node (v12.1.0, x64)
Memory used291,855k (± 0.03%)291,835k (± 0.01%)-19k (- 0.01%)291,754k291,951k
Parse Time1.23s (± 0.54%)1.23s (± 0.72%)-0.00s (- 0.24%)1.21s1.24s
Bind Time0.64s (± 0.92%)0.64s (± 0.62%)-0.00s (- 0.47%)0.63s0.65s
Check Time4.43s (± 0.35%)4.44s (± 0.49%)+0.01s (+ 0.16%)4.39s4.50s
Emit Time2.94s (± 0.92%)2.94s (± 0.77%)-0.00s (- 0.03%)2.90s3.00s
Total Time9.24s (± 0.31%)9.24s (± 0.38%)-0.00s (- 0.02%)9.20s9.33s
material-ui - node (v12.1.0, x64)
Memory used466,943k (± 0.06%)467,006k (± 0.01%)+63k (+ 0.01%)466,873k467,147k
Parse Time2.00s (± 0.41%)2.01s (± 0.63%)+0.01s (+ 0.55%)1.98s2.04s
Bind Time0.64s (± 0.63%)0.64s (± 0.57%)-0.00s (- 0.47%)0.63s0.64s
Check Time12.09s (± 0.87%)12.07s (± 0.75%)-0.02s (- 0.17%)11.90s12.25s
Emit Time0.00s (± 0.00%)0.00s (± 0.00%)0.00s ( NaN%)0.00s0.00s
Total Time14.73s (± 0.76%)14.72s (± 0.67%)-0.01s (- 0.10%)14.53s14.91s
Angular - node (v8.9.0, x64)
Memory used346,320k (± 0.02%)346,249k (± 0.02%)-70k (- 0.02%)346,040k346,350k
Parse Time2.55s (± 0.26%)2.55s (± 0.71%)-0.00s (- 0.04%)2.53s2.62s
Bind Time0.87s (± 0.75%)0.86s (± 0.60%)-0.01s (- 0.81%)0.85s0.88s
Check Time5.55s (± 0.41%)5.55s (± 0.80%)-0.00s (- 0.02%)5.45s5.64s
Emit Time6.08s (± 2.63%)6.19s (± 1.34%)+0.11s (+ 1.88%)6.06s6.39s
Total Time15.05s (± 1.05%)15.16s (± 0.79%)+0.11s (+ 0.71%)14.99s15.52s
Monaco - node (v8.9.0, x64)
Memory used355,609k (± 0.02%)355,605k (± 0.02%)-4k (- 0.00%)355,429k355,708k
Parse Time1.88s (± 0.36%)1.89s (± 0.51%)+0.00s (+ 0.05%)1.87s1.91s
Bind Time0.89s (± 0.76%)0.89s (± 0.53%)-0.00s (- 0.11%)0.88s0.90s
Check Time5.62s (± 0.45%)5.62s (± 0.40%)+0.00s (+ 0.09%)5.57s5.67s
Emit Time3.29s (± 1.35%)3.29s (± 1.27%)-0.01s (- 0.18%)3.19s3.35s
Total Time11.68s (± 0.46%)11.68s (± 0.42%)-0.00s (- 0.02%)11.59s11.81s
TFS - node (v8.9.0, x64)
Memory used309,284k (± 0.02%)309,317k (± 0.02%)+33k (+ 0.01%)309,202k309,485k
Parse Time1.55s (± 0.37%)1.55s (± 0.34%)+0.00s (+ 0.26%)1.54s1.56s
Bind Time0.67s (± 0.66%)0.68s (± 1.18%)+0.01s (+ 0.89%)0.66s0.69s
Check Time5.31s (± 0.40%)5.33s (± 0.54%)+0.02s (+ 0.38%)5.29s5.41s
Emit Time2.93s (± 0.97%)2.95s (± 0.59%)+0.02s (+ 0.51%)2.92s3.00s
Total Time10.46s (± 0.37%)10.51s (± 0.40%)+0.05s (+ 0.46%)10.44s10.63s
material-ui - node (v8.9.0, x64)
Memory used493,374k (± 0.01%)493,373k (± 0.01%)-1k (- 0.00%)493,226k493,631k
Parse Time2.41s (± 0.25%)2.41s (± 0.51%)+0.00s (+ 0.08%)2.38s2.44s
Bind Time0.81s (± 1.00%)0.81s (± 1.54%)+0.00s (+ 0.12%)0.79s0.84s
Check Time18.03s (± 0.77%)18.03s (± 0.97%)-0.00s (- 0.03%)17.78s18.56s
Emit Time0.00s (± 0.00%)0.00s (± 0.00%)0.00s ( NaN%)0.00s0.00s
Total Time21.26s (± 0.68%)21.25s (± 0.88%)-0.01s (- 0.03%)20.99s21.80s
Angular - node (v8.9.0, x86)
Memory used198,619k (± 0.03%)198,563k (± 0.03%)-56k (- 0.03%)198,411k198,676k
Parse Time2.47s (± 0.83%)2.47s (± 0.67%)+0.00s (+ 0.12%)2.45s2.53s
Bind Time1.01s (± 0.95%)1.01s (± 1.20%)-0.00s (- 0.49%)0.98s1.03s
Check Time5.00s (± 0.45%)5.00s (± 0.58%)+0.00s (+ 0.06%)4.94s5.07s
Emit Time5.92s (± 0.87%)5.95s (± 0.76%)+0.03s (+ 0.49%)5.84s6.05s
Total Time14.40s (± 0.52%)14.43s (± 0.54%)+0.03s (+ 0.20%)14.25s14.59s
Monaco - node (v8.9.0, x86)
Memory used201,423k (± 0.01%)201,444k (± 0.02%)+21k (+ 0.01%)201,364k201,525k
Parse Time1.93s (± 0.55%)1.93s (± 1.19%)0.00s ( 0.00%)1.90s2.01s
Bind Time0.70s (± 0.53%)0.71s (± 1.02%)+0.01s (+ 0.99%)0.70s0.73s
Check Time5.49s (± 1.48%)5.45s (± 0.60%)-0.04s (- 0.67%)5.39s5.52s
Emit Time3.04s (± 2.74%)3.06s (± 0.54%)+0.02s (+ 0.76%)3.03s3.10s
Total Time11.15s (± 0.36%)11.15s (± 0.45%)-0.00s (- 0.03%)11.09s11.29s
TFS - node (v8.9.0, x86)
Memory used176,823k (± 0.03%)176,823k (± 0.01%)-0k (- 0.00%)176,773k176,890k
Parse Time1.60s (± 0.99%)1.59s (± 0.86%)-0.00s (- 0.25%)1.57s1.62s
Bind Time0.65s (± 1.23%)0.64s (± 0.62%)-0.01s (- 1.08%)0.63s0.65s
Check Time4.81s (± 0.49%)4.80s (± 0.41%)-0.01s (- 0.31%)4.75s4.84s
Emit Time2.79s (± 1.13%)2.83s (± 1.30%)+0.04s (+ 1.47%)2.75s2.91s
Total Time9.84s (± 0.49%)9.86s (± 0.46%)+0.02s (+ 0.17%)9.77s9.96s
material-ui - node (v8.9.0, x86)
Memory used277,785k (± 0.01%)277,807k (± 0.02%)+22k (+ 0.01%)277,669k277,962k
Parse Time2.47s (± 0.82%)2.49s (± 0.96%)+0.02s (+ 0.61%)2.46s2.57s
Bind Time0.72s (± 4.58%)0.73s (± 5.84%)+0.01s (+ 1.67%)0.68s0.83s
Check Time16.49s (± 1.36%)16.46s (± 0.60%)-0.03s (- 0.16%)16.28s16.68s
Emit Time0.00s (± 0.00%)0.00s (± 0.00%)0.00s ( NaN%)0.00s0.00s
Total Time19.68s (± 1.24%)19.69s (± 0.69%)+0.01s (+ 0.04%)19.45s20.09s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-166-generic
Architecturex64
Available Memory16 GB
Available Memory1 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
BenchmarkNameIterations
Current4075510
Baselinemaster10

@amcasey

Copy link
Copy Markdown
MemberAuthor

This should probably use checkCrossProductUnion.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

For Uncommitted BugPR for untriaged, rejected, closed or missing bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Repeated conditional spreading of object literals results in OOM

4 participants

@amcasey@typescript-bot@DanielRosenwasser@weswigham