Don't skip elaborations when reporting errors for cached failed relations by Andarist · Pull Request #55234 · microsoft/TypeScript

Mateusz Burzyński (Andarist) · 2023-08-01T22:51:00Z

fixes#3276
to learn more about what's happening you can read my comment here

I'm just opening this as a draft, even without updating baselines. It might be interesting to run the perf suite on this one.

…ions

Ryan Cavanaugh (RyanCavanaugh) · 2023-08-02T18:20:29Z

TypeScript Bot (@typescript-bot) perf test this faster

TypeScript Bot (typescript-bot) · 2023-08-02T18:20:33Z

Heya Ryan Cavanaugh (@RyanCavanaugh), I've started to run the abridged perf test suite on this PR at eaba5a1. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-08-02T18:36:18Z

Ryan Cavanaugh (@RyanCavanaugh)
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..55234

_Metric	_main	₅₅₂₃₄	_Delta	_Best	_Worst	_p-value
_{Angular - node (v16.17.1, x64)}
_{Memory used}	_{368,212k (± 0.00%)}	_{368,215k (± 0.00%)}	_~	_368,195k	_368,243k	_{p=0.936 n=6}
_{Parse Time}	_{3.56s (± 0.63%)}	_{3.56s (± 0.63%)}	_~	_3.55s	_3.61s	_{p=1.000 n=6}
_{Bind Time}	_{1.19s (± 0.34%)}	_{1.19s (± 0.43%)}	_~	_1.18s	_1.19s	_{p=0.114 n=6}
_{Check Time}	_{9.73s (± 0.54%)}	_{9.71s (± 0.18%)}	_~	_9.69s	_9.73s	_{p=0.935 n=6}
_{Emit Time}	_{8.00s (± 0.86%)}	_{7.95s (± 0.49%)}	_~	_7.88s	_7.99s	_{p=0.377 n=6}
_{Total Time}	_{22.48s (± 0.53%)}	_{22.42s (± 0.25%)}	_~	_22.31s	_22.47s	_{p=0.689 n=6}
_{Compiler-Unions - node (v16.17.1, x64)}
_{Memory used}	_{193,820k (± 0.01%)}	_{193,832k (± 0.02%)}	_~	_193,780k	_193,880k	_{p=0.689 n=6}
_{Parse Time}	_{1.59s (± 0.56%)}	_{1.58s (± 1.31%)}	_~	_1.55s	_1.61s	_{p=0.804 n=6}
_{Bind Time}	_{0.83s (± 1.25%)}	_{0.82s (± 0.77%)}	_~	_0.81s	_0.83s	_{p=0.203 n=6}
_{Check Time}	_{10.13s (± 0.39%)}	_{10.13s (± 0.54%)}	_~	_10.06s	_10.22s	_{p=0.872 n=6}
_{Emit Time}	_{3.01s (± 1.38%)}	_{2.98s (± 0.78%)}	_~	_2.94s	_3.01s	_{p=0.258 n=6}
_{Total Time}	_{15.56s (± 0.41%)}	_{15.51s (± 0.51%)}	_~	_15.39s	_15.62s	_{p=0.423 n=6}
_{Monaco - node (v16.17.1, x64)}
_{Memory used}	_{347,071k (± 0.01%)}	_{347,072k (± 0.00%)}	_~	_347,051k	_347,085k	_{p=1.000 n=6}
_{Parse Time}	_{2.78s (± 0.63%)}	_{2.76s (± 0.48%)}	_~	_2.74s	_2.78s	_{p=0.143 n=6}
_{Bind Time}	_{1.08s (± 0.59%)}	_{1.07s (± 0.38%)}	_{-0.01s (- 0.77%)}	_1.07s	_1.08s	_{p=0.033 n=6}
_{Check Time}	_{8.03s (± 0.38%)}	_{8.01s (± 0.46%)}	_~	_7.94s	_8.04s	_{p=0.373 n=6}
_{Emit Time}	_{4.47s (± 0.45%)}	_{4.46s (± 0.87%)}	_~	_4.41s	_4.51s	_{p=0.744 n=6}
_{Total Time}	_{16.36s (± 0.24%)}	_{16.30s (± 0.33%)}	_~	_16.24s	_16.37s	_{p=0.107 n=6}
_{TFS - node (v16.17.1, x64)}
_{Memory used}	_{301,124k (± 0.01%)}	_{301,114k (± 0.01%)}	_~	_301,081k	_301,142k	_{p=0.630 n=6}
_{Parse Time}	_{2.20s (± 0.60%)}	_{2.19s (± 0.63%)}	_~	_2.18s	_2.21s	_{p=0.805 n=6}
_{Bind Time}	_{1.22s (± 0.96%)}	_{1.20s (± 0.43%)}	_{-0.02s (- 1.78%)}	_1.19s	_1.20s	_{p=0.004 n=6}
_{Check Time}	_{7.32s (± 0.26%)}	_{7.31s (± 0.24%)}	_~	_7.28s	_7.33s	_{p=0.415 n=6}
_{Emit Time}	_{4.31s (± 0.91%)}	_{4.32s (± 0.56%)}	_~	_4.28s	_4.35s	_{p=0.686 n=6}
_{Total Time}	_{15.04s (± 0.37%)}	_{15.01s (± 0.32%)}	_~	_14.95s	_15.09s	_{p=0.296 n=6}
_{material-ui - node (v16.17.1, x64)}
_{Memory used}	_{481,871k (± 0.01%)}	_{481,880k (± 0.01%)}	_~	_481,821k	_481,921k	_{p=0.873 n=6}
_{Parse Time}	_{3.25s (± 0.36%)}	_{3.25s (± 0.49%)}	_~	_3.24s	_3.28s	_{p=0.863 n=6}
_{Bind Time}	_{0.96s (± 0.78%)}	_{0.95s (± 1.10%)}	_~	_0.94s	_0.97s	_{p=0.611 n=6}
_{Check Time}	_{18.30s (± 0.64%)}	_{18.32s (± 0.41%)}	_~	_18.24s	_18.43s	_{p=0.375 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{22.50s (± 0.48%)}	_{22.53s (± 0.36%)}	_~	_22.43s	_22.63s	_{p=0.423 n=6}
_{xstate - node (v16.17.1, x64)}
_{Memory used}	_{561,397k (± 0.01%)}	_{561,338k (± 0.02%)}	_~	_561,247k	_561,540k	_{p=0.298 n=6}
_{Parse Time}	_{4.04s (± 0.34%)}	_{4.01s (± 0.20%)}	_{-0.02s (- 0.54%)}	_4.00s	_4.02s	_{p=0.009 n=6}
_{Bind Time}	_{1.70s (± 6.37%)}	_{1.70s (± 4.91%)}	_~	_1.58s	_1.78s	_{p=0.747 n=6}
_{Check Time}	_{3.12s (± 2.88%)}	_{3.08s (± 2.87%)}	_~	_3.02s	_3.25s	_{p=0.147 n=6}
_{Emit Time}	_{0.09s (± 4.45%)}	_{0.09s (± 5.53%)}	_~	_0.09s	_0.10s	_{p=0.595 n=6}
_{Total Time}	_{8.94s (± 0.59%)}	_{8.89s (± 0.79%)}	_~	_8.77s	_8.98s	_{p=0.261 n=6}

System

_{Machine Name}	_ts-ci-ubuntu
_Platform	_{linux 5.4.0-148-generic}
_Architecture	_x64
_{Available Memory}	_{16 GB}
_{Available Memory}	_{15 GB}
_CPUs	_{4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz}

Hosts

node (v16.17.1, x64)

Scenarios

Angular - node (v16.17.1, x64)
Compiler-Unions - node (v16.17.1, x64)
Monaco - node (v16.17.1, x64)
TFS - node (v16.17.1, x64)
material-ui - node (v16.17.1, x64)
xstate - node (v16.17.1, x64)

_Benchmark	_Name	_Iterations
_Current	₅₅₂₃₄	₆
_Baseline	_main	₆

Developer Information:

Download Benchmark

…-on-cached-entries

Mateusz Burzyński (Andarist) · 2023-08-06T19:52:22Z

Ryan Cavanaugh (@RyanCavanaugh) since the perf run came up clean I updated the baselines and cleaned up this a little bit further. Looking at the changed baselines it seems like a neat improvement to me :)

Wesley Wigham (weswigham) · 2023-08-14T20:16:21Z

TypeScript Bot (@typescript-bot) test this
TypeScript Bot (@typescript-bot) run dt
TypeScript Bot (@typescript-bot) test top100
TypeScript Bot (@typescript-bot) new perf test this

TypeScript Bot (typescript-bot) · 2023-08-14T20:16:23Z

Heya Wesley Wigham (@weswigham), I've started to run the diff-based top-repos suite on this PR at 4a88fbd. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-08-14T20:16:23Z

Heya Wesley Wigham (@weswigham), I've started to run the regular perf test suite on this PR at 4a88fbd. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-08-14T20:16:23Z

Heya Wesley Wigham (@weswigham), I've started to run the parallelized Definitely Typed test suite on this PR at 4a88fbd. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-08-14T20:46:45Z

Wesley Wigham (@weswigham)
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Angular - node (v16.17.1, x64)}
_{Memory used}	_{300,343k (± 0.00%)}	_{300,351k (± 0.00%)}	_~	_300,343k	_300,360k	_{p=0.064 n=6}
_{Parse Time}	_{3.03s (± 0.13%)}	_{3.03s (± 0.17%)}	_~	_3.02s	_3.03s	_{p=0.595 n=6}
_{Bind Time}	_{0.93s (± 0.44%)}	_{0.93s (± 0.00%)}	_~	_0.93s	_0.93s	_{p=0.405 n=6}
_{Check Time}	_{9.45s (± 0.20%)}	_{9.45s (± 0.18%)}	_~	_9.43s	_9.47s	_{p=0.685 n=6}
_{Emit Time}	_{7.62s (± 0.35%)}	_{7.64s (± 0.18%)}	_~	_7.62s	_7.66s	_{p=0.512 n=6}
_{Total Time}	_{21.04s (± 0.16%)}	_{21.05s (± 0.06%)}	_~	_21.03s	_21.07s	_{p=0.410 n=6}
_{Compiler-Unions - node (v16.17.1, x64)}
_{Memory used}	_{193,891k (± 0.01%)}	_{193,879k (± 0.01%)}	_~	_193,865k	_193,899k	_{p=0.575 n=6}
_{Parse Time}	_{1.58s (± 0.26%)}	_{1.57s (± 0.26%)}	_{-0.01s (- 0.42%)}	_1.57s	_1.58s	_{p=0.034 n=6}
_{Bind Time}	_{0.79s (± 0.00%)}	_{0.79s (± 0.00%)}	_~	_0.79s	_0.79s	_{p=1.000 n=6}
_{Check Time}	_{9.93s (± 0.56%)}	_{9.91s (± 0.46%)}	_~	_9.84s	_9.97s	_{p=0.520 n=6}
_{Emit Time}	_{2.75s (± 0.27%)}	_{2.74s (± 0.30%)}	_~	_2.73s	_2.75s	_{p=0.383 n=6}
_{Total Time}	_{15.04s (± 0.37%)}	_{15.02s (± 0.26%)}	_~	_14.96s	_15.07s	_{p=0.422 n=6}
_{Monaco - node (v16.17.1, x64)}
_{Memory used}	_{347,031k (± 0.01%)}	_{347,036k (± 0.01%)}	_~	_347,022k	_347,068k	_{p=0.336 n=6}
_{Parse Time}	_{2.69s (± 0.19%)}	_{2.69s (± 0.38%)}	_~	_2.67s	_2.70s	_{p=0.928 n=6}
_{Bind Time}	_{0.99s (± 0.00%)}	_{0.99s (± 0.41%)}	_~	_0.98s	_0.99s	_{p=0.405 n=6}
_{Check Time}	_{7.90s (± 0.49%)}	_{7.89s (± 0.26%)}	_~	_7.85s	_7.91s	_{p=0.629 n=6}
_{Emit Time}	_{4.27s (± 0.76%)}	_{4.28s (± 0.32%)}	_~	_4.26s	_4.30s	_{p=0.331 n=6}
_{Total Time}	_{15.84s (± 0.43%)}	_{15.84s (± 0.18%)}	_~	_15.79s	_15.87s	_{p=0.748 n=6}
_{TFS - node (v16.17.1, x64)}
_{Memory used}	_{301,106k (± 0.00%)}	_{301,112k (± 0.00%)}	_~	_301,104k	_301,118k	_{p=0.469 n=6}
_{Parse Time}	_{2.16s (± 0.45%)}	_{2.16s (± 0.54%)}	_~	_2.15s	_2.18s	_{p=1.000 n=6}
_{Bind Time}	_{1.11s (± 0.00%)}	_{1.11s (± 0.00%)}	_~	_1.11s	_1.11s	_{p=1.000 n=6}
_{Check Time}	_{7.23s (± 0.36%)}	_{7.20s (± 0.20%)}	_~	_7.19s	_7.23s	_{p=0.088 n=6}
_{Emit Time}	_{3.97s (± 0.19%)}	_{3.98s (± 0.52%)}	_~	_3.96s	_4.01s	_{p=0.209 n=6}
_{Total Time}	_{14.47s (± 0.19%)}	_{14.46s (± 0.29%)}	_~	_14.41s	_14.53s	_{p=0.373 n=6}
_{material-ui - node (v16.17.1, x64)}
_{Memory used}	_{479,372k (± 0.00%)}	_{479,368k (± 0.00%)}	_~	_479,356k	_479,397k	_{p=0.378 n=6}
_{Parse Time}	_{3.15s (± 0.13%)}	_{3.15s (± 0.47%)}	_~	_3.14s	_3.18s	_{p=1.000 n=6}
_{Bind Time}	_{0.91s (± 0.00%)}	_{0.91s (± 0.00%)}	_~	_0.91s	_0.91s	_{p=1.000 n=6}
_{Check Time}	_{17.89s (± 0.31%)}	_{17.90s (± 0.49%)}	_~	_17.77s	_18.03s	_{p=0.936 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{21.95s (± 0.25%)}	_{21.96s (± 0.42%)}	_~	_21.83s	_22.09s	_{p=1.000 n=6}
_{xstate - node (v16.17.1, x64)}
_{Memory used}	_{542,764k (± 0.01%)}	_{542,800k (± 0.02%)}	_~	_542,710k	_542,996k	_{p=0.873 n=6}
_{Parse Time}	_{3.69s (± 0.14%)}	_{3.69s (± 0.14%)}	_~	_3.69s	_3.70s	_{p=1.000 n=6}
_{Bind Time}	_{1.46s (± 0.28%)}	_{1.46s (± 0.00%)}	_~	_1.46s	_1.46s	_{p=0.405 n=6}
_{Check Time}	_{3.16s (± 0.35%)}	_{3.17s (± 0.26%)}	_{+0.01s (+ 0.42%)}	_3.16s	_3.18s	_{p=0.029 n=6}
_{Emit Time}	_{0.08s (± 0.00%)}	_{0.08s (± 4.99%)}	_~	_0.08s	_0.09s	_{p=0.405 n=6}
_{Total Time}	_{8.39s (± 0.16%)}	_{8.41s (± 0.17%)}	_~	_8.39s	_8.43s	_{p=0.060 n=6}

System info unknown

Hosts

node (v16.17.1, x64)

Scenarios

Angular - node (v16.17.1, x64)
Compiler-Unions - node (v16.17.1, x64)
Monaco - node (v16.17.1, x64)
TFS - node (v16.17.1, x64)
material-ui - node (v16.17.1, x64)
xstate - node (v16.17.1, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsserver

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-UnionsTSServer - node (v16.17.1, x64)}
_{Req 1 - updateOpen}	_{2,475ms (± 0.07%)}	_{2,476ms (± 0.18%)}	_~	_2,469ms	_2,483ms	_{p=0.517 n=6}
_{Req 2 - geterr}	_{5,956ms (± 0.40%)}	_{5,955ms (± 0.67%)}	_~	_5,914ms	_6,029ms	_{p=0.936 n=6}
_{Req 3 - references}	_{340ms (± 0.64%)}	_{340ms (± 0.71%)}	_~	_338ms	_344ms	_{p=1.000 n=6}
_{Req 4 - navto}	_{279ms (± 0.43%)}	_{279ms (± 0.55%)}	_~	_277ms	_280ms	_{p=0.923 n=6}
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_~	_1,356	_1,356	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{84ms (± 2.70%)}	_{85ms (± 4.20%)}	_~	_82ms	_90ms	_{p=0.507 n=6}
_{CompilerTSServer - node (v16.17.1, x64)}
_{Req 1 - updateOpen}	_{2,610ms (± 1.02%)}	_{2,609ms (± 0.87%)}	_~	_2,582ms	_2,636ms	_{p=0.936 n=6}
_{Req 2 - geterr}	_{4,721ms (± 0.18%)}	_{4,728ms (± 0.29%)}	_~	_4,705ms	_4,743ms	_{p=0.230 n=6}
_{Req 3 - references}	_{350ms (± 0.15%)}	_{350ms (± 0.23%)}	_~	_349ms	_351ms	_{p=0.929 n=6}
_{Req 4 - navto}	_{269ms (± 0.28%)}	_{269ms (± 0.28%)}	_~	_268ms	_270ms	_{p=0.487 n=6}
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_~	_1,518	_1,518	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{79ms (± 0.00%)}	_{79ms (± 0.52%)}	_~	_78ms	_79ms	_{p=0.405 n=6}
_{xstateTSServer - node (v16.17.1, x64)}
_{Req 1 - updateOpen}	_{2,717ms (± 0.26%)}	_{2,717ms (± 0.29%)}	_~	_2,703ms	_2,725ms	_{p=1.000 n=6}
_{Req 2 - geterr}	_{1,947ms (± 1.84%)}	_{1,952ms (± 0.37%)}	_~	_1,941ms	_1,959ms	_{p=0.196 n=6}
_{Req 3 - references}	_{126ms (±10.44%)}	_{129ms (± 8.41%)}	_~	_116ms	_143ms	_{p=0.419 n=6}
_{Req 4 - navto}	_{354ms (± 0.64%)}	_{352ms (± 0.43%)}	_~	_350ms	_354ms	_{p=0.103 n=6}
_{Req 5 - completionInfo count}	_{2,071 (± 0.00%)}	_{2,071 (± 0.00%)}	_~	_2,071	_2,071	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{317ms (± 0.83%)}	_{319ms (± 1.58%)}	_~	_315ms	_329ms	_{p=0.934 n=6}

System info unknown

Hosts

node (v16.17.1, x64)

Scenarios

CompilerTSServer - node (v16.17.1, x64)
Compiler-UnionsTSServer - node (v16.17.1, x64)
xstateTSServer - node (v16.17.1, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Startup

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{tsc-startup - node (v16.17.1, x64)}
_{Execution time}	_{155.34ms (± 0.18%)}	_{155.28ms (± 0.15%)}	_{-0.06ms (- 0.04%)}	_154.46ms	_159.44ms	_{p=0.013 n=600}
_{tsserver-startup - node (v16.17.1, x64)}
_{Execution time}	_{230.59ms (± 0.14%)}	_{230.62ms (± 0.13%)}	_~	_229.67ms	_236.65ms	_{p=0.535 n=600}
_{tsserverlibrary-startup - node (v16.17.1, x64)}
_{Execution time}	_{234.25ms (± 0.17%)}	_{235.20ms (± 0.14%)}	_{+0.95ms (+ 0.41%)}	_233.81ms	_237.83ms	_{p=0.000 n=600}
_{typescript-startup - node (v16.17.1, x64)}
_{Execution time}	_{217.65ms (± 0.13%)}	_{217.55ms (± 0.13%)}	_{-0.10ms (- 0.04%)}	_216.26ms	_222.65ms	_{p=0.001 n=600}

System info unknown

Hosts

node (v16.17.1, x64)

Scenarios

tsc-startup - node (v16.17.1, x64)
tsserver-startup - node (v16.17.1, x64)
tsserverlibrary-startup - node (v16.17.1, x64)
typescript-startup - node (v16.17.1, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

TypeScript Bot (typescript-bot) · 2023-08-14T21:47:43Z

Wesley Wigham (@weswigham) Here are the results of running the top-repos suite comparing main and refs/pull/55234/merge:

Everything looks good!

TypeScript Bot (typescript-bot) · 2023-08-14T22:16:59Z

Hey Wesley Wigham (@weswigham), the results of running the DT tests are ready.
Everything looks the same!
You can check the log here.

…-on-cached-entries

Nathan Shively-Sanders (sandersn) · 2023-11-29T23:26:25Z

Wesley Wigham (@weswigham)Anders Hejlsberg (@ahejlsberg) I'd like to merge this at the start of 5.4 development. Is it good to go?

…-on-cached-entries

Mateusz Burzyński (Andarist) · 2023-11-30T09:13:36Z

 // Record this relation as having failed such that we don't attempt the overflowing operation again.
 const id = getRelationKey(source, target, /*intersectionState*/ IntersectionState.None, relation, /*ignoreConstraints*/ false);
- relation.set(id, RelationComparisonResult.Reported | RelationComparisonResult.Failed);
+ relation.set(id, RelationComparisonResult.Failed);


Nathan Shively-Sanders (@sandersn)Anders Hejlsberg (@ahejlsberg) those are lines that were not at all here when I created this PR. When syncing with main now I removed the RelationComparisonResult.Reported - just like it was done everywhere else here but I'm not sure if this should actually be done here.
Those lines come from #55851

Hmm, the CI shows that this actually is important as now it's doing a lot of heavy work twice in this pathological case and the CI timeouts.
If we take a closer look at this baseline (even on main) we might notice that it's actually reporting 2 errors at the same location:
TypeScript/tests/baselines/reference/relationComplexityError.errors.txt
Lines 1 to 2 in 7216ee0
relationComplexityError.ts(12,5): error TS2322: Type 'T1 & T2' is not assignable to type 'T1 | null'.
relationComplexityError.ts(12,5): error TS2859: Excessive complexity comparing types 'T1 & T2' and 'T1 | null'.
So perhaps If I manage to report this once, the perf problems will go away naturally.

Nathan Shively-Sanders (sandersn) · 2023-11-30T14:26:54Z

TypeScript Bot (@typescript-bot) perf test this

TypeScript Bot (typescript-bot) · 2023-11-30T14:26:57Z

Heya Nathan Shively-Sanders (@sandersn), I've started to run the regular perf test suite on this PR at 0611fbb. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-11-30T14:47:30Z

Nathan Shively-Sanders (@sandersn)
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Angular - node (v18.15.0, x64)}
_{Memory used}	_{295,301k (± 0.01%)}	_{295,288k (± 0.01%)}	_~	_295,268k	_295,325k	_{p=0.173 n=6}
_{Parse Time}	_{2.64s (± 0.31%)}	_{2.65s (± 0.37%)}	_~	_2.64s	_2.67s	_{p=0.177 n=6}
_{Bind Time}	_{0.82s (± 0.00%)}	_{0.82s (± 0.50%)}	_~	_0.82s	_0.83s	_{p=0.405 n=6}
_{Check Time}	_{8.05s (± 0.22%)}	_{8.06s (± 0.21%)}	_~	_8.04s	_8.08s	_{p=0.288 n=6}
_{Emit Time}	_{7.08s (± 0.30%)}	_{7.08s (± 0.23%)}	_~	_7.06s	_7.09s	_{p=0.675 n=6}
_{Total Time}	_{18.59s (± 0.10%)}	_{18.61s (± 0.13%)}	_~	_18.58s	_18.64s	_{p=0.122 n=6}
_{Compiler-Unions - node (v18.15.0, x64)}
_{Memory used}	_{195,142k (± 1.55%)}	_{194,070k (± 1.62%)}	_~	_191,169k	_196,952k	_{p=0.092 n=6}
_{Parse Time}	_{1.36s (± 0.90%)}	_{1.36s (± 1.11%)}	_~	_1.34s	_1.38s	_{p=1.000 n=6}
_{Bind Time}	_{0.72s (± 0.57%)}	_{0.72s (± 0.00%)}	_~	_0.72s	_0.72s	_{p=0.405 n=6}
_{Check Time}	_{9.25s (± 0.22%)}	_{9.20s (± 0.47%)}	_{-0.05s (- 0.52%)}	_9.14s	_9.27s	_{p=0.043 n=6}
_{Emit Time}	_{2.64s (± 0.40%)}	_{2.59s (± 0.20%)}	_{-0.04s (- 1.58%)}	_2.59s	_2.60s	_{p=0.004 n=6}
_{Total Time}	_{13.96s (± 0.13%)}	_{13.87s (± 0.29%)}	_{-0.09s (- 0.62%)}	_13.83s	_13.94s	_{p=0.008 n=6}
_{Monaco - node (v18.15.0, x64)}
_{Memory used}	_{347,358k (± 0.00%)}	_{347,348k (± 0.01%)}	_~	_347,312k	_347,368k	_{p=0.574 n=6}
_{Parse Time}	_{2.46s (± 0.49%)}	_{2.46s (± 0.47%)}	_~	_2.44s	_2.47s	_{p=0.502 n=6}
_{Bind Time}	_{0.92s (± 0.82%)}	_{0.93s (± 0.44%)}	_~	_0.92s	_0.93s	_{p=0.100 n=6}
_{Check Time}	_{6.91s (± 0.53%)}	_{6.90s (± 0.52%)}	_~	_6.84s	_6.94s	_{p=0.935 n=6}
_{Emit Time}	_{4.06s (± 0.64%)}	_{4.05s (± 0.45%)}	_~	_4.02s	_4.07s	_{p=0.568 n=6}
_{Total Time}	_{14.35s (± 0.36%)}	_{14.34s (± 0.43%)}	_~	_14.22s	_14.39s	_{p=0.810 n=6}
_{TFS - node (v18.15.0, x64)}
_{Memory used}	_{302,656k (± 0.01%)}	_{302,634k (± 0.01%)}	_~	_302,620k	_302,664k	_{p=0.470 n=6}
_{Parse Time}	_{2.00s (± 0.80%)}	_{2.01s (± 1.04%)}	_~	_1.99s	_2.05s	_{p=0.154 n=6}
_{Bind Time}	_{1.00s (± 1.36%)}	_{1.01s (± 1.32%)}	_~	_0.99s	_1.02s	_{p=0.550 n=6}
_{Check Time}	_{6.26s (± 0.22%)}	_{6.25s (± 0.39%)}	_~	_6.21s	_6.28s	_{p=0.682 n=6}
_{Emit Time}	_{3.59s (± 0.38%)}	_{3.59s (± 0.35%)}	_~	_3.58s	_3.61s	_{p=0.868 n=6}
_{Total Time}	_{12.84s (± 0.18%)}	_{12.85s (± 0.21%)}	_~	_12.81s	_12.89s	_{p=0.406 n=6}
_{material-ui - node (v18.15.0, x64)}
_{Memory used}	_{506,728k (± 0.01%)}	_{506,722k (± 0.01%)}	_~	_506,676k	_506,752k	_{p=0.873 n=6}
_{Parse Time}	_{2.58s (± 0.49%)}	_{2.58s (± 0.45%)}	_~	_2.57s	_2.60s	_{p=0.931 n=6}
_{Bind Time}	_{0.99s (± 0.76%)}	_{0.99s (± 1.23%)}	_~	_0.98s	_1.01s	_{p=0.388 n=6}
_{Check Time}	_{16.78s (± 0.23%)}	_{16.72s (± 0.47%)}	_~	_16.60s	_16.82s	_{p=0.128 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{20.35s (± 0.18%)}	_{20.29s (± 0.36%)}	_~	_20.18s	_20.37s	_{p=0.199 n=6}
_{xstate - node (v18.15.0, x64)}
_{Memory used}	_{512,886k (± 0.01%)}	_{512,828k (± 0.02%)}	_~	_512,736k	_512,987k	_{p=0.199 n=6}
_{Parse Time}	_{3.27s (± 0.25%)}	_{3.28s (± 0.12%)}	_~	_3.27s	_3.28s	_{p=0.248 n=6}
_{Bind Time}	_{1.53s (± 0.36%)}	_{1.54s (± 0.34%)}	_~	_1.53s	_1.54s	_{p=0.640 n=6}
_{Check Time}	_{2.81s (± 0.37%)}	_{2.81s (± 0.54%)}	_~	_2.79s	_2.83s	_{p=0.934 n=6}
_{Emit Time}	_{0.08s (± 0.00%)}	_{0.08s (± 4.99%)}	_~	_0.08s	_0.09s	_{p=0.405 n=6}
_{Total Time}	_{7.70s (± 0.15%)}	_{7.71s (± 0.28%)}	_~	_7.68s	_7.74s	_{p=0.334 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Angular - node (v18.15.0, x64)
Compiler-Unions - node (v18.15.0, x64)
Monaco - node (v18.15.0, x64)
TFS - node (v18.15.0, x64)
material-ui - node (v18.15.0, x64)
xstate - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsserver

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-UnionsTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,337ms (± 0.45%)}	_{2,340ms (± 0.36%)}	_~	_2,330ms	_2,353ms	_{p=1.000 n=6}
_{Req 2 - geterr}	_{5,467ms (± 1.46%)}	_{5,444ms (± 1.72%)}	_~	_5,333ms	_5,539ms	_{p=0.936 n=6}
_{Req 3 - references}	_{328ms (± 1.89%)}	_{329ms (± 1.58%)}	_~	_323ms	_336ms	_{p=1.000 n=6}
_{Req 4 - navto}	_{276ms (± 1.27%)}	_{277ms (± 1.27%)}	_~	_273ms	_280ms	_{p=0.734 n=6}
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_~	_1,356	_1,356	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{87ms (± 5.09%)}	_{85ms (± 6.91%)}	_~	_76ms	_90ms	_{p=0.804 n=6}
_{CompilerTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,474ms (± 0.89%)}	_{2,480ms (± 0.65%)}	_~	_2,459ms	_2,499ms	_{p=0.336 n=6}
_{Req 2 - geterr}	_{4,113ms (± 2.05%)}	_{4,077ms (± 1.61%)}	_~	_4,026ms	_4,207ms	_{p=0.936 n=6}
_{Req 3 - references}	_{339ms (± 1.52%)}	_{342ms (± 1.44%)}	_~	_334ms	_346ms	_{p=0.418 n=6}
_{Req 4 - navto}	_{284ms (± 0.36%)}	_{284ms (± 0.43%)}	_~	_283ms	_286ms	_{p=1.000 n=6}
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_~	_1,518	_1,518	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{83ms (± 7.55%)}	_{86ms (± 6.45%)}	_~	_79ms	_90ms	_{p=0.279 n=6}
_{xstateTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,598ms (± 0.74%)}	_{2,600ms (± 0.68%)}	_~	_2,571ms	_2,625ms	_{p=0.936 n=6}
_{Req 2 - geterr}	_{1,669ms (± 1.80%)}	_{1,668ms (± 1.95%)}	_~	_1,636ms	_1,717ms	_{p=1.000 n=6}
_{Req 3 - references}	_{115ms (± 8.39%)}	_{115ms (± 8.71%)}	_~	_103ms	_124ms	_{p=0.870 n=6}
_{Req 4 - navto}	_{364ms (± 1.01%)}	_{368ms (± 1.40%)}	_~	_365ms	_378ms	_{p=0.618 n=6}
_{Req 5 - completionInfo count}	_{2,073 (± 0.00%)}	_{2,073 (± 0.00%)}	_~	_2,073	_2,073	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{308ms (± 1.31%)}	_{308ms (± 1.44%)}	_~	_305ms	_316ms	_{p=1.000 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

CompilerTSServer - node (v18.15.0, x64)
Compiler-UnionsTSServer - node (v18.15.0, x64)
xstateTSServer - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Startup

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{tsc-startup - node (v18.15.0, x64)}
_{Execution time}	_{153.06ms (± 0.21%)}	_{152.73ms (± 0.18%)}	_{-0.33ms (- 0.21%)}	_151.61ms	_156.15ms	_{p=0.000 n=600}
_{tsserver-startup - node (v18.15.0, x64)}
_{Execution time}	_{228.13ms (± 0.17%)}	_{227.64ms (± 0.15%)}	_{-0.49ms (- 0.22%)}	_226.26ms	_231.15ms	_{p=0.000 n=600}
_{tsserverlibrary-startup - node (v18.15.0, x64)}
_{Execution time}	_{229.40ms (± 0.19%)}	_{229.25ms (± 0.17%)}	_{-0.15ms (- 0.06%)}	_227.61ms	_232.08ms	_{p=0.001 n=600}
_{typescript-startup - node (v18.15.0, x64)}
_{Execution time}	_{229.02ms (± 0.19%)}	_{229.08ms (± 0.20%)}	_~	_227.32ms	_236.04ms	_{p=0.155 n=600}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

tsc-startup - node (v18.15.0, x64)
tsserver-startup - node (v18.15.0, x64)
tsserverlibrary-startup - node (v18.15.0, x64)
typescript-startup - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

Gabriela Araujo Britto (gabritto) · 2024-04-20T00:02:16Z

We had this PR all this time and never merged it? Oh well.

Mateusz Burzyński (Andarist) · 2024-04-20T07:58:02Z

Gabriela Araujo Britto (@gabritto) does it overlap with some other fresh work/issue? I can certainly try to prioritize this PR more and bring it up to speed

Jake Bailey (jakebailey) · 2024-04-20T14:44:35Z

Yeah, #57842, since we may report diagnostics out of order and therefore elaborations may jump around.

Gabriela Araujo Britto (gabritto) · 2024-04-22T22:40:09Z

Yeah, #57842, since we may report diagnostics out of order and therefore elaborations may jump around.

To give more details, I'm doing some diagnostics work in preparation for #57842. Right now I'm trying to have less duplicated diagnostics, but next on my list is to provide assignability elaboration consistently in editor scenarios, and as a consequence get rid of some of the diagnostics differences that arise when using #57842 vs not using them.
Where things might overlap is that the fact that assignability diagnostics varies depending on cached relations sometimes makes it harder to detect duplication, so making the assignability diagnostics more consistent could help with that as well.

Mateusz Burzyński (Andarist) · 2024-07-15T18:33:54Z

This work has landed as part of #58859

Gabriela Araujo Britto (gabritto) · 2024-07-15T18:50:16Z

This work has landed as part of #58859

Thanks for doing this!

Don't skip elaborations when reporting errors for cached failed relat…
eaba5a1
…ions

TypeScript Bot (typescript-bot) added the For Backlog Bug PRs that fix a backlog bug label Aug 1, 2023

Mateusz Burzyński (Andarist) commented Aug 1, 2023
View reviewed changes

Comment threadsrc/compiler/checker.ts

Mateusz Burzyński (Andarist) added 4 commits August 6, 2023 20:49

Merge remote-tracking branch 'origin/main' into dont-skip-elaboration…
c7e33ff
…-on-cached-entries

update existing baselines
8a0b6c0

Remove RelationComparisonResult.Reported entirely
16085d5

new test case
4a88fbd

Mateusz Burzyński (Andarist)force-pushed the dont-skip-elaboration-on-cached-entries branch from 2bf3cf9 to 4a88fbdCompare August 6, 2023 19:51

Nathan Shively-Sanders (sandersn) requested a review from Anders Hejlsberg (ahejlsberg)August 11, 2023 22:21

Nathan Shively-Sanders (sandersn) assigned Anders Hejlsberg (ahejlsberg)Aug 11, 2023

Nathan Shively-Sanders (sandersn) requested a review from Wesley Wigham (weswigham)August 11, 2023 22:22

Wesley Wigham (weswigham) approved these changes Aug 14, 2023
View reviewed changes

Merge remote-tracking branch 'origin/main' into dont-skip-elaboration…
add4655
…-on-cached-entries

Mateusz Burzyński (Andarist) added 2 commits November 30, 2023 10:07

Merge remote-tracking branch 'origin/main' into dont-skip-elaboration…
09a199d
…-on-cached-entries

Update RelationComparisonResult
b29b639

Mateusz Burzyński (Andarist) commented Nov 30, 2023
View reviewed changes

update baseline
0611fbb

Gabriela Araujo Britto (gabritto) mentioned this pull request Jun 13, 2024
When reporting errors, compute diagnostics even if type relation has cached result #58859
Merged

Mateusz Burzyński (Andarist) closed this Jul 15, 2024

Nathan Shively-Sanders (sandersn) removed this from PR Backlog Apr 22, 2025

Microsoft (microsoft) locked as resolved and limited conversation to collaborators Oct 16, 2025

	relationComplexityError.ts(12,5): error TS2322: Type 'T1 & T2' is not assignable to type 'T1 \| null'.
	relationComplexityError.ts(12,5): error TS2859: Excessive complexity comparing types 'T1 & T2' and 'T1 \| null'.

Uh oh!

Conversation

Mateusz Burzyński (Andarist) commented Aug 1, 2023

Uh oh!

Uh oh!

Ryan Cavanaugh (RyanCavanaugh) commented Aug 2, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Aug 2, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Aug 2, 2023

Uh oh!

Mateusz Burzyński (Andarist) commented Aug 6, 2023

Uh oh!

Wesley Wigham (weswigham) commented Aug 14, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Aug 14, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Aug 14, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Aug 14, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Aug 14, 2023

Compiler

tsserver

Startup

Uh oh!

TypeScript Bot (typescript-bot) commented Aug 14, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Aug 14, 2023

Uh oh!

Nathan Shively-Sanders (sandersn) commented Nov 29, 2023

Uh oh!

Mateusz Burzyński (Andarist)Nov 30, 2023

Choose a reason for hiding this comment

Uh oh!

Mateusz Burzyński (Andarist)Nov 30, 2023

Choose a reason for hiding this comment

Uh oh!

Nathan Shively-Sanders (sandersn) commented Nov 30, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Nov 30, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Nov 30, 2023

Compiler

tsserver

Startup

Uh oh!

Gabriela Araujo Britto (gabritto) commented Apr 20, 2024

Uh oh!

Mateusz Burzyński (Andarist) commented Apr 20, 2024

Uh oh!

Jake Bailey (jakebailey) commented Apr 20, 2024

Uh oh!

Gabriela Araujo Britto (gabritto) commented Apr 22, 2024

Uh oh!

Mateusz Burzyński (Andarist) commented Jul 15, 2024

Uh oh!

Gabriela Araujo Britto (gabritto) commented Jul 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

TypeScript Bot (typescript-bot) commented Aug 2, 2023•
edited
Loading

TypeScript Bot (typescript-bot) commented Aug 14, 2023•
edited
Loading

TypeScript Bot (typescript-bot) commented Aug 14, 2023•
edited
Loading

TypeScript Bot (typescript-bot) commented Aug 14, 2023•
edited
Loading

TypeScript Bot (typescript-bot) commented Nov 30, 2023•
edited
Loading