Skip to content

Improve diagnostics deduplication 2 - #58318

Merged
Gabriela Araujo Britto (gabritto) merged 11 commits into
mainfrom
gabritto/issue58207pt2
Apr 29, 2024
Merged

Improve diagnostics deduplication 2#58318
Gabriela Araujo Britto (gabritto) merged 11 commits into
mainfrom
gabritto/issue58207pt2

Conversation

@gabritto

@gabrittoGabriela Araujo Britto (gabritto) commented Apr 25, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Fixes#58207.
Follow-up to #58220.

This applies deduplication for the two scenarios I'm aware of where we can produce different diagnostics (with different message and code) that report the same problem at the same location, depending on the current state of the type checker:
(1) not found diagnostics that may or may not have suggestions, and (2) type comparison diagnostics that may or may not suppress the top part of a diagnostic chain depending on whether we have a more useful elaboration or not.

Case (1) depends on the state of the checker because we have a constant limit on how many times we try to look for suggestion, for performance reasons. Case (2) depends on the state of the checker because we don't provide a more detailed elaboration when we already have a cached result for the type comparison.

The general idea here is to make these different diagnostics that refer to the same problem point to the same canonical diagnostic, and this canonical diagnostic is used for comparison and deciding equivalence.
e.g. in case (1), we can produce both "Cannot find name 'x'." and "Cannot find name 'x'. Did you mean 'y'?", so now the diagnostic with message "Cannot find name 'x'. Did you mean 'y'?" will have a canonical diagnostic of "Cannot find name 'x'.", and will be considered equivalent to "Cannot find name 'x'." (but the diagnostic with the suggestion will be preferred).

@typescript-botTypeScript Bot (typescript-bot) added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Apr 25, 2024
@gabritto

Gabriela Araujo Britto (gabritto) commented Apr 25, 2024

Copy link
Copy Markdown
MemberAuthor

TypeScript Bot (@typescript-bot) test it

@typescript-bot

TypeScript Bot (typescript-bot) commented Apr 25, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
test top400✅ Started✅ Results
user test this✅ Started✅ Results
run dt✅ Started✅ Results
perf test this faster✅ Started👀 Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Apr 25, 2024

Copy link
Copy Markdown
Contributor

Hey Gabriela Araujo Britto (@gabritto), the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot

TypeScript Bot (typescript-bot) commented Apr 25, 2024

Copy link
Copy Markdown
Contributor

Gabriela Araujo Britto (@gabritto) Here are the results of running the user tests comparing main and refs/pull/58318/merge:

Everything looks good!

@typescript-bot

TypeScript Bot (typescript-bot) commented Apr 25, 2024

Copy link
Copy Markdown
Contributor

Gabriela Araujo Britto (@gabritto)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3030~~~p=1.000 n=6
Symbols62,15462,154~~~p=1.000 n=6
Types50,27350,273~~~p=1.000 n=6
Memory used193,385k (± 0.97%)192,935k (± 0.73%)~192,128k195,786kp=0.936 n=6
Parse Time1.35s (± 2.01%)1.35s (± 1.30%)~1.32s1.36sp=1.000 n=6
Bind Time0.72s (± 0.57%)0.72s~~~p=0.405 n=6
Check Time9.56s (± 0.41%)9.60s (± 0.53%)~9.55s9.69sp=0.257 n=6
Emit Time2.62s (± 0.62%)2.62s (± 0.52%)~2.60s2.64sp=0.512 n=6
Total Time14.25s (± 0.22%)14.28s (± 0.29%)~14.24s14.35sp=0.295 n=6
angular-1 - node (v18.15.0, x64)
Errors55~~~p=1.000 n=6
Symbols945,172945,172~~~p=1.000 n=6
Types408,068408,068~~~p=1.000 n=6
Memory used1,221,995k (± 0.00%)1,222,000k (± 0.00%)~1,221,935k1,222,047kp=0.689 n=6
Parse Time6.92s (± 0.66%)6.91s (± 0.35%)~6.88s6.95sp=0.686 n=6
Bind Time1.87s (± 0.74%)1.87s (± 0.73%)~1.84s1.88sp=0.933 n=6
Check Time31.39s (± 0.16%)31.32s (± 0.42%)~31.13s31.47sp=0.520 n=6
Emit Time14.64s (± 0.67%)14.66s (± 0.46%)~14.53s14.72sp=0.229 n=6
Total Time54.83s (± 0.23%)54.76s (± 0.33%)~54.44s54.92sp=0.748 n=6
mui-docs - node (v18.15.0, x64)
Errors55~~~p=1.000 n=6
Symbols1,954,5981,954,598~~~p=1.000 n=6
Types676,387676,387~~~p=1.000 n=6
Memory used1,753,330k (± 0.00%)1,753,397k (± 0.00%)+67k (+ 0.00%)1,753,348k1,753,416kp=0.008 n=6
Parse Time6.88s (± 0.17%)6.89s (± 0.16%)~6.88s6.91sp=0.244 n=6
Bind Time2.30s (± 0.27%)2.30s (± 0.33%)~2.29s2.31sp=0.718 n=6
Check Time56.78s (± 0.51%)56.96s (± 0.44%)~56.63s57.29sp=0.261 n=6
Emit Time0.14s (± 2.95%)0.14s~~~p=0.405 n=6
Total Time66.10s (± 0.44%)66.29s (± 0.36%)~65.97s66.60sp=0.173 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,215,0711,215,421+350 (+ 0.03%)~~p=0.001 n=6
Types257,359257,437+78 (+ 0.03%)~~p=0.001 n=6
Memory used2,322,684k (± 0.03%)2,324,520k (± 0.02%)+1,836k (+ 0.08%)2,323,931k2,325,243kp=0.005 n=6
Parse Time5.06s (± 0.86%)5.07s (± 0.77%)~5.02s5.13sp=0.873 n=6
Bind Time1.86s (± 0.66%)1.89s (± 0.80%)+0.02s (+ 1.07%)1.86s1.90sp=0.048 n=6
Check Time33.94s (± 0.29%)33.84s (± 0.27%)~33.71s33.96sp=0.149 n=6
Emit Time2.71s (± 2.09%)2.65s (± 2.10%)~2.59s2.74sp=0.078 n=6
Total Time43.61s (± 0.33%)43.46s (± 0.20%)~43.30s43.56sp=0.128 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,215,0711,215,421+350 (+ 0.03%)~~p=0.001 n=6
Types257,359257,437+78 (+ 0.03%)~~p=0.001 n=6
Memory used2,397,952k (± 0.04%)2,398,954k (± 0.03%)~2,398,073k2,399,698kp=0.066 n=6
Parse Time6.32s (± 0.92%)6.33s (± 0.78%)~6.27s6.41sp=0.748 n=6
Bind Time2.00s (± 1.38%)2.03s (± 1.20%)~1.99s2.06sp=0.146 n=6
Check Time40.49s (± 0.44%)40.58s (± 0.33%)~40.41s40.72sp=0.575 n=6
Emit Time3.27s (± 2.71%)3.26s (± 2.68%)~3.13s3.34sp=0.810 n=6
Total Time52.09s (± 0.27%)52.21s (± 0.46%)~51.96s52.53sp=0.471 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols256,220256,295+75 (+ 0.03%)~~p=0.001 n=6
Types103,640103,661+21 (+ 0.02%)~~p=0.001 n=6
Memory used424,115k (± 0.01%)424,284k (± 0.01%)+169k (+ 0.04%)424,238k424,361kp=0.005 n=6
Parse Time3.49s (± 1.19%)3.51s (± 0.88%)~3.46s3.55sp=0.629 n=6
Bind Time1.31s (± 0.79%)1.33s (± 3.02%)~1.31s1.41sp=0.210 n=6
Check Time18.21s (± 0.37%)18.26s (± 0.30%)~18.19s18.32sp=0.127 n=6
Emit Time1.40s (± 0.84%)1.37s (± 1.87%)-0.03s (- 2.15%)1.33s1.41sp=0.042 n=6
Total Time24.40s (± 0.39%)24.47s (± 0.33%)~24.40s24.62sp=0.420 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors3535~~~p=1.000 n=6
Symbols224,824224,824~~~p=1.000 n=6
Types93,39093,390~~~p=1.000 n=6
Memory used369,284k (± 0.01%)369,337k (± 0.02%)~369,285k369,456kp=0.108 n=6
Parse Time3.67s (± 0.53%)3.68s (± 0.64%)~3.65s3.71sp=0.568 n=6
Bind Time1.94s (± 0.69%)1.94s (± 0.96%)~1.91s1.96sp=0.935 n=6
Check Time19.44s (± 0.37%)19.46s (± 0.18%)~19.41s19.50sp=0.936 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time25.05s (± 0.27%)25.07s (± 0.23%)~25.00s25.16sp=0.575 n=6
vscode - node (v18.15.0, x64)
Errors44~~~p=1.000 n=6
Symbols2,795,8942,795,894~~~p=1.000 n=6
Types949,958949,958~~~p=1.000 n=6
Memory used2,924,603k (± 0.01%)2,924,669k (± 0.00%)~2,924,526k2,924,783kp=0.378 n=6
Parse Time13.44s (± 0.58%)13.43s (± 0.28%)~13.38s13.46sp=0.808 n=6
Bind Time4.04s (± 0.31%)4.08s (± 1.80%)~4.04s4.23sp=0.087 n=6
Check Time72.24s (± 0.46%)72.70s (± 0.66%)+0.46s (+ 0.64%)72.23s73.30sp=0.045 n=6
Emit Time21.32s (± 9.87%)22.05s (± 9.17%)~19.38s23.53sp=0.378 n=6
Total Time111.04s (± 1.84%)112.26s (± 1.89%)~109.28s114.01sp=0.093 n=6
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols265,853265,853~~~p=1.000 n=6
Types108,438108,438~~~p=1.000 n=6
Memory used410,402k (± 0.03%)410,367k (± 0.02%)~410,285k410,429kp=0.810 n=6
Parse Time4.90s (± 0.84%)4.89s (± 0.98%)~4.80s4.93sp=0.746 n=6
Bind Time2.07s (± 0.83%)2.06s (± 0.51%)~2.05s2.08sp=0.566 n=6
Check Time21.15s (± 0.22%)21.18s (± 0.41%)~21.01s21.26sp=0.261 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time28.13s (± 0.29%)28.14s (± 0.27%)~28.02s28.21sp=1.000 n=6
xstate-main - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols523,981523,981~~~p=1.000 n=6
Types178,708178,708~~~p=1.000 n=6
Memory used461,250k (± 0.02%)461,296k (± 0.02%)~461,212k461,461kp=0.689 n=6
Parse Time3.99s (± 0.47%)4.02s (± 0.84%)~3.99s4.08sp=0.073 n=6
Bind Time1.46s (± 1.54%)1.47s (± 0.67%)~1.46s1.48sp=0.324 n=6
Check Time22.52s (± 0.42%)22.52s (± 0.56%)~22.31s22.66sp=1.000 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time27.98s (± 0.35%)28.02s (± 0.40%)~27.85s28.17sp=0.689 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@gabritto

Gabriela Araujo Britto (gabritto) commented Apr 25, 2024

Copy link
Copy Markdown
MemberAuthor

TypeScript Bot (@typescript-bot) perf test this

@typescript-bot

TypeScript Bot (typescript-bot) commented Apr 25, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test this✅ Started👀 Results

@gabrittoGabriela Araujo Britto (gabritto) changed the title Gabritto/issue58207pt2Improve diagnostics deduplication 2Apr 25, 2024
@typescript-bot

TypeScript Bot (typescript-bot) commented Apr 25, 2024

Copy link
Copy Markdown
Contributor

Gabriela Araujo Britto (@gabritto)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3030~~~p=1.000 n=6
Symbols62,15462,154~~~p=1.000 n=6
Types50,27350,273~~~p=1.000 n=6
Memory used192,709k (± 0.73%)192,785k (± 0.77%)~192,065k195,786kp=0.936 n=6
Parse Time1.65s (± 1.52%)1.63s (± 1.58%)~1.61s1.67sp=0.514 n=6
Bind Time0.87s (± 0.94%)0.86s (± 0.94%)~0.85s0.87sp=0.666 n=6
Check Time11.34s (± 0.35%)11.33s (± 0.42%)~11.27s11.39sp=0.688 n=6
Emit Time3.15s (± 0.80%)3.14s (± 0.56%)~3.11s3.16sp=0.517 n=6
Total Time17.00s (± 0.27%)16.96s (± 0.17%)~16.93s17.00sp=0.170 n=6
angular-1 - node (v18.15.0, x64)
Errors55~~~p=1.000 n=6
Symbols945,172945,172~~~p=1.000 n=6
Types408,068408,068~~~p=1.000 n=6
Memory used1,221,963k (± 0.01%)1,221,999k (± 0.00%)~1,221,950k1,222,087kp=0.471 n=6
Parse Time8.21s (± 0.37%)8.22s (± 0.33%)~8.18s8.25sp=0.375 n=6
Bind Time2.23s (± 0.57%)2.23s (± 0.49%)~2.22s2.25sp=0.869 n=6
Check Time36.66s (± 0.28%)36.68s (± 0.32%)~36.58s36.90sp=0.630 n=6
Emit Time17.47s (± 0.45%)17.45s (± 0.35%)~17.37s17.52sp=0.575 n=6
Total Time64.57s (± 0.18%)64.58s (± 0.19%)~64.49s64.76sp=0.521 n=6
mui-docs - node (v18.15.0, x64)
Errors55~~~p=1.000 n=6
Symbols1,954,6021,954,602~~~p=1.000 n=6
Types676,397676,397~~~p=1.000 n=6
Memory used1,753,359k (± 0.00%)1,753,398k (± 0.00%)~1,753,322k1,753,443kp=0.128 n=6
Parse Time9.99s (± 0.48%)9.98s (± 0.46%)~9.92s10.06sp=0.872 n=6
Bind Time3.36s (± 0.70%)3.37s (± 0.83%)~3.34s3.41sp=0.935 n=6
Check Time82.24s (± 0.41%)82.15s (± 0.41%)~81.65s82.54sp=0.810 n=6
Emit Time0.20s (± 2.54%)0.21s (± 6.37%)~0.19s0.22sp=0.546 n=6
Total Time95.79s (± 0.39%)95.71s (± 0.39%)~95.10s96.08sp=0.810 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,215,0711,215,421+350 (+ 0.03%)~~p=0.001 n=6
Types257,359257,437+78 (+ 0.03%)~~p=0.001 n=6
Memory used2,323,779k (± 0.05%)2,325,334k (± 0.02%)+1,556k (+ 0.07%)2,324,591k2,325,767kp=0.031 n=6
Parse Time6.16s (± 0.79%)6.13s (± 1.28%)~6.08s6.28sp=0.470 n=6
Bind Time2.22s (± 1.25%)2.24s (± 0.40%)~2.23s2.25sp=0.172 n=6
Check Time39.97s (± 0.43%)39.90s (± 0.18%)~39.78s39.99sp=0.689 n=6
Emit Time3.23s (± 4.20%)3.08s (± 0.99%)🟩-0.15s (- 4.74%)3.04s3.12sp=0.013 n=6
Total Time51.59s (± 0.58%)51.37s (± 0.22%)~51.22s51.56sp=0.173 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,215,0711,215,421+350 (+ 0.03%)~~p=0.001 n=6
Types257,359257,437+78 (+ 0.03%)~~p=0.001 n=6
Memory used2,398,198k (± 0.04%)2,399,227k (± 0.03%)~2,398,272k2,399,905kp=0.128 n=6
Parse Time7.84s (± 0.92%)7.85s (± 0.77%)~7.75s7.91sp=0.748 n=6
Bind Time2.46s (± 1.52%)2.46s (± 0.49%)~2.45s2.48sp=0.747 n=6
Check Time50.09s (± 0.36%)50.25s (± 0.47%)~50.02s50.68sp=0.471 n=6
Emit Time4.01s (± 4.10%)3.96s (± 1.38%)~3.89s4.03sp=0.521 n=6
Total Time64.42s (± 0.33%)64.54s (± 0.41%)~64.27s65.00sp=0.471 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols256,220256,295+75 (+ 0.03%)~~p=0.001 n=6
Types103,640103,661+21 (+ 0.02%)~~p=0.001 n=6
Memory used424,088k (± 0.01%)424,291k (± 0.01%)+203k (+ 0.05%)424,253k424,341kp=0.005 n=6
Parse Time4.31s (± 0.60%)4.32s (± 0.45%)~4.29s4.34sp=0.518 n=6
Bind Time1.60s (± 1.46%)1.59s (± 0.32%)~1.59s1.60sp=0.276 n=6
Check Time22.45s (± 0.45%)22.52s (± 0.29%)~22.44s22.61sp=0.148 n=6
Emit Time1.74s (± 1.65%)1.72s (± 0.97%)~1.70s1.75sp=0.192 n=6
Total Time30.11s (± 0.44%)30.16s (± 0.19%)~30.10s30.25sp=0.572 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors3535~~~p=1.000 n=6
Symbols224,824224,824~~~p=1.000 n=6
Types93,39093,390~~~p=1.000 n=6
Memory used369,276k (± 0.01%)369,385k (± 0.05%)~369,251k369,730kp=0.173 n=6
Parse Time3.68s (± 0.83%)3.68s (± 0.68%)~3.64s3.70sp=0.807 n=6
Bind Time1.93s (± 1.01%)1.93s (± 1.00%)~1.91s1.96sp=0.872 n=6
Check Time19.47s (± 0.38%)19.44s (± 0.24%)~19.40s19.51sp=0.460 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time25.07s (± 0.32%)25.05s (± 0.19%)~24.99s25.13sp=0.746 n=6
vscode - node (v18.15.0, x64)
Errors44~~~p=1.000 n=6
Symbols2,795,8942,795,894~~~p=1.000 n=6
Types949,958949,958~~~p=1.000 n=6
Memory used2,924,684k (± 0.00%)2,924,772k (± 0.00%)~2,924,689k2,924,848kp=0.092 n=6
Parse Time16.65s (± 0.42%)16.58s (± 0.22%)~16.53s16.63sp=0.077 n=6
Bind Time4.97s (± 0.55%)4.97s (± 0.32%)~4.94s4.98sp=0.742 n=6
Check Time88.66s (± 0.60%)88.48s (± 0.18%)~88.20s88.65sp=0.936 n=6
Emit Time24.75s (± 7.62%)23.91s (± 0.93%)~23.62s24.20sp=0.575 n=6
Total Time135.03s (± 1.38%)133.94s (± 0.27%)~133.44s134.35sp=0.298 n=6
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols265,853265,853~~~p=1.000 n=6
Types108,438108,438~~~p=1.000 n=6
Memory used410,405k (± 0.03%)410,345k (± 0.01%)~410,303k410,410kp=0.378 n=6
Parse Time3.25s (± 0.96%)3.25s (± 0.79%)~3.21s3.27sp=0.624 n=6
Bind Time1.41s (± 0.98%)1.41s (± 0.98%)~1.38s1.42sp=1.000 n=6
Check Time14.46s (± 0.24%)14.46s (± 0.26%)~14.41s14.51sp=1.000 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time19.11s (± 0.31%)19.11s (± 0.11%)~19.09s19.14sp=0.936 n=6
xstate-main - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols523,981523,981~~~p=1.000 n=6
Types178,708178,708~~~p=1.000 n=6
Memory used461,214k (± 0.02%)461,228k (± 0.01%)~461,178k461,306kp=0.521 n=6
Parse Time3.25s (± 0.64%)3.23s (± 0.57%)~3.22s3.26sp=0.182 n=6
Bind Time1.18s (± 0.46%)1.18s (± 0.76%)~1.17s1.19sp=0.341 n=6
Check Time18.16s (± 0.29%)18.19s (± 0.31%)~18.12s18.27sp=1.000 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time22.59s (± 0.26%)22.60s (± 0.28%)~22.51s22.67sp=1.000 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

tsserver

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,364ms (± 0.37%)2,347ms (± 0.48%)-16ms (- 0.69%)2,334ms2,368msp=0.031 n=6
Req 2 - geterr5,102ms (± 0.53%)5,146ms (± 0.74%)+44ms (+ 0.87%)5,101ms5,193msp=0.031 n=6
Req 3 - references288ms (± 1.40%)289ms (± 1.34%)~286ms296msp=0.462 n=6
Req 4 - navto227ms (± 0.72%)226ms (± 0.56%)~225ms228msp=0.183 n=6
Req 5 - completionInfo count1,3571,357~~~p=1.000 n=6
Req 5 - completionInfo80ms (± 8.28%)80ms (± 8.54%)~76ms93msp=0.867 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,483ms (± 0.79%)2,470ms (± 1.18%)~2,417ms2,501msp=0.470 n=6
Req 2 - geterr3,833ms (± 0.29%)3,834ms (± 0.17%)~3,828ms3,846msp=0.748 n=6
Req 3 - references301ms (± 0.25%)299ms (± 0.46%)-3ms (- 0.89%)297ms301msp=0.014 n=6
Req 4 - navto228ms (± 0.24%)227ms (± 0.33%)~226ms228msp=0.476 n=6
Req 5 - completionInfo count1,5191,519~~~p=1.000 n=6
Req 5 - completionInfo75ms (± 7.01%)74ms (± 6.45%)~71ms84msp=0.742 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen5,182ms (± 0.22%)5,184ms (± 0.19%)~5,168ms5,199msp=0.575 n=6
Req 2 - geterr1,129ms (± 0.58%)1,130ms (± 0.76%)~1,125ms1,147msp=1.000 n=6
Req 3 - references85ms (± 1.21%)85ms (± 1.22%)~84ms86msp=0.311 n=6
Req 4 - navto447ms (± 0.26%)449ms (± 0.76%)~446ms454msp=0.184 n=6
Req 5 - completionInfo count3,4133,413~~~p=1.000 n=6
Req 5 - completionInfo839ms (± 0.53%)841ms (± 1.55%)~833ms867msp=0.809 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

startup

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
tsc-startup - node (v18.15.0, x64)
Execution time156.93ms (± 0.16%)157.02ms (± 0.18%)+0.09ms (+ 0.06%)155.86ms160.96msp=0.007 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time239.28ms (± 0.14%)239.35ms (± 0.14%)+0.07ms (+ 0.03%)238.20ms242.24msp=0.045 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time235.09ms (± 0.12%)235.35ms (± 0.15%)+0.26ms (+ 0.11%)233.74ms240.83msp=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time235.42ms (± 0.80%)235.29ms (± 0.14%)-0.13ms (- 0.05%)233.67ms240.98msp=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@typescript-bot

TypeScript Bot (typescript-bot) commented Apr 25, 2024

Copy link
Copy Markdown
Contributor

Gabriela Araujo Britto (@gabritto) Here are the results of running the top 400 repos comparing main and refs/pull/58318/merge:

Everything looks good!

Merged
@gabritto
Gabriela Araujo Britto (gabritto) changed the base branch from main to gabritto/issue58207April 26, 2024 19:38
@gabritto
Gabriela Araujo Britto (gabritto) marked this pull request as ready for review April 26, 2024 19:50
andrewbranch
Andrew Branch (andrewbranch) approved these changes Apr 26, 2024
View reviewed changes

@andrewbranchAndrew Branch (andrewbranch) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me, though I would wait for another review from someone who knows more about captureErrorCalculationState/resetErrorInfo

andrewbranch
Andrew Branch (andrewbranch) reviewed Apr 26, 2024
View reviewed changes
Comment threadtests/cases/compiler/duplicateErrorNameNotFound.ts

export type {
RoomInterface
} No newline at end of file

@andrewbranchAndrew Branch (andrewbranch)Apr 26, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did these tests actually produce duplicates, or were they just useful in getting near the code path for debugging?

@gabrittoGabriela Araujo Britto (gabritto)Apr 26, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They produced duplicates, yes.

weswigham
Wesley Wigham (weswigham) reviewed Apr 26, 2024
View reviewed changes
Comment threadtests/baselines/reference/mappedTypeRecursiveInference.types
Type Count: 10,000
Instantiation count: 250,000
Symbol count: 100,000
Symbol count: 250,000

@weswighamWesley Wigham (weswigham)Apr 26, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...That's a big bump. Is this just caused by the extra reportRelationError call expanding the recursive type way more? Still feels like a lot more symbols.

@weswighamWesley Wigham (weswigham)Apr 26, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't necessarily something that aughta get fixed in this PR, but understanding where so many symbols popped in from could be useful. It might indicate that we're not caching something we aughta be.

@gabrittoGabriela Araujo Britto (gabritto)Apr 26, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just caused by the extra reportRelationError call expanding the recursive type way more?

I believe so. Since this test has an infinitely recurring type, it didn't seem like a great reason for concern to me, but I could be wrong.

sheetalkamat
Sheetal Nandi (sheetalkamat) reviewed Apr 26, 2024
View reviewed changes
Comment threadsrc/compiler/utilities.ts
category: messageChain.category,
messageText: messageChain.next ? messageChain : messageChain.messageText,
relatedInformation,
canonicalHead: messageChain.canonicalHead,

@sheetalkamatSheetal Nandi (sheetalkamat)Apr 26, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be serialized in buildInfo when serializing diagnostic. Will we need this to deduplicate across multiple files?

@gabrittoGabriela Araujo Britto (gabritto)Apr 26, 2024•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will we need this to deduplicate across multiple files?

I don't think so, this should only be relevant for deduplicating diagnostics of a single file.

Does this need to be serialized in buildInfo when serializing diagnostic?

What's the criterion for needing to serialize this in buildInfo?

@sheetalkamatSheetal Nandi (sheetalkamat)Apr 27, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you would need this info to deduplicate with errors from other files or errors emitted for same file in other phase (which obviously isn't true - like say same error is reported by declaration emit and semantic diagnostic just giving hypothetical example here)

@sheetalkamatSheetal Nandi (sheetalkamat)Apr 27, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given this doesnt get used beyond file boundaries this should be ok to be not serialized.
That is checking files in any order will result in same error in that file (which we were anyways assuming till now) this is good.

@weswigham

Wesley Wigham (weswigham) commented Apr 26, 2024

Copy link
Copy Markdown
Member

TypeScript Bot (@typescript-bot) perf test this faster

Juuuust in case

@typescript-bot

TypeScript Bot (typescript-bot) commented Apr 26, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test this faster✅ Started👀 Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Apr 26, 2024

Copy link
Copy Markdown
Contributor

Wesley Wigham (@weswigham)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3030~~~p=1.000 n=6
Symbols62,15462,154~~~p=1.000 n=6
Types50,27350,273~~~p=1.000 n=6
Memory used192,439k (± 0.78%)193,658k (± 1.03%)~191,803k195,613kp=0.230 n=6
Parse Time1.35s (± 0.77%)1.35s (± 1.09%)~1.33s1.36sp=0.676 n=6
Bind Time0.72s0.72s~~~p=1.000 n=6
Check Time9.57s (± 0.24%)9.60s (± 0.42%)~9.54s9.65sp=0.291 n=6
Emit Time2.63s (± 0.31%)2.63s (± 0.52%)~2.61s2.65sp=0.605 n=6
Total Time14.26s (± 0.14%)14.29s (± 0.25%)~14.24s14.33sp=0.326 n=6
angular-1 - node (v18.15.0, x64)
Errors55~~~p=1.000 n=6
Symbols945,073945,073~~~p=1.000 n=6
Types408,052408,052~~~p=1.000 n=6
Memory used1,221,332k (± 0.00%)1,221,323k (± 0.00%)~1,221,287k1,221,396kp=0.810 n=6
Parse Time8.20s (± 0.53%)8.17s (± 0.44%)~8.12s8.22sp=0.260 n=6
Bind Time2.22s (± 1.04%)2.21s (± 0.68%)~2.20s2.24sp=1.000 n=6
Check Time36.33s (± 0.33%)36.30s (± 0.40%)~36.06s36.44sp=0.936 n=6
Emit Time17.23s (± 0.43%)17.29s (± 0.97%)~17.05s17.57sp=0.378 n=6
Total Time63.99s (± 0.26%)63.97s (± 0.38%)~63.64s64.27sp=1.000 n=6
mui-docs - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,954,5351,954,535~~~p=1.000 n=6
Types676,397676,397~~~p=1.000 n=6
Memory used1,752,887k (± 0.00%)1,752,949k (± 0.00%)~1,752,891k1,753,021kp=0.078 n=6
Parse Time8.06s (± 0.93%)8.03s (± 0.51%)~7.98s8.09sp=0.520 n=6
Bind Time2.72s (± 0.43%)2.72s (± 0.19%)~2.72s2.73sp=0.542 n=6
Check Time67.18s (± 0.24%)67.18s (± 0.27%)~67.02s67.51sp=0.810 n=6
Emit Time0.16s0.16s (± 2.52%)~0.16s0.17sp=0.405 n=6
Total Time78.12s (± 0.16%)78.10s (± 0.21%)~77.92s78.37sp=0.467 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,208,9151,209,024+109 (+ 0.01%)~~p=0.001 n=6
Types255,383255,395+12 (+ 0.00%)~~p=0.001 n=6
Memory used2,307,557k (± 0.04%)2,307,399k (± 0.02%)~2,306,588k2,308,227kp=0.810 n=6
Parse Time6.07s (± 0.50%)6.06s (± 0.61%)~6.00s6.11sp=0.688 n=6
Bind Time2.23s (± 0.90%)2.23s (± 0.72%)~2.21s2.25sp=1.000 n=6
Check Time39.82s (± 0.46%)39.82s (± 0.35%)~39.65s40.06sp=0.936 n=6
Emit Time3.14s (± 2.75%)3.16s (± 4.42%)~3.04s3.41sp=0.936 n=6
Total Time51.28s (± 0.34%)51.29s (± 0.53%)~50.99s51.63sp=1.000 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,208,9151,209,024+109 (+ 0.01%)~~p=0.001 n=6
Types255,383255,395+12 (+ 0.00%)~~p=0.001 n=6
Memory used2,382,115k (± 0.04%)2,382,713k (± 0.04%)~2,381,083k2,384,129kp=0.378 n=6
Parse Time7.67s (± 0.73%)7.67s (± 0.53%)~7.62s7.74sp=0.936 n=6
Bind Time2.50s (± 0.93%)2.55s (± 3.81%)~2.51s2.75sp=0.462 n=6
Check Time49.68s (± 0.28%)49.64s (± 0.44%)~49.24s49.90sp=0.810 n=6
Emit Time3.90s (± 2.33%)3.89s (± 1.66%)~3.81s3.98sp=0.689 n=6
Total Time63.77s (± 0.30%)63.76s (± 0.29%)~63.58s64.10sp=0.936 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols253,659253,639-20 (- 0.01%)~~p=0.001 n=6
Types101,551101,548-3 (- 0.00%)~~p=0.001 n=6
Memory used419,363k (± 0.01%)419,398k (± 0.01%)~419,373k419,441kp=0.199 n=6
Parse Time4.20s (± 0.83%)4.21s (± 0.49%)~4.19s4.24sp=0.622 n=6
Bind Time1.60s (± 1.62%)1.61s (± 1.57%)~1.56s1.63sp=0.256 n=6
Check Time22.36s (± 0.40%)22.37s (± 0.45%)~22.23s22.48sp=1.000 n=6
Emit Time1.68s (± 0.97%)1.68s (± 1.53%)~1.64s1.71sp=0.806 n=6
Total Time29.84s (± 0.34%)29.87s (± 0.47%)~29.67s30.03sp=0.689 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors3434~~~p=1.000 n=6
Symbols224,824224,824~~~p=1.000 n=6
Types93,39093,390~~~p=1.000 n=6
Memory used368,977k (± 0.01%)368,983k (± 0.00%)~368,956k369,005kp=1.000 n=6
Parse Time3.64s (± 0.56%)3.65s (± 0.77%)~3.64s3.71sp=0.246 n=6
Bind Time1.93s (± 1.76%)1.90s (± 0.79%)~1.89s1.93sp=0.170 n=6
Check Time19.42s (± 0.50%)19.41s (± 0.34%)~19.34s19.50sp=0.809 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time24.98s (± 0.44%)24.97s (± 0.29%)~24.87s25.05sp=0.936 n=6
vscode - node (v18.15.0, x64)
Errors44~~~p=1.000 n=6
Symbols2,796,2892,796,289~~~p=1.000 n=6
Types949,823949,823~~~p=1.000 n=6
Memory used2,924,219k (± 0.00%)2,924,357k (± 0.00%)+137k (+ 0.00%)2,924,319k2,924,386kp=0.020 n=6
Parse Time16.53s (± 0.45%)16.54s (± 0.22%)~16.48s16.59sp=0.630 n=6
Bind Time4.92s (± 0.62%)4.92s (± 0.46%)~4.90s4.95sp=1.000 n=6
Check Time88.09s (± 0.39%)87.84s (± 0.35%)~87.47s88.13sp=0.470 n=6
Emit Time26.06s (± 9.63%)23.70s (± 0.46%)~23.56s23.84sp=0.065 n=6
Total Time135.60s (± 1.91%)133.00s (± 0.26%)-2.60s (- 1.92%)132.63s133.45sp=0.045 n=6
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols265,779265,779~~~p=1.000 n=6
Types108,400108,400~~~p=1.000 n=6
Memory used409,822k (± 0.02%)409,888k (± 0.03%)~409,765k410,044kp=0.199 n=6
Parse Time4.86s (± 0.60%)4.86s (± 0.58%)~4.83s4.89sp=0.685 n=6
Bind Time2.08s (± 1.37%)2.06s (± 0.59%)~2.05s2.08sp=0.287 n=6
Check Time21.03s (± 0.48%)21.03s (± 0.28%)~20.95s21.09sp=1.000 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time27.98s (± 0.33%)27.95s (± 0.26%)~27.86s28.05sp=0.748 n=6
xstate-main - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols523,904523,904~~~p=1.000 n=6
Types178,697178,697~~~p=1.000 n=6
Memory used460,732k (± 0.01%)460,802k (± 0.01%)+70k (+ 0.02%)460,767k460,838kp=0.006 n=6
Parse Time2.68s (± 0.30%)2.70s (± 0.59%)~2.68s2.72sp=0.118 n=6
Bind Time0.98s (± 0.77%)0.98s (± 0.56%)~0.98s0.99sp=0.476 n=6
Check Time15.40s (± 0.23%)15.42s (± 0.35%)~15.36s15.49sp=0.936 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time19.07s (± 0.24%)19.10s (± 0.28%)~19.05s19.17sp=0.228 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@weswigham

Wesley Wigham (weswigham) commented Apr 26, 2024 via email •
edited
Loading

Copy link
Copy Markdown
Member

@jakebailey

Jake Bailey (jakebailey) commented Apr 26, 2024

Copy link
Copy Markdown
Member

That result is right on the cusp of significance, and its before value is +/- 2%, so it's possible that it's not a real result, unfortunately. We're still p-hacking ourselves...

@gabritto

Gabriela Araujo Britto (gabritto) commented Apr 26, 2024

Copy link
Copy Markdown
MemberAuthor

That result is right on the cusp of significance, and its before value is +/- 2%, so it's possible that it's not a real result, unfortunately. We're still p-hacking ourselves...

Yep, I'd suspect it's just noise. I don't expect anything to get faster, and I think nothing changed on extended tests, so doesn't seem like vscode has duplicate diagnostics. On the upside, nothing got significantly worse either :)

@weswigham

Wesley Wigham (weswigham) commented Apr 26, 2024

Copy link
Copy Markdown
Member

We're still p-hacking ourselves...

Should we un-fix n and instead fix p?

@jakebailey

Jake Bailey (jakebailey) commented Apr 26, 2024

Copy link
Copy Markdown
Member

Running the vscode benchmark already takes half an hour, I don't see us being able to change n that much.

Base automatically changed from gabritto/issue58207 to mainApril 26, 2024 22:44
@gabritto
Gabriela Araujo Britto (gabritto) deleted the gabritto/issue58207pt2 branch April 29, 2024 17:24
@AndaristMateusz Burzyński (Andarist) mentioned this pull request Jul 10, 2024
Merged
@RyanCavanaughRyan Cavanaugh (RyanCavanaugh) mentioned this pull request Jul 10, 2025
Closed
@microsoftMicrosoft (microsoft) locked as resolved and limited conversation to collaborators Oct 16, 2025
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@andrewbranchAndrew Branch (andrewbranch)andrewbranch approved these changes
@weswighamWesley Wigham (weswigham)weswigham approved these changes
@sheetalkamatSheetal Nandi (sheetalkamat)sheetalkamat approved these changes

Assignees

@gabrittoGabriela Araujo Britto (gabritto)

Labels

Author: TeamFor Uncommitted BugPR for untriaged, rejected, closed or missing bug

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Duplicate diagnostics

6 participants

@gabritto@typescript-bot@weswigham@jakebailey@andrewbranch@sheetalkamat