Skip to content

[WIP] Experimental Slim AstNode API - #59992

Closed
Ron Buckton (rbuckton) wants to merge 12 commits into
mainfrom
slim-ast-node
Closed

[WIP] Experimental Slim AstNode API #59992
Ron Buckton (rbuckton) wants to merge 12 commits into
mainfrom
slim-ast-node

Conversation

@rbuckton

@rbucktonRon Buckton (rbuckton) commented Sep 17, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

This adds an experimental "slim" API for AST nodes that is intended to be highly monomorphic while maintaining the existing public API. This will also allow for gradual adoption of the new slim API internally.

This is a very early proof-of-concept designed to test the ramifications of this change and whether they will be beneficial to compiler performance long-term. The general principle of the change is that compiler internals will no longer depend on the highly polymorphic Node type, and instead depend on a slimmer AstNode class with a fixed layout that is roughly the following:

classAstNode<NextendsNode>{// lazy-initialized extra fields for a node (id, original, emitNode, transformFlags, // modifierFlagsCache, etc.)private_extra: AstNodeExtraFields|undefined;// Constructor used to produce a `Node` facade for the public APIprivate_nodeConstructor: (new(ast: AstNode<N>)=>N)|undefined;// Lazy-initialized `Node` facade for the public APIprivate_node: N|undefined;kind: N["kind"];data: N["data"];parent: AstNode|undefined;flags: NodeFlags;pos: number;end: number;constructor(kind: N["kind"],data: N["data"],nodeConstructor: new(ast: AstNode<N>)=>N){
...
}// lazy-initializes the `Node` facade for this AstNodegetnode(){ ... }// lazy-initialized properties of an AstNodegetid(){ ... }setid(value){ ... }getoriginal(){ ... }setoriginal(value){ ... }getemitNode(){ ... }setemitNode(value){ ... }gettransformFlags(){ ... }settransformFlags(value){ ... }}

This "slim" node is designed to be small and allow for monomorphic access to the most common shared properties of our AST nodes. In place of the existing Node type, the public API would surface a facade based on something like this definition of Node:

classNode<KextendsSyntaxKind,TextendsAstData>{/** @internal */readonlyast: AstNode<Node<K,T>>;constructor(ast: AstNode<Node<K,T>>){this.ast=ast;}/** @internal */getdata(): T{returnthis.ast.data;}getkind(): K{returnthis.ast.kind;}getpos(){returnthis.ast.pos;}getend(){returnthis.ast.end;}getflags(){returnthis.ast.flags;}getparent(){returnthis.ast.parent;}getid(){returnthis.ast.id;}// etc.}

Node-specific fields will live in objects held by the data property, such as this class that describes a BinaryExpression:

classAstBinaryExpressionDataextendsAstData{left!: AstNode<Expression>;operatorToken!: AstNode<BinaryOperatorToken>;right!: AstNode<Expression>;// etc.}

While the BinaryExpression we provide today would be replaced by a facade:

classBinaryExpressionextendsNode<SyntaxKind.BinaryExpression,AstBinaryExpressionData>{declare_expressionBrand: any;declare_declarationBrand: any;declare_jsdocContainerBrand: any;/** @internal */declarereadonlyast: AstBinaryExpression;getleft(){returnthis.ast.data.left?.node;}setleft(value){this.ast.data.left=value?.ast;}getoperatorToken(){returnthis.ast.data.operatorToken?.node;}setoperatorToken(value){this.ast.data.operatorToken=value?.ast;}getright(){returnthis.ast.data.right?.node;}setright(value){this.ast.data.right=value?.ast;}// etc.}typeAstBinaryExpression=AstNode<BinaryExpression>;

Work on this PR is expected to progress in stages. Until work has progressed far enough along, we expect performance numbers to be much worse before they get better due to the overhead from the compiler internals relying on the facade vs the slim node.

There will hopefully be some additional benefits to this change if it shows promise:

  • Removes use of objectAllocator for node constructors
  • Moves all language service Node extensions to the compiler (e.g., no more NodeObject)
  • Common structure of AstNode could allow for more efficient encodings of pos/end/flags/etc.
  • Potentially easy to swap out internally with shared structs in the future.

TODO

  • Initial "slim" AstNode implementation
  • Minimal AstNode version of NodeFactory and ParenthesizerRules
  • Redirect NodeFactory and ParenthesizerRules to leverage AstNode-versions
  • Migrate parser.ts to use AstNode
  • Migrate binder.ts to use AstNode
  • Migrate checker.ts to use AstNode
  • Migrate transformers to use AstNode
  • Migrate emitter.ts to use AstNode
  • Migrate language service to use AstNode

Related #59190
Related #58928
Related #51682
Related #51913

@typescript-botTypeScript Bot (typescript-bot) added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Sep 17, 2024
@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 17, 2024

Copy link
Copy Markdown
Contributor

Looks like you're introducing a change to the public API surface area. If this includes breaking changes, please document them on our wiki's API Breaking Changes page.

Also, please make sure Daniel Rosenwasser (@DanielRosenwasser) and Ryan Cavanaugh (@RyanCavanaugh) are aware of the changes, just as a heads up.

@rbuckton

Ron Buckton (rbuckton) commented Sep 17, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test
TypeScript Bot (@typescript-bot) test top400
TypeScript Bot (@typescript-bot) test tsserver top100
TypeScript Bot (@typescript-bot) user test this
TypeScript Bot (@typescript-bot) user test tsserver

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 17, 2024

Copy link
Copy Markdown
Contributor

Hey Ron Buckton (@rbuckton), this PR is in an unmergable state, so is missing a merge commit to run against; please resolve conflicts and try again.

@rbuckton

Ron Buckton (rbuckton) commented Sep 17, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test
TypeScript Bot (@typescript-bot) test top400
TypeScript Bot (@typescript-bot) test tsserver top100
TypeScript Bot (@typescript-bot) user test this
TypeScript Bot (@typescript-bot) user test tsserver

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 17, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test✅ Started👀 Results
test top400✅ Started✅ Results
test tsserver top100✅ Started👀 Results
user test this✅ Started✅ Results
user test tsserver✅ Started✅ Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 17, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton) Here are the results of running the user tests with tsserver comparing main and refs/pull/59992/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Unknown failure"

Otherwise...

Everything looks good!

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 17, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton) Here are the results of running the user tests with tsc comparing main and refs/pull/59992/merge:

Everything looks good!

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 17, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3030~~~p=1.000 n=6
Symbols62,15362,153~~~p=1.000 n=6
Types50,24250,242~~~p=1.000 n=6
Memory used193,087k (± 0.76%)263,539k (± 0.02%)🔻+70,452k (+36.49%)263,476k263,599kp=0.005 n=6
Parse Time1.31s (± 0.93%)1.77s🔻+0.46s (+35.46%)~~p=0.003 n=6
Bind Time0.71s1.13s (± 0.36%)🔻+0.42s (+58.92%)1.12s1.13sp=0.002 n=6
Check Time9.54s (± 0.38%)11.72s (± 0.47%)🔻+2.18s (+22.83%)11.63s11.78sp=0.005 n=6
Emit Time2.73s (± 0.95%)4.05s (± 0.31%)🔻+1.32s (+48.53%)4.03s4.06sp=0.005 n=6
Total Time14.28s (± 0.15%)18.67s (± 0.26%)🔻+4.38s (+30.68%)18.59s18.72sp=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors77~~~p=1.000 n=6
Symbols945,753945,753~~~p=1.000 n=6
Types410,067410,067~~~p=1.000 n=6
Memory used1,222,737k (± 0.00%)1,670,641k (± 0.01%)🔻+447,904k (+36.63%)1,670,496k1,670,735kp=0.005 n=6
Parse Time7.93s (± 0.52%)11.97s (± 0.58%)🔻+4.04s (+50.95%)11.88s12.03sp=0.005 n=6
Bind Time2.22s (± 0.54%)4.09s (± 0.45%)🔻+1.87s (+84.03%)4.06s4.11sp=0.005 n=6
Check Time36.45s (± 0.29%)50.23s (± 0.33%)🔻+13.78s (+37.81%)50.02s50.43sp=0.005 n=6
Emit Time17.90s (± 0.31%)26.21s (± 0.38%)🔻+8.32s (+46.47%)26.08s26.32sp=0.005 n=6
Total Time64.49s (± 0.18%)92.51s (± 0.18%)🔻+28.01s (+43.43%)92.28s92.66sp=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols2,520,5512,520,551~~~p=1.000 n=6
Types935,995935,995~~~p=1.000 n=6
Memory used2,350,355k (± 0.00%)2,729,076k (± 0.00%)🔻+378,721k (+16.11%)2,728,900k2,729,237kp=0.005 n=6
Parse Time11.02s (± 0.51%)14.85s (± 0.29%)🔻+3.83s (+34.79%)14.80s14.92sp=0.005 n=6
Bind Time2.56s (± 0.80%)4.47s (± 0.33%)🔻+1.91s (+74.66%)4.45s4.49sp=0.005 n=6
Check Time86.26s (± 0.34%)105.84s (± 2.23%)🔻+19.58s (+22.70%)104.54s110.63sp=0.005 n=6
Emit Time0.33s (± 2.71%)0.34s+0.01s (+ 3.03%)~~p=0.028 n=6
Total Time100.17s (± 0.35%)125.51s (± 1.90%)🔻+25.34s (+25.30%)124.24s130.36sp=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,245,9521,428,855🔻+182,903 (+14.68%)~~p=0.001 n=6
Types264,213277,478🔻+13,265 (+ 5.02%)~~p=0.001 n=6
Memory used2,397,953k (± 0.05%)3,390,884k (± 0.02%)🔻+992,932k (+41.41%)3,389,986k3,391,681kp=0.005 n=6
Parse Time6.07s (± 0.44%)10.22s (± 0.47%)🔻+4.15s (+68.36%)10.18s10.29sp=0.005 n=6
Bind Time2.26s (± 1.00%)4.34s (± 0.58%)🔻+2.08s (+92.32%)4.30s4.37sp=0.005 n=6
Check Time41.37s (± 0.73%)57.08s (± 0.98%)🔻+15.71s (+37.97%)56.36s57.66sp=0.005 n=6
Emit Time3.61s (± 3.38%)5.02s (± 0.87%)🔻+1.41s (+39.17%)4.95s5.08sp=0.005 n=6
Total Time53.31s (± 0.35%)76.68s (± 0.67%)🔻+23.37s (+43.83%)75.96s77.25sp=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,245,9521,428,855🔻+182,903 (+14.68%)~~p=0.001 n=6
Types264,213277,478🔻+13,265 (+ 5.02%)~~p=0.001 n=6
Memory used2,472,518k (± 0.01%)3,514,815k (± 2.44%)🔻+1,042,297k (+42.16%)3,479,298k3,690,236kp=0.005 n=6
Parse Time7.89s (± 0.59%)13.01s (± 0.55%)🔻+5.12s (+64.95%)12.92s13.11sp=0.005 n=6
Bind Time2.53s (± 0.72%)5.31s (± 0.71%)🔻+2.78s (+110.09%)5.26s5.37sp=0.005 n=6
Check Time51.21s (± 0.93%)71.22s (± 0.19%)🔻+20.02s (+39.09%)71.06s71.41sp=0.005 n=6
Emit Time4.52s (± 3.40%)6.18s (± 0.32%)🔻+1.67s (+36.85%)6.15s6.20sp=0.005 n=6
Total Time66.14s (± 0.55%)95.76s (± 0.16%)🔻+29.61s (+44.77%)95.50s95.91sp=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols259,885331,296🔻+71,411 (+27.48%)~~p=0.001 n=6
Types106,140119,398🔻+13,258 (+12.49%)~~p=0.001 n=6
Memory used435,581k (± 0.01%)680,639k (± 0.01%)🔻+245,058k (+56.26%)680,542k680,759kp=0.005 n=6
Parse Time4.24s (± 0.53%)6.45s (± 0.51%)🔻+2.21s (+52.08%)6.42s6.49sp=0.005 n=6
Bind Time1.61s (± 0.47%)2.93s (± 0.84%)🔻+1.32s (+82.18%)2.91s2.97sp=0.005 n=6
Check Time22.42s (± 0.19%)33.36s (± 0.28%)🔻+10.94s (+48.79%)33.24s33.48sp=0.005 n=6
Emit Time1.88s (± 0.78%)3.34s (± 0.69%)🔻+1.46s (+77.73%)3.31s3.37sp=0.005 n=6
Total Time30.14s (± 0.17%)46.08s (± 0.26%)🔻+15.93s (+52.85%)45.94s46.29sp=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors6868~~~p=1.000 n=6
Symbols225,018225,018~~~p=1.000 n=6
Types94,24994,249~~~p=1.000 n=6
Memory used370,239k (± 0.02%)495,006k (± 0.01%)🔻+124,767k (+33.70%)494,975k495,074kp=0.005 n=6
Parse Time2.75s (± 0.99%)3.90s (± 0.56%)🔻+1.15s (+41.90%)3.86s3.92sp=0.005 n=6
Bind Time1.55s (± 1.03%)2.37s (± 0.64%)🔻+0.82s (+52.74%)2.34s2.38sp=0.005 n=6
Check Time15.73s (± 0.18%)21.10s (± 0.24%)🔻+5.37s (+34.12%)21.04s21.15sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time20.03s (± 0.23%)27.36s (± 0.15%)🔻+7.33s (+36.61%)27.30s27.40sp=0.005 n=6
vscode - node (v18.15.0, x64)
Errors10~~~p=1.000 n=6+0
Symbols3,071,6380~~~p=1.000 n=6+0
Types1,060,2660~~~p=1.000 n=6+0
Memory used3,175,799k (± 0.00%)0k~~~p=1.000 n=6+0
Parse Time17.11s (± 0.25%)0s~~~p=1.000 n=6+0
Bind Time5.30s (± 0.44%)0s~~~p=1.000 n=6+0
Check Time99.16s (± 0.59%)0s~~~p=1.000 n=6+0
Emit Time28.12s (± 7.42%)0s~~~p=1.000 n=6+0
Total Time149.69s (± 1.17%)0s~~~p=1.000 n=6+0
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols277,157277,157~~~p=1.000 n=6
Types112,949112,949~~~p=1.000 n=6
Memory used426,908k (± 0.01%)539,573k (± 0.01%)🔻+112,665k (+26.39%)539,532k539,626kp=0.005 n=6
Parse Time3.96s (± 0.43%)5.30s (± 0.40%)🔻+1.34s (+33.74%)5.26s5.32sp=0.005 n=6
Bind Time1.74s (± 1.12%)2.70s (± 0.80%)🔻+0.96s (+55.03%)2.67s2.72sp=0.005 n=6
Check Time17.67s (± 0.29%)22.83s (± 0.24%)🔻+5.16s (+29.22%)22.77s22.90sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time23.37s (± 0.32%)30.82s (± 0.24%)🔻+7.45s (+31.88%)30.76s30.93sp=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols531,582531,582~~~p=1.000 n=6
Types181,676181,676~~~p=1.000 n=6
Memory used463,664k (± 0.01%)548,851k (± 0.01%)🔻+85,187k (+18.37%)548,810k548,878kp=0.005 n=6
Parse Time2.60s (± 0.31%)3.59s (± 0.23%)🔻+0.98s (+37.77%)3.58s3.60sp=0.004 n=6
Bind Time0.93s1.53s (± 0.53%)🔻+0.60s (+64.16%)1.52s1.54sp=0.003 n=6
Check Time15.41s (± 0.24%)17.97s (± 0.21%)🔻+2.56s (+16.61%)17.93s18.02sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time18.95s (± 0.19%)23.09s (± 0.14%)🔻+4.14s (+21.84%)23.04s23.13sp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

tsserver

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,297ms (± 0.52%)3,187ms (± 0.35%)🔻+890ms (+38.74%)3,165ms3,196msp=0.005 n=6
Req 2 - geterr5,164ms (± 0.45%)6,581ms (± 0.67%)🔻+1,417ms (+27.44%)6,544ms6,660msp=0.005 n=6
Req 3 - references263ms (± 0.96%)361ms (± 0.46%)🔻+98ms (+37.26%)359ms363msp=0.005 n=6
Req 4 - navto227ms (± 0.36%)296ms (± 0.35%)🔻+69ms (+30.37%)294ms297msp=0.004 n=6
Req 5 - completionInfo count1,3571,357~~~p=1.000 n=6
Req 5 - completionInfo81ms (± 7.01%)85ms (± 7.92%)~81ms98msp=0.684 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,415ms (± 1.47%)3,359ms (± 0.47%)🔻+944ms (+39.08%)3,341ms3,388msp=0.005 n=6
Req 2 - geterr3,922ms (± 0.44%)5,046ms (± 1.41%)🔻+1,124ms (+28.67%)4,943ms5,122msp=0.005 n=6
Req 3 - references274ms (± 1.26%)397ms (± 2.17%)🔻+123ms (+44.88%)390ms413msp=0.005 n=6
Req 4 - navto232ms (± 3.02%)306ms (± 1.47%)🔻+74ms (+32.04%)297ms309msp=0.005 n=6
Req 5 - completionInfo count1,5191,519~~~p=1.000 n=6
Req 5 - completionInfo77ms (± 8.85%)84ms (± 1.00%)~83ms85msp=0.220 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen5,193ms (± 0.23%)7,215ms (± 3.09%)🔻+2,022ms (+38.95%)7,067ms7,516msp=0.005 n=6
Req 2 - geterr1,120ms (± 0.37%)1,263ms (± 0.96%)🔻+143ms (+12.79%)1,248ms1,276msp=0.005 n=6
Req 3 - references86ms (± 4.89%)108ms (± 3.74%)🔻+22ms (+25.93%)104ms112msp=0.004 n=6
Req 4 - navto441ms (± 0.24%)502ms (± 0.96%)🔻+62ms (+14.00%)496ms508msp=0.005 n=6
Req 5 - completionInfo count3,4443,444~~~p=1.000 n=6
Req 5 - completionInfo840ms (± 1.43%)895ms (± 1.17%)🔻+56ms (+ 6.63%)875ms903msp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

startup

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
tsc-startup - node (v18.15.0, x64)
Execution time188.86ms (± 0.19%)212.21ms (± 0.22%)🔻+23.35ms (+12.36%)208.70ms219.53msp=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time232.34ms (± 0.12%)248.96ms (± 0.14%)🔻+16.62ms (+ 7.15%)247.54ms254.80msp=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time227.71ms (± 0.14%)243.82ms (± 0.15%)🔻+16.11ms (+ 7.07%)242.51ms248.92msp=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time226.97ms (± 0.14%)243.14ms (± 0.15%)🔻+16.18ms (+ 7.13%)241.61ms250.40msp=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@rbuckton

Ron Buckton (rbuckton) commented Sep 17, 2024

Copy link
Copy Markdown
ContributorAuthor

As expected, the facade is significantly slower, though it is quite a bit slower than I was hoping. Hopefully we'll start seeing better numbers once I finish migrating the parser.

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 17, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton) Here are the results of running the top 200 repos with tsserver comparing main and refs/pull/59992/merge:

Something interesting changed - please have a look.

Details

Server exited prematurely with code unknown and signal SIGABRT 

Server exited prematurely with code unknown and signal SIGABRT

Affected repos

backstage/backstage Raw error text: RepoResults8/backstage.backstage.rawError.txt in the artifact folder
Replay commands: RepoResults8/backstage.backstage.replay.txt in the artifact folder

Last few requests

{"seq":134,"type":"request","command":"completionInfo","arguments":{"file":"@PROJECT_ROOT@/scripts/templates/knex-migration.stub.js","line":29,"offset":4,"includeExternalModuleExports":false,"triggerKind":1}}
{"seq":135,"type":"request","command":"completionEntryDetails","arguments":{"file":"@PROJECT_ROOT@/scripts/templates/knex-migration.stub.js","line":29,"offset":4,"entryNames":["@abstract"]}}
{"seq":136,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":["@PROJECT_ROOT@/scripts/list-backend-feature.js"],"openFiles":[]}}
{"seq":137,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/packages/eslint-plugin/index.js","projectRootPath":"@PROJECT_ROOT@"}]}}

Repro steps

#!/bin/bash
git clone https://github.com/backstage/backstage --recurse-submodules
git -C "./backstage" reset --hard 6f8d71b74ca53b6a62d0ff6325f7849d3b48a822
# Install packages (exact steps are below, but it might be easier to follow the repo readme)
yarn --cwd "./backstage" install --no-immutable --mode=skip-build
yarn --cwd "./backstage/storybook" install --no-immutable --mode=skip-build
yarn --cwd "./backstage/microsite" install --no-immutable --mode=skip-build
downloadUrl=$(curl -s "https://typescript.visualstudio.com/TypeScript/_apis/build/builds/163602/artifacts?artifactName=RepoResults8&api-version=7.0"| jq -r ".resource.downloadUrl")
wget -O RepoResults8.zip "$downloadUrl"
unzip -p RepoResults8.zip RepoResults8/backstage.backstage.replay.txt > backstage.backstage.replay.txt
npm install --no-save @typescript/server-replay

To run the repro:

# `npx tsreplay --help` to learn about helpful switches for debugging, logging, etc.
npx tsreplay ./backstage ./backstage.backstage.replay.txt <PATH_TO_tsserver.js>

Server exited prematurely with code unknown and signal SIGABRT 

Server exited prematurely with code unknown and signal SIGABRT

Affected repos

n8n-io/n8n Raw error text: RepoResults3/n8n-io.n8n.rawError.txt in the artifact folder
Replay commands: RepoResults3/n8n-io.n8n.replay.txt in the artifact folder

Last few requests

{"seq":995,"type":"request","command":"completionInfo","arguments":{"file":"@PROJECT_ROOT@/packages/editor-ui/src/Interface.ts","line":1583,"offset":18,"includeExternalModuleExports":false,"triggerKind":2,"triggerCharacter":"'"}}
{"seq":996,"type":"request","command":"definitionAndBoundSpan","arguments":{"file":"@PROJECT_ROOT@/packages/editor-ui/src/Interface.ts","line":1589,"offset":4}}
{"seq":997,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":["@PROJECT_ROOT@/packages/nodes-base/credentials/CarbonBlackApi.credentials.ts"],"openFiles":[]}}
{"seq":998,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/packages/design-system/src/plugin.ts","projectRootPath":"@PROJECT_ROOT@"}]}}

Repro steps

#!/bin/bash
git clone https://github.com/n8n-io/n8n --recurse-submodules
git -C "./n8n" reset --hard 989f69d1f4f63d3f0c35341d310bc78914137677
pnpm --dir "./n8n" install --no-frozen-lockfile --prefer-offline --ignore-scripts --reporter=silent
downloadUrl=$(curl -s "https://typescript.visualstudio.com/TypeScript/_apis/build/builds/163602/artifacts?artifactName=RepoResults3&api-version=7.0"| jq -r ".resource.downloadUrl")
wget -O RepoResults3.zip "$downloadUrl"
unzip -p RepoResults3.zip RepoResults3/n8n-io.n8n.replay.txt > n8n-io.n8n.replay.txt
npm install --no-save @typescript/server-replay

To run the repro:

# `npx tsreplay --help` to learn about helpful switches for debugging, logging, etc.
npx tsreplay ./n8n ./n8n-io.n8n.replay.txt <PATH_TO_tsserver.js>

Server exited prematurely with code unknown and signal SIGABRT 

Server exited prematurely with code unknown and signal SIGABRT

Affected repos

remotion-dev/remotion Raw error text: RepoResults14/remotion-dev.remotion.rawError.txt in the artifact folder
Replay commands: RepoResults14/remotion-dev.remotion.replay.txt in the artifact folder

Last few requests

{"seq":256,"type":"request","command":"navbar","arguments":{"file":"@PROJECT_ROOT@/packages/cli/remotionb-cli.js"}}
{"seq":257,"type":"request","command":"updateOpen","arguments":{"changedFiles":[{"fileName":"@PROJECT_ROOT@/packages/cli/remotionb-cli.js","textChanges":[{"newText":" //comment","start":{"line":1,"offset":19},"end":{"line":1,"offset":19}}]}],"closedFiles":[],"openFiles":[]}}
{"seq":258,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":["@PROJECT_ROOT@/packages/compositor-linux-x64-gnu/index.js"],"openFiles":[]}}
{"seq":259,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/packages/astro-example/test.js","projectRootPath":"@PROJECT_ROOT@"}]}}

Repro steps

#!/bin/bash
git clone https://github.com/remotion-dev/remotion --recurse-submodules
git -C "./remotion" reset --hard 6acfd716dc5e3bad7efd4414a7392455694ed00b
pnpm --dir "./remotion" install --no-frozen-lockfile --prefer-offline --ignore-scripts --reporter=silent
downloadUrl=$(curl -s "https://typescript.visualstudio.com/TypeScript/_apis/build/builds/163602/artifacts?artifactName=RepoResults14&api-version=7.0"| jq -r ".resource.downloadUrl")
wget -O RepoResults14.zip "$downloadUrl"
unzip -p RepoResults14.zip RepoResults14/remotion-dev.remotion.replay.txt > remotion-dev.remotion.replay.txt
npm install --no-save @typescript/server-replay

To run the repro:

# `npx tsreplay --help` to learn about helpful switches for debugging, logging, etc.
npx tsreplay ./remotion ./remotion-dev.remotion.replay.txt <PATH_TO_tsserver.js>
elastic/kibana Raw error text: RepoResults14/elastic.kibana.rawError.txt in the artifact folder
Replay commands: RepoResults14/elastic.kibana.replay.txt in the artifact folder

Last few requests

{"seq":96,"type":"request","command":"definitionAndBoundSpan","arguments":{"file":"@PROJECT_ROOT@/packages/kbn-cli-dev-mode/jest.integration.config.js","line":11,"offset":12}}
{"seq":97,"type":"request","command":"references","arguments":{"file":"@PROJECT_ROOT@/packages/kbn-cli-dev-mode/jest.integration.config.js","line":11,"offset":12}}
{"seq":98,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":["@PROJECT_ROOT@/packages/kbn-telemetry-tools/jest.config.js"],"openFiles":[]}}
{"seq":99,"type":"request","command":"updateOpen","arguments":{"changedFiles":[],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/x-pack/test/upgrade_assistant_integration/config.js","projectRootPath":"@PROJECT_ROOT@"}]}}

Repro steps

#!/bin/bash
git clone https://github.com/elastic/kibana --recurse-submodules
git -C "./kibana" reset --hard 48f6f945bef098cd5ef667b7e6a01dd2182b6698
downloadUrl=$(curl -s "https://typescript.visualstudio.com/TypeScript/_apis/build/builds/163602/artifacts?artifactName=RepoResults14&api-version=7.0"| jq -r ".resource.downloadUrl")
wget -O RepoResults14.zip "$downloadUrl"
unzip -p RepoResults14.zip RepoResults14/elastic.kibana.replay.txt > elastic.kibana.replay.txt
npm install --no-save @typescript/server-replay

To run the repro:

# `npx tsreplay --help` to learn about helpful switches for debugging, logging, etc.
npx tsreplay ./kibana ./elastic.kibana.replay.txt <PATH_TO_tsserver.js>

Server exited prematurely with code unknown and signal SIGABRT 

Server exited prematurely with code unknown and signal SIGABRT

Affected repos

withfig/autocomplete Raw error text: RepoResults10/withfig.autocomplete.rawError.txt in the artifact folder
Replay commands: RepoResults10/withfig.autocomplete.replay.txt in the artifact folder

Last few requests

{"seq":274,"type":"request","command":"completionEntryDetails","arguments":{"file":"@PROJECT_ROOT@/src/ykman.ts","line":1137,"offset":19,"entryNames":["configModeGenerator"]}}
{"seq":275,"type":"request","command":"definitionAndBoundSpan","arguments":{"file":"@PROJECT_ROOT@/src/ykman.ts","line":1174,"offset":23}}
{"seq":276,"type":"request","command":"completionInfo","arguments":{"file":"@PROJECT_ROOT@/src/ykman.ts","line":1183,"offset":29,"includeExternalModuleExports":false,"triggerKind":2,"triggerCharacter":"\""}}
{"seq":277,"type":"request","command":"references","arguments":{"file":"@PROJECT_ROOT@/src/ykman.ts","line":1252,"offset":15}}

Repro steps

#!/bin/bash
git clone https://github.com/withfig/autocomplete --recurse-submodules
git -C "./autocomplete" reset --hard 35977c9b68cecd1755b6c9c09d85f8d567ddedb9
pnpm --dir "./autocomplete" install --no-frozen-lockfile --prefer-offline --ignore-scripts --reporter=silent
downloadUrl=$(curl -s "https://typescript.visualstudio.com/TypeScript/_apis/build/builds/163602/artifacts?artifactName=RepoResults10&api-version=7.0"| jq -r ".resource.downloadUrl")
wget -O RepoResults10.zip "$downloadUrl"
unzip -p RepoResults10.zip RepoResults10/withfig.autocomplete.replay.txt > withfig.autocomplete.replay.txt
npm install --no-save @typescript/server-replay

To run the repro:

# `npx tsreplay --help` to learn about helpful switches for debugging, logging, etc.
npx tsreplay ./autocomplete ./withfig.autocomplete.replay.txt <PATH_TO_tsserver.js>

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 17, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton) Here are the results of running the top 400 repos with tsc comparing main and refs/pull/59992/merge:

Everything looks good!

@rbuckton

Ron Buckton (rbuckton) commented Sep 18, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 18, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test✅ Started👀 Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 18, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3030~~~p=1.000 n=6
Symbols62,15362,153~~~p=1.000 n=6
Types50,24250,242~~~p=1.000 n=6
Memory used194,850k (± 0.90%)264,879k (± 0.72%)🔻+70,030k (+35.94%)263,605k267,413kp=0.005 n=6
Parse Time1.57s (± 1.12%)2.11s (± 1.35%)🔻+0.54s (+34.72%)2.06s2.14sp=0.004 n=6
Bind Time0.85s (± 1.61%)1.32s (± 0.48%)🔻+0.47s (+54.39%)1.31s1.33sp=0.004 n=6
Check Time11.38s (± 0.20%)13.89s (± 0.53%)🔻+2.51s (+22.06%)13.76s13.96sp=0.005 n=6
Emit Time3.24s (± 0.45%)4.84s (± 0.95%)🔻+1.60s (+49.36%)4.76s4.88sp=0.005 n=6
Total Time17.04s (± 0.29%)22.16s (± 0.58%)🔻+5.12s (+30.02%)21.93s22.28sp=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors77~~~p=1.000 n=6
Symbols945,753945,753~~~p=1.000 n=6
Types410,067410,067~~~p=1.000 n=6
Memory used1,222,749k (± 0.01%)1,670,645k (± 0.00%)🔻+447,896k (+36.63%)1,670,595k1,670,728kp=0.005 n=6
Parse Time7.89s (± 0.74%)11.92s (± 0.59%)🔻+4.04s (+51.24%)11.85s12.04sp=0.005 n=6
Bind Time2.21s (± 0.40%)4.04s (± 0.29%)🔻+1.83s (+82.73%)4.02s4.05sp=0.005 n=6
Check Time36.22s (± 0.39%)49.78s (± 0.38%)🔻+13.55s (+37.42%)49.51s50.08sp=0.005 n=6
Emit Time17.88s (± 0.51%)26.06s (± 0.49%)🔻+8.18s (+45.75%)25.87s26.18sp=0.005 n=6
Total Time64.19s (± 0.22%)91.80s (± 0.34%)🔻+27.61s (+43.02%)91.33s92.25sp=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols2,529,1562,529,156~~~p=1.000 n=6
Types936,017936,017~~~p=1.000 n=6
Memory used2,364,302k (± 0.00%)2,745,907k (± 0.01%)🔻+381,605k (+16.14%)2,745,531k2,746,195kp=0.005 n=6
Parse Time11.16s (± 0.29%)15.08s (± 1.60%)🔻+3.92s (+35.14%)14.87s15.56sp=0.005 n=6
Bind Time2.61s (± 0.40%)4.55s (± 0.32%)🔻+1.94s (+74.49%)4.52s4.56sp=0.004 n=6
Check Time86.33s (± 0.70%)106.45s (± 2.04%)🔻+20.12s (+23.30%)104.84s110.25sp=0.005 n=6
Emit Time0.33s (± 1.24%)0.34s (± 3.42%)🔻+0.01s (+ 4.06%)0.33s0.36sp=0.025 n=6
Total Time100.43s (± 0.59%)126.42s (± 1.65%)🔻+25.99s (+25.88%)124.79s130.15sp=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,245,9521,428,767🔻+182,815 (+14.67%)~~p=0.001 n=6
Types264,213277,448🔻+13,235 (+ 5.01%)~~p=0.001 n=6
Memory used2,397,820k (± 0.03%)3,388,854k (± 0.01%)🔻+991,034k (+41.33%)3,388,250k3,389,346kp=0.005 n=6
Parse Time6.11s (± 0.59%)10.28s (± 0.42%)🔻+4.16s (+68.06%)10.22s10.33sp=0.005 n=6
Bind Time2.26s (± 0.78%)4.30s (± 1.04%)🔻+2.04s (+90.07%)4.24s4.35sp=0.005 n=6
Check Time41.39s (± 0.52%)57.04s (± 0.72%)🔻+15.66s (+37.83%)56.58s57.47sp=0.005 n=6
Emit Time3.56s (± 2.29%)5.04s (± 0.63%)🔻+1.48s (+41.53%)4.99s5.08sp=0.005 n=6
Total Time53.34s (± 0.54%)76.67s (± 0.53%)🔻+23.33s (+43.75%)76.22s77.09sp=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,245,9521,428,767🔻+182,815 (+14.67%)~~p=0.001 n=6
Types264,213277,448🔻+13,235 (+ 5.01%)~~p=0.001 n=6
Memory used2,472,614k (± 0.02%)3,477,928k (± 0.01%)🔻+1,005,314k (+40.66%)3,477,333k3,478,364kp=0.005 n=6
Parse Time7.89s (± 0.44%)13.00s (± 0.39%)🔻+5.11s (+64.79%)12.92s13.08sp=0.005 n=6
Bind Time2.52s (± 0.65%)5.28s (± 0.56%)🔻+2.76s (+109.25%)5.25s5.32sp=0.005 n=6
Check Time51.26s (± 0.83%)70.59s (± 0.72%)🔻+19.33s (+37.70%)69.84s71.21sp=0.005 n=6
Emit Time4.35s (± 1.74%)6.28s (± 3.26%)🔻+1.93s (+44.33%)6.11s6.68sp=0.005 n=6
Total Time66.02s (± 0.58%)95.15s (± 0.40%)🔻+29.13s (+44.11%)94.55s95.59sp=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols259,885331,208🔻+71,323 (+27.44%)~~p=0.001 n=6
Types106,140119,368🔻+13,228 (+12.46%)~~p=0.001 n=6
Memory used435,589k (± 0.02%)680,131k (± 0.01%)🔻+244,543k (+56.14%)680,073k680,256kp=0.005 n=6
Parse Time3.41s (± 0.49%)5.20s (± 1.10%)🔻+1.79s (+52.59%)5.16s5.31sp=0.005 n=6
Bind Time1.29s (± 1.26%)2.33s (± 1.21%)🔻+1.04s (+80.15%)2.29s2.37sp=0.004 n=6
Check Time18.16s (± 0.39%)26.83s (± 0.39%)🔻+8.68s (+47.80%)26.74s26.96sp=0.005 n=6
Emit Time1.53s (± 2.19%)2.72s (± 0.86%)🔻+1.19s (+78.00%)2.69s2.75sp=0.005 n=6
Total Time24.39s (± 0.37%)37.09s (± 0.18%)🔻+12.70s (+52.09%)36.99s37.18sp=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors6868~~~p=1.000 n=6
Symbols225,018225,018~~~p=1.000 n=6
Types94,24994,249~~~p=1.000 n=6
Memory used370,253k (± 0.02%)495,011k (± 0.01%)🔻+124,758k (+33.70%)494,946k495,052kp=0.005 n=6
Parse Time3.44s (± 1.21%)4.88s (± 0.90%)🔻+1.44s (+41.69%)4.82s4.95sp=0.005 n=6
Bind Time1.95s (± 1.07%)2.94s (± 0.91%)🔻+0.99s (+50.51%)2.90s2.97sp=0.005 n=6
Check Time19.42s (± 0.50%)26.01s (± 0.11%)🔻+6.58s (+33.90%)25.96s26.04sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time24.81s (± 0.32%)33.82s (± 0.08%)🔻+9.01s (+36.31%)33.79s33.87sp=0.005 n=6
vscode - node (v18.15.0, x64)
Errors10~~~p=1.000 n=6+0
Symbols3,074,1410~~~p=1.000 n=6+0
Types1,060,9330~~~p=1.000 n=6+0
Memory used3,177,732k (± 0.00%)0k~~~p=1.000 n=6+0
Parse Time17.15s (± 0.25%)0s~~~p=1.000 n=6+0
Bind Time5.33s (± 0.83%)0s~~~p=1.000 n=6+0
Check Time99.32s (± 0.26%)0s~~~p=1.000 n=6+0
Emit Time27.18s (± 0.28%)0s~~~p=1.000 n=6+0
Total Time148.98s (± 0.19%)0s~~~p=1.000 n=6+0
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols277,157277,157~~~p=1.000 n=6
Types112,949112,949~~~p=1.000 n=6
Memory used426,931k (± 0.02%)539,845k (± 0.02%)🔻+112,914k (+26.45%)539,746k539,977kp=0.005 n=6
Parse Time3.95s (± 0.69%)5.29s (± 0.25%)🔻+1.34s (+33.91%)5.28s5.31sp=0.004 n=6
Bind Time1.72s (± 1.36%)2.71s (± 0.31%)🔻+1.00s (+58.16%)2.71s2.73sp=0.004 n=6
Check Time17.63s (± 0.46%)22.92s (± 0.38%)🔻+5.28s (+29.95%)22.80s23.00sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time23.30s (± 0.26%)30.92s (± 0.24%)🔻+7.62s (+32.68%)30.83s31.00sp=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols531,594531,594~~~p=1.000 n=6
Types181,677181,677~~~p=1.000 n=6
Memory used463,716k (± 0.02%)548,893k (± 0.01%)🔻+85,177k (+18.37%)548,843k548,929kp=0.005 n=6
Parse Time2.60s (± 0.58%)3.60s (± 0.48%)🔻+0.99s (+38.22%)3.58s3.62sp=0.005 n=6
Bind Time0.93s1.51s (± 0.50%)🔻+0.58s (+62.19%)1.50s1.52sp=0.003 n=6
Check Time15.40s (± 0.39%)17.98s (± 0.26%)🔻+2.57s (+16.68%)17.91s18.01sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time18.94s (± 0.32%)23.08s (± 0.26%)🔻+4.15s (+21.91%)23.00s23.14sp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

tsserver

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,293ms (± 0.22%)3,148ms (± 0.29%)🔻+855ms (+37.29%)3,134ms3,158msp=0.005 n=6
Req 2 - geterr5,177ms (± 0.33%)6,572ms (± 0.41%)🔻+1,396ms (+26.96%)6,551ms6,623msp=0.005 n=6
Req 3 - references262ms (± 0.20%)391ms (± 0.67%)🔻+129ms (+49.05%)387ms394msp=0.004 n=6
Req 4 - navto227ms (± 0.60%)288ms (± 1.72%)🔻+61ms (+26.99%)283ms293msp=0.004 n=6
Req 5 - completionInfo count1,3571,357~~~p=1.000 n=6
Req 5 - completionInfo83ms (± 6.68%)81ms (± 0.63%)~81ms82msp=0.365 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,401ms (± 1.03%)3,341ms (± 0.39%)🔻+940ms (+39.14%)3,327ms3,357msp=0.005 n=6
Req 2 - geterr3,920ms (± 0.60%)5,042ms (± 1.36%)🔻+1,122ms (+28.63%)4,907ms5,095msp=0.005 n=6
Req 3 - references274ms (± 1.47%)403ms (± 3.11%)🔻+129ms (+47.23%)387ms419msp=0.005 n=6
Req 4 - navto232ms (± 2.85%)299ms (± 2.81%)🔻+67ms (+29.09%)289ms307msp=0.005 n=6
Req 5 - completionInfo count1,5191,519~~~p=1.000 n=6
Req 5 - completionInfo74ms (± 7.87%)80ms (± 3.31%)~77ms83msp=0.064 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen5,181ms (± 0.25%)7,148ms (± 2.35%)🔻+1,967ms (+37.97%)7,040ms7,470msp=0.005 n=6
Req 2 - geterr1,131ms (± 1.25%)1,263ms (± 0.92%)🔻+132ms (+11.63%)1,250ms1,277msp=0.005 n=6
Req 3 - references87ms (± 2.49%)106ms (± 3.04%)🔻+19ms (+21.73%)104ms112msp=0.004 n=6
Req 4 - navto441ms (± 0.12%)492ms (± 3.29%)🔻+51ms (+11.56%)469ms511msp=0.004 n=6
Req 5 - completionInfo count3,4443,444~~~p=1.000 n=6
Req 5 - completionInfo844ms (± 0.91%)892ms (± 0.89%)🔻+48ms (+ 5.65%)880ms902msp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

startup

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
tsc-startup - node (v18.15.0, x64)
Execution time158.56ms (± 0.19%)176.19ms (± 0.16%)🔻+17.63ms (+11.12%)175.13ms179.84msp=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time231.96ms (± 0.12%)248.54ms (± 0.13%)🔻+16.58ms (+ 7.15%)247.30ms254.45msp=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time227.82ms (± 0.16%)243.73ms (± 0.15%)🔻+15.91ms (+ 6.98%)242.52ms250.14msp=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time227.03ms (± 0.15%)242.86ms (± 0.14%)🔻+15.83ms (+ 6.97%)241.54ms248.37msp=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@rbuckton

Ron Buckton (rbuckton) commented Sep 18, 2024

Copy link
Copy Markdown
ContributorAuthor

Being less aggressive with respect to optional chaining in the facade accessors didn't seem to have much of an impact. I suspect the biggest impact is that the per-node constructors are resulting in far more megamorphic ICs. I might be able to address this by duplicating all of the accessors on Node on each of the individual subtypes so that the accessors are monomorphic. I'll push up an experimental commit to test that this afternoon, but I think the major perf improvements will come from migrating the internals to use the monomorphic slim AST.

@rbuckton

Ron Buckton (rbuckton) commented Sep 18, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 18, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test✅ Started👀 Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 18, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3030~~~p=1.000 n=6
Symbols62,15362,153~~~p=1.000 n=6
Types50,24250,242~~~p=1.000 n=6
Memory used193,603k (± 0.94%)265,714k (± 0.58%)🔻+72,111k (+37.25%)264,930k268,878kp=0.005 n=6
Parse Time1.57s (± 0.94%)2.15s (± 0.38%)🔻+0.58s (+37.01%)2.14s2.16sp=0.005 n=6
Bind Time0.85s (± 0.96%)1.39s (± 1.06%)🔻+0.54s (+63.09%)1.37s1.41sp=0.005 n=6
Check Time11.37s (± 0.65%)14.36s (± 0.36%)🔻+2.99s (+26.32%)14.27s14.42sp=0.005 n=6
Emit Time3.23s (± 0.69%)5.06s (± 3.15%)🔻+1.83s (+56.73%)4.99s5.39sp=0.005 n=6
Total Time17.02s (± 0.45%)22.97s (± 0.76%)🔻+5.95s (+34.92%)22.78s23.30sp=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors77~~~p=1.000 n=6
Symbols945,753945,753~~~p=1.000 n=6
Types410,067410,067~~~p=1.000 n=6
Memory used1,222,741k (± 0.00%)1,672,763k (± 0.00%)🔻+450,023k (+36.80%)1,672,723k1,672,804kp=0.005 n=6
Parse Time6.67s (± 0.44%)10.24s (± 0.51%)🔻+3.58s (+53.64%)10.18s10.30sp=0.005 n=6
Bind Time1.86s (± 0.28%)3.51s (± 0.39%)🔻+1.65s (+88.64%)3.50s3.53sp=0.004 n=6
Check Time31.19s (± 0.22%)45.35s (± 0.48%)🔻+14.16s (+45.40%)45.15s45.66sp=0.005 n=6
Emit Time15.08s (± 0.31%)22.79s (± 0.74%)🔻+7.71s (+51.15%)22.53s22.95sp=0.005 n=6
Total Time54.80s (± 0.19%)81.90s (± 0.22%)🔻+27.10s (+49.46%)81.77s82.25sp=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols2,529,1562,529,156~~~p=1.000 n=6
Types936,017936,017~~~p=1.000 n=6
Memory used2,364,315k (± 0.00%)2,747,791k (± 0.01%)🔻+383,476k (+16.22%)2,747,489k2,747,957kp=0.005 n=6
Parse Time11.15s (± 0.32%)15.37s (± 0.71%)🔻+4.22s (+37.86%)15.19s15.52sp=0.005 n=6
Bind Time2.60s (± 0.40%)4.62s (± 0.29%)🔻+2.02s (+77.42%)4.60s4.64sp=0.005 n=6
Check Time86.35s (± 0.39%)106.59s (± 0.73%)🔻+20.24s (+23.44%)105.87s107.93sp=0.005 n=6
Emit Time0.33s (± 1.68%)0.34s (± 1.19%)🔻+0.02s (+ 5.13%)0.34s0.35sp=0.003 n=6
Total Time100.43s (± 0.32%)126.93s (± 0.57%)🔻+26.50s (+26.38%)126.19s128.08sp=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,245,9521,455,886🔻+209,934 (+16.85%)~~p=0.001 n=6
Types264,213277,450🔻+13,237 (+ 5.01%)~~p=0.001 n=6
Memory used2,398,064k (± 0.04%)3,483,558k (± 0.02%)🔻+1,085,494k (+45.27%)3,482,941k3,484,290kp=0.005 n=6
Parse Time6.12s (± 1.08%)10.60s (± 0.30%)🔻+4.48s (+73.11%)10.57s10.66sp=0.005 n=6
Bind Time2.27s (± 0.77%)4.46s (± 0.46%)🔻+2.19s (+96.69%)4.43s4.49sp=0.005 n=6
Check Time41.07s (± 0.33%)59.74s (± 0.60%)🔻+18.68s (+45.48%)59.27s60.17sp=0.005 n=6
Emit Time3.65s (± 3.29%)5.42s (± 5.13%)🔻+1.77s (+48.58%)5.21s5.82sp=0.005 n=6
Total Time53.13s (± 0.36%)80.24s (± 0.34%)🔻+27.11s (+51.02%)79.73s80.47sp=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,245,9521,455,886🔻+209,934 (+16.85%)~~p=0.001 n=6
Types264,213277,450🔻+13,237 (+ 5.01%)~~p=0.001 n=6
Memory used2,472,200k (± 0.04%)3,570,912k (± 0.01%)🔻+1,098,711k (+44.44%)3,570,533k3,571,320kp=0.005 n=6
Parse Time7.84s (± 0.67%)13.35s (± 0.54%)🔻+5.50s (+70.14%)13.28s13.47sp=0.005 n=6
Bind Time2.54s (± 0.35%)5.47s (± 1.05%)🔻+2.93s (+115.42%)5.39s5.54sp=0.005 n=6
Check Time51.55s (± 0.41%)74.03s (± 0.64%)🔻+22.47s (+43.59%)73.10s74.36sp=0.005 n=6
Emit Time4.44s (± 1.45%)6.55s (± 3.87%)🔻+2.11s (+47.60%)6.40s7.06sp=0.005 n=6
Total Time66.38s (± 0.25%)99.40s (± 0.23%)🔻+33.02s (+49.75%)99.01s99.65sp=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols259,885334,435🔻+74,550 (+28.69%)~~p=0.001 n=6
Types106,140119,370🔻+13,230 (+12.46%)~~p=0.001 n=6
Memory used435,627k (± 0.01%)696,228k (± 0.02%)🔻+260,602k (+59.82%)696,069k696,392kp=0.005 n=6
Parse Time3.42s (± 0.44%)5.33s (± 0.56%)🔻+1.90s (+55.57%)5.30s5.38sp=0.005 n=6
Bind Time1.29s (± 1.17%)2.47s (± 0.59%)🔻+1.19s (+92.10%)2.45s2.49sp=0.005 n=6
Check Time18.18s (± 0.29%)28.34s (± 0.35%)🔻+10.15s (+55.85%)28.17s28.48sp=0.005 n=6
Emit Time1.53s (± 1.35%)2.91s (± 0.92%)🔻+1.38s (+90.61%)2.86s2.94sp=0.005 n=6
Total Time24.42s (± 0.24%)39.04s (± 0.29%)🔻+14.63s (+59.90%)38.91s39.24sp=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors6868~~~p=1.000 n=6
Symbols225,018225,018~~~p=1.000 n=6
Types94,24994,249~~~p=1.000 n=6
Memory used370,263k (± 0.02%)496,547k (± 0.00%)🔻+126,284k (+34.11%)496,524k496,574kp=0.005 n=6
Parse Time2.28s (± 0.33%)3.32s (± 0.54%)🔻+1.04s (+45.72%)3.30s3.35sp=0.005 n=6
Bind Time1.30s (± 2.15%)2.04s (± 0.59%)🔻+0.74s (+56.78%)2.03s2.06sp=0.004 n=6
Check Time13.36s (± 0.25%)18.68s (± 0.21%)🔻+5.31s (+39.77%)18.61s18.72sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time16.94s (± 0.37%)24.04s (± 0.22%)🔻+7.10s (+41.92%)23.96s24.11sp=0.005 n=6
vscode - node (v18.15.0, x64)
Errors10~~~p=1.000 n=6+0
Symbols3,077,1450~~~p=1.000 n=6+0
Types1,061,6080~~~p=1.000 n=6+0
Memory used3,179,554k (± 0.00%)0k~~~p=1.000 n=6+0
Parse Time17.20s (± 0.42%)0s~~~p=1.000 n=6+0
Bind Time5.30s (± 0.28%)0s~~~p=1.000 n=6+0
Check Time99.15s (± 0.43%)0s~~~p=1.000 n=6+0
Emit Time27.18s (± 0.72%)0s~~~p=1.000 n=6+0
Total Time148.83s (± 0.34%)0s~~~p=1.000 n=6+0
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols277,157277,157~~~p=1.000 n=6
Types112,949112,949~~~p=1.000 n=6
Memory used427,038k (± 0.01%)541,537k (± 0.01%)🔻+114,499k (+26.81%)541,398k541,582kp=0.005 n=6
Parse Time4.90s (± 0.33%)6.83s (± 0.45%)🔻+1.93s (+39.33%)6.79s6.88sp=0.005 n=6
Bind Time2.13s (± 0.91%)3.49s (± 0.15%)🔻+1.36s (+63.82%)3.48s3.49sp=0.004 n=6
Check Time21.89s (± 0.30%)29.51s (± 0.47%)🔻+7.62s (+34.83%)29.35s29.69sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time28.92s (± 0.29%)39.83s (± 0.38%)🔻+10.91s (+37.72%)39.65s40.03sp=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols531,594531,594~~~p=1.000 n=6
Types181,677181,677~~~p=1.000 n=6
Memory used463,730k (± 0.01%)550,468k (± 0.01%)🔻+86,738k (+18.70%)550,439k550,507kp=0.005 n=6
Parse Time3.12s (± 1.06%)4.35s (± 0.58%)🔻+1.23s (+39.35%)4.31s4.38sp=0.005 n=6
Bind Time1.11s (± 0.37%)1.87s (± 0.94%)🔻+0.75s (+67.92%)1.84s1.89sp=0.004 n=6
Check Time18.27s (± 0.13%)21.65s (± 0.21%)🔻+3.38s (+18.52%)21.61s21.73sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time22.50s (± 0.21%)27.86s (± 0.20%)🔻+5.36s (+23.85%)27.80s27.96sp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

tsserver

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,294ms (± 0.93%)3,253ms (± 0.10%)🔻+959ms (+41.82%)3,248ms3,257msp=0.005 n=6
Req 2 - geterr5,161ms (± 0.29%)6,743ms (± 0.51%)🔻+1,582ms (+30.65%)6,694ms6,779msp=0.005 n=6
Req 3 - references264ms (± 1.67%)388ms (± 3.81%)🔻+125ms (+47.28%)375ms407msp=0.005 n=6
Req 4 - navto227ms (± 0.65%)293ms (± 0.14%)🔻+66ms (+28.91%)292ms293msp=0.004 n=6
Req 5 - completionInfo count1,3571,357~~~p=1.000 n=6
Req 5 - completionInfo80ms (± 5.91%)86ms (± 7.20%)~82ms94msp=0.121 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,404ms (± 0.85%)3,434ms (± 0.15%)🔻+1,030ms (+42.87%)3,430ms3,442msp=0.005 n=6
Req 2 - geterr3,912ms (± 0.20%)5,339ms (± 0.40%)🔻+1,427ms (+36.48%)5,320ms5,378msp=0.005 n=6
Req 3 - references274ms (± 0.15%)422ms (± 0.92%)🔻+148ms (+54.05%)414ms424msp=0.003 n=6
Req 4 - navto227ms (± 0.33%)291ms (± 2.59%)🔻+64ms (+28.25%)282ms305msp=0.004 n=6
Req 5 - completionInfo count1,5191,519~~~p=1.000 n=6
Req 5 - completionInfo78ms (± 8.13%)82ms (± 7.76%)~78ms94msp=0.226 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen6,366ms (± 4.95%)8,652ms (± 0.32%)🔻+2,286ms (+35.91%)8,619ms8,685msp=0.005 n=6
Req 2 - geterr1,683ms (± 1.44%)1,732ms (±10.96%)~1,549ms1,922msp=1.000 n=6
Req 3 - references129ms (± 2.79%)154ms (± 8.96%)🔻+25ms (+18.94%)127ms167msp=0.024 n=6
Req 4 - navto603ms (± 1.30%)710ms (± 6.62%)🔻+107ms (+17.75%)615ms741msp=0.005 n=6
Req 5 - completionInfo count3,4443,444~~~p=1.000 n=6
Req 5 - completionInfo1,270ms (± 0.51%)1,285ms (± 8.77%)~1,055ms1,343msp=0.066 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

startup

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
tsc-startup - node (v18.15.0, x64)
Execution time158.16ms (± 0.17%)180.07ms (± 0.17%)🔻+21.92ms (+13.86%)179.06ms184.18msp=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time232.33ms (± 0.15%)254.84ms (± 0.13%)🔻+22.51ms (+ 9.69%)253.52ms260.12msp=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time227.46ms (± 0.14%)248.16ms (± 0.13%)🔻+20.69ms (+ 9.10%)246.67ms251.64msp=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time277.16ms (± 0.28%)304.16ms (± 0.32%)🔻+27.00ms (+ 9.74%)295.62ms307.79msp=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@rbuckton

Ron Buckton (rbuckton) commented Sep 18, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 18, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test✅ Started👀 Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 18, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3030~~~p=1.000 n=6
Symbols62,15362,153~~~p=1.000 n=6
Types50,24250,242~~~p=1.000 n=6
Memory used193,282k (± 0.74%)270,314k (± 0.57%)🔻+77,032k (+39.85%)269,464k273,439kp=0.005 n=6
Parse Time1.56s (± 1.24%)1.66s (± 0.45%)🔻+0.10s (+ 6.40%)1.65s1.67sp=0.004 n=6
Bind Time0.86s (± 0.73%)1.55s (± 0.75%)🔻+0.69s (+80.43%)1.54s1.57sp=0.004 n=6
Check Time11.40s (± 0.14%)14.16s (± 1.58%)🔻+2.77s (+24.29%)13.78s14.35sp=0.005 n=6
Emit Time3.22s (± 0.95%)5.04s (± 4.13%)🔻+1.82s (+56.65%)4.89s5.33sp=0.005 n=6
Total Time17.03s (± 0.23%)22.42s (± 1.31%)🔻+5.39s (+31.65%)21.88s22.76sp=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors77~~~p=1.000 n=6
Symbols945,753945,753~~~p=1.000 n=6
Types410,067410,067~~~p=1.000 n=6
Memory used1,222,713k (± 0.00%)1,725,853k (± 0.00%)🔻+503,140k (+41.15%)1,725,780k1,725,948kp=0.005 n=6
Parse Time7.94s (± 0.85%)9.67s (± 0.42%)🔻+1.73s (+21.74%)9.62s9.72sp=0.005 n=6
Bind Time2.23s (± 0.77%)5.77s (± 0.61%)🔻+3.54s (+158.94%)5.72s5.81sp=0.005 n=6
Check Time36.45s (± 0.48%)50.49s (± 0.32%)🔻+14.04s (+38.52%)50.28s50.74sp=0.005 n=6
Emit Time17.86s (± 0.32%)26.43s (± 0.31%)🔻+8.57s (+47.97%)26.33s26.53sp=0.005 n=6
Total Time64.47s (± 0.34%)92.35s (± 0.26%)🔻+27.88s (+43.24%)91.97s92.63sp=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols2,529,1562,529,156~~~p=1.000 n=6
Types936,017936,017~~~p=1.000 n=6
Memory used2,364,411k (± 0.00%)2,787,320k (± 0.00%)🔻+422,910k (+17.89%)2,787,264k2,787,399kp=0.005 n=6
Parse Time13.79s (± 0.27%)16.55s (± 0.23%)🔻+2.76s (+20.04%)16.51s16.62sp=0.005 n=6
Bind Time3.20s (± 0.86%)7.10s (± 0.28%)🔻+3.90s (+121.87%)7.07s7.13sp=0.005 n=6
Check Time105.85s (± 0.38%)129.36s (± 0.22%)🔻+23.51s (+22.21%)128.98s129.71sp=0.005 n=6
Emit Time0.39s (± 3.08%)0.41s (± 2.52%)🔻+0.02s (+ 5.51%)0.40s0.43sp=0.018 n=6
Total Time123.23s (± 0.36%)153.42s (± 0.20%)🔻+30.19s (+24.50%)153.04s153.86sp=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,246,5521,434,615🔻+188,063 (+15.09%)~~p=0.001 n=6
Types264,445278,258🔻+13,813 (+ 5.22%)~~p=0.001 n=6
Memory used2,459,084k (± 5.93%)3,489,219k (± 0.01%)🔻+1,030,135k (+41.89%)3,488,782k3,489,769kp=0.005 n=6
Parse Time7.63s (± 0.48%)10.19s (± 0.60%)🔻+2.56s (+33.57%)10.11s10.28sp=0.005 n=6
Bind Time2.81s (± 0.42%)7.03s (± 0.91%)🔻+4.22s (+150.33%)6.96s7.13sp=0.005 n=6
Check Time50.94s (± 1.44%)71.55s (± 0.69%)🔻+20.61s (+40.45%)70.93s72.07sp=0.005 n=6
Emit Time4.46s (± 4.40%)6.59s (± 4.85%)🔻+2.13s (+47.67%)6.34s7.05sp=0.005 n=6
Total Time65.85s (± 1.07%)95.38s (± 0.26%)🔻+29.53s (+44.84%)95.12s95.75sp=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,246,5521,434,615🔻+188,063 (+15.09%)~~p=0.001 n=6
Types264,445278,258🔻+13,813 (+ 5.22%)~~p=0.001 n=6
Memory used2,474,074k (± 0.03%)3,578,729k (± 0.01%)🔻+1,104,655k (+44.65%)3,578,410k3,579,132kp=0.005 n=6
Parse Time6.28s (± 0.75%)8.34s (± 0.45%)🔻+2.06s (+32.86%)8.31s8.41sp=0.005 n=6
Bind Time2.04s (± 0.51%)5.63s (± 0.93%)🔻+3.59s (+175.39%)5.58s5.72sp=0.005 n=6
Check Time41.66s (± 0.57%)57.38s (± 0.72%)🔻+15.72s (+37.72%)56.86s57.93sp=0.005 n=6
Emit Time3.52s (± 1.77%)5.31s (± 4.49%)🔻+1.78s (+50.61%)5.13s5.78sp=0.005 n=6
Total Time53.53s (± 0.56%)76.66s (± 0.49%)🔻+23.13s (+43.20%)76.25s77.14sp=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols259,902333,924🔻+74,022 (+28.48%)~~p=0.001 n=6
Types106,141119,820🔻+13,679 (+12.89%)~~p=0.001 n=6
Memory used435,632k (± 0.03%)700,413k (± 0.01%)🔻+264,781k (+60.78%)700,298k700,455kp=0.005 n=6
Parse Time3.43s (± 1.07%)4.28s (± 0.50%)🔻+0.85s (+24.94%)4.26s4.31sp=0.005 n=6
Bind Time1.29s (± 0.94%)2.99s (± 0.91%)🔻+1.69s (+131.06%)2.96s3.02sp=0.005 n=6
Check Time18.13s (± 0.45%)27.28s (± 0.38%)🔻+9.15s (+50.45%)27.15s27.40sp=0.005 n=6
Emit Time1.53s (± 1.37%)2.81s (± 0.65%)🔻+1.28s (+83.77%)2.79s2.83sp=0.005 n=6
Total Time24.38s (± 0.33%)37.36s (± 0.28%)🔻+12.98s (+53.23%)37.22s37.52sp=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors6868~~~p=1.000 n=6
Symbols225,018225,018~~~p=1.000 n=6
Types94,24994,249~~~p=1.000 n=6
Memory used370,241k (± 0.02%)509,399k (± 0.00%)🔻+139,158k (+37.59%)509,365k509,419kp=0.005 n=6
Parse Time2.29s (± 0.51%)2.68s (± 0.91%)🔻+0.38s (+16.73%)2.65s2.72sp=0.005 n=6
Bind Time1.32s (± 1.30%)2.40s (± 0.97%)🔻+1.07s (+81.34%)2.37s2.43sp=0.005 n=6
Check Time13.39s (± 0.16%)18.06s (± 0.36%)🔻+4.67s (+34.89%)17.96s18.16sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time17.00s (± 0.13%)23.13s (± 0.44%)🔻+6.13s (+36.06%)23.00s23.30sp=0.005 n=6
vscode - node (v18.15.0, x64)
Errors10~~~p=1.000 n=6+0
Symbols3,077,2660~~~p=1.000 n=6+0
Types1,061,6680~~~p=1.000 n=6+0
Memory used3,179,782k (± 0.00%)0k~~~p=1.000 n=6+0
Parse Time13.98s (± 0.48%)0s~~~p=1.000 n=6+0
Bind Time4.36s (± 0.30%)0s~~~p=1.000 n=6+0
Check Time81.72s (± 0.35%)0s~~~p=1.000 n=6+0
Emit Time22.20s (± 0.37%)0s~~~p=1.000 n=6+0
Total Time122.27s (± 0.24%)0s~~~p=1.000 n=6+0
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols277,157277,157~~~p=1.000 n=6
Types112,949112,949~~~p=1.000 n=6
Memory used426,912k (± 0.02%)553,555k (± 0.01%)🔻+126,643k (+29.66%)553,499k553,630kp=0.005 n=6
Parse Time3.96s (± 0.21%)4.53s (± 0.90%)🔻+0.57s (+14.41%)4.47s4.58sp=0.005 n=6
Bind Time1.72s (± 0.32%)2.96s (± 0.41%)🔻+1.25s (+72.79%)2.95s2.98sp=0.004 n=6
Check Time17.63s (± 0.17%)23.10s (± 0.33%)🔻+5.47s (+31.00%)23.00s23.18sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time23.30s (± 0.11%)30.58s (± 0.23%)🔻+7.28s (+31.24%)30.51s30.68sp=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols531,594531,594~~~p=1.000 n=6
Types181,677181,677~~~p=1.000 n=6
Memory used463,710k (± 0.01%)561,753k (± 0.02%)🔻+98,043k (+21.14%)561,601k561,867kp=0.005 n=6
Parse Time3.12s (± 0.82%)3.61s (± 0.57%)🔻+0.49s (+15.59%)3.58s3.64sp=0.005 n=6
Bind Time1.11s (± 0.37%)2.25s (± 0.78%)🔻+1.13s (+101.95%)2.22s2.27sp=0.003 n=6
Check Time18.21s (± 0.28%)21.31s (± 0.29%)🔻+3.11s (+17.06%)21.22s21.38sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time22.44s (± 0.27%)27.17s (± 0.22%)🔻+4.73s (+21.06%)27.07s27.24sp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

tsserver

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,860ms (± 9.43%)3,641ms (± 3.07%)🔻+781ms (+27.32%)3,583ms3,869msp=0.005 n=6
Req 2 - geterr6,631ms (±10.45%)9,031ms (± 9.89%)🔻+2,401ms (+36.21%)7,867ms9,870msp=0.005 n=6
Req 3 - references392ms (± 1.69%)512ms (±10.46%)🔻+120ms (+30.47%)464ms580msp=0.005 n=6
Req 4 - navto337ms (± 2.33%)427ms (± 7.66%)🔻+90ms (+26.61%)360ms442msp=0.005 n=6
Req 5 - completionInfo count1,3571,357~~~p=1.000 n=6
Req 5 - completionInfo118ms (± 7.77%)136ms (± 6.85%)🔻+17ms (+14.51%)118ms146msp=0.016 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen3,126ms (±11.90%)4,074ms (± 8.45%)🔻+948ms (+30.32%)3,825ms4,724msp=0.005 n=6
Req 2 - geterr5,373ms (±10.23%)6,795ms (±11.77%)🔻+1,422ms (+26.46%)6,009ms7,613msp=0.005 n=6
Req 3 - references390ms (± 8.91%)570ms (± 9.48%)🔻+180ms (+46.21%)518ms643msp=0.005 n=6
Req 4 - navto338ms (± 2.42%)411ms (± 8.96%)🔻+74ms (+21.78%)362ms438msp=0.005 n=6
Req 5 - completionInfo count1,5191,519~~~p=1.000 n=6
Req 5 - completionInfo115ms (± 5.83%)113ms (± 8.56%)~94ms119msp=0.329 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen6,476ms (± 6.25%)8,568ms (± 5.63%)🔻+2,092ms (+32.31%)8,112ms9,015msp=0.005 n=6
Req 2 - geterr1,689ms (± 0.58%)1,763ms (±10.37%)~1,525ms1,894msp=0.378 n=6
Req 3 - references128ms (± 2.84%)152ms (± 3.15%)🔻+24ms (+18.88%)146ms156msp=0.004 n=6
Req 4 - navto600ms (± 2.64%)691ms (± 1.39%)🔻+91ms (+15.10%)684ms709msp=0.005 n=6
Req 5 - completionInfo count3,4443,444~~~p=1.000 n=6
Req 5 - completionInfo1,257ms (± 1.17%)1,305ms (± 0.55%)+48ms (+ 3.79%)1,297ms1,317msp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

startup

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
tsc-startup - node (v18.15.0, x64)
Execution time189.74ms (± 0.19%)210.37ms (± 0.18%)🔻+20.63ms (+10.87%)207.35ms214.11msp=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time285.00ms (± 0.27%)306.29ms (± 0.30%)🔻+21.29ms (+ 7.47%)298.29ms309.65msp=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time277.37ms (± 0.27%)297.86ms (± 0.33%)🔻+20.50ms (+ 7.39%)289.74ms301.95msp=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time278.46ms (± 0.29%)299.06ms (± 0.34%)🔻+20.59ms (+ 7.40%)290.15ms304.24msp=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@rbuckton

Ron Buckton (rbuckton) commented Sep 20, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 20, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test✅ Started👀 Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Sep 20, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3030~~~p=1.000 n=6
Symbols62,15362,153~~~p=1.000 n=6
Types50,24250,242~~~p=1.000 n=6
Memory used193,675k (± 0.92%)271,009k (± 0.74%)🔻+77,334k (+39.93%)269,506k273,665kp=0.005 n=6
Parse Time1.56s (± 0.52%)1.70s (± 0.88%)🔻+0.14s (+ 8.96%)1.69s1.73sp=0.004 n=6
Bind Time0.86s (± 1.20%)1.56s (± 0.52%)🔻+0.70s (+81.71%)1.55s1.57sp=0.004 n=6
Check Time11.38s (± 0.48%)13.96s (± 0.51%)🔻+2.58s (+22.66%)13.87s14.07sp=0.005 n=6
Emit Time3.23s (± 0.91%)5.06s (± 3.68%)🔻+1.82s (+56.34%)4.92s5.31sp=0.005 n=6
Total Time17.04s (± 0.55%)22.28s (± 1.05%)🔻+5.25s (+30.79%)22.07s22.60sp=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors77~~~p=1.000 n=6
Symbols945,753945,753~~~p=1.000 n=6
Types410,067410,067~~~p=1.000 n=6
Memory used1,222,733k (± 0.00%)1,725,860k (± 0.00%)🔻+503,127k (+41.15%)1,725,803k1,725,911kp=0.005 n=6
Parse Time6.67s (± 0.71%)8.31s (± 0.56%)🔻+1.65s (+24.73%)8.24s8.37sp=0.005 n=6
Bind Time1.86s (± 0.28%)4.89s (± 0.36%)🔻+3.02s (+162.25%)4.86s4.91sp=0.004 n=6
Check Time31.19s (± 0.15%)43.46s (± 0.31%)🔻+12.27s (+39.33%)43.28s43.63sp=0.005 n=6
Emit Time15.02s (± 1.01%)22.53s (± 0.65%)🔻+7.51s (+49.96%)22.33s22.64sp=0.005 n=6
Total Time54.74s (± 0.32%)79.19s (± 0.31%)🔻+24.44s (+44.65%)78.83s79.52sp=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols2,529,4722,529,472~~~p=1.000 n=6
Types936,018936,018~~~p=1.000 n=6
Memory used2,364,794k (± 0.00%)2,787,683k (± 0.00%)🔻+422,889k (+17.88%)2,787,649k2,787,743kp=0.005 n=6
Parse Time11.17s (± 0.16%)13.42s (± 0.21%)🔻+2.25s (+20.14%)13.40s13.47sp=0.005 n=6
Bind Time2.61s (± 0.31%)5.81s (± 0.47%)🔻+3.20s (+122.51%)5.78s5.86sp=0.005 n=6
Check Time86.33s (± 0.43%)105.85s (± 0.22%)🔻+19.52s (+22.61%)105.50s106.14sp=0.005 n=6
Emit Time0.33s0.35s (± 3.35%)🔻+0.02s (+ 5.56%)0.33s0.36sp=0.009 n=6
Total Time100.45s (± 0.36%)125.44s (± 0.19%)🔻+24.99s (+24.87%)125.05s125.69sp=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,246,5501,560,508🔻+313,958 (+25.19%)~~p=0.001 n=6
Types264,444293,720🔻+29,276 (+11.07%)~~p=0.001 n=6
Memory used2,398,746k (± 0.03%)3,507,887k (± 0.02%)🔻+1,109,141k (+46.24%)3,506,214k3,508,490kp=0.005 n=6
Parse Time6.10s (± 0.60%)8.10s (± 0.61%)🔻+2.00s (+32.80%)8.03s8.16sp=0.005 n=6
Bind Time2.27s (± 0.87%)5.65s (± 0.32%)🔻+3.38s (+149.26%)5.63s5.68sp=0.005 n=6
Check Time41.22s (± 0.91%)60.31s (± 0.75%)🔻+19.09s (+46.31%)59.60s60.96sp=0.005 n=6
Emit Time3.62s (± 4.27%)5.37s (± 0.52%)🔻+1.75s (+48.32%)5.33s5.41sp=0.005 n=6
Total Time53.23s (± 0.80%)79.44s (± 0.55%)🔻+26.21s (+49.25%)78.77s80.06sp=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,246,5501,560,508🔻+313,958 (+25.19%)~~p=0.001 n=6
Types264,444293,720🔻+29,276 (+11.07%)~~p=0.001 n=6
Memory used2,473,341k (± 0.01%)3,608,594k (± 0.01%)🔻+1,135,254k (+45.90%)3,607,830k3,609,111kp=0.005 n=6
Parse Time7.91s (± 0.80%)10.28s (± 0.41%)🔻+2.38s (+30.04%)10.22s10.35sp=0.005 n=6
Bind Time2.54s (± 0.70%)6.95s (± 0.70%)🔻+4.41s (+173.82%)6.89s7.02sp=0.005 n=6
Check Time51.62s (± 0.81%)74.63s (± 0.39%)🔻+23.02s (+44.59%)74.13s74.89sp=0.005 n=6
Emit Time4.47s (± 3.19%)6.64s (± 0.86%)🔻+2.16s (+48.42%)6.54s6.70sp=0.005 n=6
Total Time66.56s (± 0.56%)98.49s (± 0.36%)🔻+31.93s (+47.98%)97.86s98.82sp=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols259,902419,353🔻+159,451 (+61.35%)~~p=0.001 n=6
Types106,141127,296🔻+21,155 (+19.93%)~~p=0.001 n=6
Memory used435,647k (± 0.01%)744,542k (± 0.07%)🔻+308,895k (+70.91%)744,155k745,554kp=0.005 n=6
Parse Time4.25s (± 0.80%)5.33s (± 0.44%)🔻+1.07s (+25.19%)5.30s5.36sp=0.005 n=6
Bind Time1.59s (± 1.56%)3.72s (± 0.14%)🔻+2.13s (+134.00%)3.71s3.72sp=0.004 n=6
Check Time22.44s (± 0.42%)34.81s (± 0.65%)🔻+12.37s (+55.13%)34.41s35.08sp=0.005 n=6
Emit Time1.88s (± 0.88%)4.34s (±10.18%)🔻+2.46s (+131.38%)3.46s4.64sp=0.005 n=6
Total Time30.16s (± 0.34%)48.19s (± 0.73%)🔻+18.03s (+59.80%)47.62s48.53sp=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors6868~~~p=1.000 n=6
Symbols225,018225,018~~~p=1.000 n=6
Types94,24994,249~~~p=1.000 n=6
Memory used370,248k (± 0.01%)509,340k (± 0.01%)🔻+139,091k (+37.57%)509,288k509,458kp=0.005 n=6
Parse Time2.75s (± 1.41%)3.24s (± 0.41%)🔻+0.48s (+17.62%)3.23s3.26sp=0.004 n=6
Bind Time1.57s (± 0.66%)2.85s (± 0.98%)🔻+1.29s (+82.13%)2.82s2.90sp=0.004 n=6
Check Time15.71s (± 0.21%)21.09s (± 0.44%)🔻+5.39s (+34.31%)20.97s21.21sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time20.03s (± 0.24%)27.19s (± 0.41%)🔻+7.16s (+35.75%)27.04s27.35sp=0.005 n=6
vscode - node (v18.15.0, x64)
Errors10~~~p=1.000 n=6+0
Symbols3,093,6900~~~p=1.000 n=6+0
Types1,066,2560~~~p=1.000 n=6+0
Memory used3,191,489k (± 0.01%)0k~~~p=1.000 n=6+0
Parse Time11.65s (± 0.41%)0s~~~p=1.000 n=6+0
Bind Time3.68s (± 2.13%)0s~~~p=1.000 n=6+0
Check Time70.82s (± 0.23%)0s~~~p=1.000 n=6+0
Emit Time18.93s (± 0.95%)0s~~~p=1.000 n=6+0
Total Time105.08s (± 0.26%)0s~~~p=1.000 n=6+0
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols277,196277,196~~~p=1.000 n=6
Types112,950112,950~~~p=1.000 n=6
Memory used427,009k (± 0.01%)553,479k (± 0.01%)🔻+126,470k (+29.62%)553,409k553,537kp=0.005 n=6
Parse Time4.91s (± 0.60%)5.63s (± 0.26%)🔻+0.73s (+14.78%)5.61s5.65sp=0.005 n=6
Bind Time2.15s (± 0.76%)3.69s (± 0.70%)🔻+1.54s (+71.59%)3.67s3.73sp=0.005 n=6
Check Time21.73s (± 0.32%)28.53s (± 0.64%)🔻+6.80s (+31.31%)28.33s28.75sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time28.79s (± 0.25%)37.86s (± 0.49%)🔻+9.07s (+31.51%)37.62s38.11sp=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols539,943539,943~~~p=1.000 n=6
Types181,263181,263~~~p=1.000 n=6
Memory used483,352k (± 0.01%)587,368k (± 0.01%)🔻+104,016k (+21.52%)587,327k587,454kp=0.005 n=6
Parse Time3.23s (± 0.47%)3.67s (± 0.76%)🔻+0.44s (+13.64%)3.63s3.70sp=0.005 n=6
Bind Time1.16s (± 0.65%)2.40s (± 0.44%)🔻+1.23s (+106.17%)2.38s2.41sp=0.005 n=6
Check Time18.38s (± 0.34%)21.46s (± 0.31%)🔻+3.08s (+16.78%)21.37s21.56sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time22.77s (± 0.30%)27.53s (± 0.31%)🔻+4.76s (+20.91%)27.39s27.65sp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

tsserver

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,299ms (± 0.56%)3,023ms (± 0.29%)🔻+723ms (+31.46%)3,011ms3,032msp=0.005 n=6
Req 2 - geterr5,169ms (± 0.37%)6,595ms (± 0.33%)🔻+1,426ms (+27.58%)6,567ms6,630msp=0.005 n=6
Req 3 - references261ms (± 0.91%)398ms (± 0.54%)🔻+137ms (+52.30%)395ms401msp=0.005 n=6
Req 4 - navto228ms (± 0.54%)296ms (± 0.30%)🔻+69ms (+30.11%)295ms297msp=0.004 n=6
Req 5 - completionInfo count1,3571,357~~~p=1.000 n=6
Req 5 - completionInfo84ms (± 3.91%)89ms (± 9.62%)~81ms97msp=0.415 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,397ms (± 1.20%)3,222ms (± 0.43%)🔻+825ms (+34.43%)3,203ms3,240msp=0.005 n=6
Req 2 - geterr3,919ms (± 0.21%)5,153ms (± 0.30%)🔻+1,234ms (+31.50%)5,125ms5,168msp=0.005 n=6
Req 3 - references271ms (± 0.86%)431ms (± 2.08%)🔻+159ms (+58.72%)413ms437msp=0.005 n=6
Req 4 - navto234ms (± 3.23%)295ms🔻+61ms (+25.98%)~~p=0.003 n=6
Req 5 - completionInfo count1,5191,519~~~p=1.000 n=6
Req 5 - completionInfo74ms (± 8.62%)79ms (± 1.04%)~78ms80msp=0.063 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen6,493ms (± 6.00%)8,274ms (± 0.65%)🔻+1,781ms (+27.44%)8,218ms8,340msp=0.005 n=6
Req 2 - geterr1,689ms (± 0.62%)1,692ms (±11.64%)~1,505ms1,911msp=1.000 n=6
Req 3 - references131ms (± 3.24%)147ms (± 7.82%)~124ms153msp=0.063 n=6
Req 4 - navto592ms (± 1.57%)698ms (± 1.86%)🔻+106ms (+17.87%)686ms722msp=0.005 n=6
Req 5 - completionInfo count3,4443,444~~~p=1.000 n=6
Req 5 - completionInfo1,258ms (± 1.19%)1,325ms (± 1.65%)🔻+67ms (+ 5.35%)1,291ms1,351msp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

startup

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
tsc-startup - node (v18.15.0, x64)
Execution time158.42ms (± 0.19%)170.68ms (± 0.18%)🔻+12.26ms (+ 7.74%)169.67ms175.04msp=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time232.43ms (± 0.15%)256.35ms (± 0.13%)🔻+23.92ms (+10.29%)255.05ms263.13msp=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time278.42ms (± 0.29%)300.36ms (± 0.28%)🔻+21.94ms (+ 7.88%)293.01ms302.91msp=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time227.48ms (± 0.15%)245.01ms (± 0.13%)🔻+17.53ms (+ 7.70%)243.41ms249.77msp=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@rbuckton

Ron Buckton (rbuckton) commented Oct 1, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test

@typescript-bot

TypeScript Bot (typescript-bot) commented Oct 1, 2024

Copy link
Copy Markdown
Contributor

Hey Ron Buckton (@rbuckton), this PR is in an unmergable state, so is missing a merge commit to run against; please resolve conflicts and try again.

@rbuckton

Ron Buckton (rbuckton) commented Oct 1, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test

@typescript-bot

TypeScript Bot (typescript-bot) commented Oct 1, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test✅ Started👀 Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Oct 1, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3131~~~p=1.000 n=6
Symbols62,34062,340~~~p=1.000 n=6
Types50,37950,379~~~p=1.000 n=6
Memory used195,816k (± 0.73%)273,023k (± 0.09%)🔻+77,207k (+39.43%)272,672k273,282kp=0.005 n=6
Parse Time1.62s (± 1.28%)1.74s (± 2.40%)🔻+0.12s (+ 7.42%)1.67s1.77sp=0.005 n=6
Bind Time0.87s (± 1.03%)1.37s (± 0.75%)🔻+0.50s (+57.85%)1.36s1.39sp=0.005 n=6
Check Time11.76s (± 0.49%)12.31s (± 0.86%)🔻+0.55s (+ 4.65%)12.22s12.52sp=0.005 n=6
Emit Time3.30s (± 0.76%)4.26s (± 5.80%)🔻+0.97s (+29.32%)4.08s4.62sp=0.005 n=6
Total Time17.55s (± 0.37%)19.68s (± 1.26%)🔻+2.14s (+12.19%)19.45s19.99sp=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors3333~~~p=1.000 n=6
Symbols947,886947,886~~~p=1.000 n=6
Types410,840410,840~~~p=1.000 n=6
Memory used1,224,797k (± 0.01%)1,726,906k (± 0.00%)🔻+502,109k (+41.00%)1,726,824k1,726,947kp=0.005 n=6
Parse Time8.09s (± 0.83%)9.73s (± 0.39%)🔻+1.64s (+20.28%)9.69s9.79sp=0.005 n=6
Bind Time2.26s (± 0.65%)5.21s (± 0.43%)🔻+2.95s (+130.21%)5.17s5.23sp=0.005 n=6
Check Time37.69s (± 0.48%)43.74s (± 0.37%)🔻+6.05s (+16.05%)43.60s43.95sp=0.005 n=6
Emit Time18.19s (± 0.50%)22.27s (± 0.43%)🔻+4.09s (+22.47%)22.09s22.35sp=0.005 n=6
Total Time66.23s (± 0.41%)80.96s (± 0.26%)🔻+14.72s (+22.23%)80.67s81.19sp=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols2,522,0382,522,038~~~p=1.000 n=6
Types936,053936,053~~~p=1.000 n=6
Memory used2,352,836k (± 0.01%)2,770,805k (± 0.00%)🔻+417,968k (+17.76%)2,770,695k2,771,022kp=0.005 n=6
Parse Time11.20s (± 0.53%)13.10s (± 0.53%)🔻+1.91s (+17.05%)12.98s13.19sp=0.005 n=6
Bind Time2.61s (± 1.29%)5.13s (± 0.90%)🔻+2.52s (+96.30%)5.08s5.19sp=0.005 n=6
Check Time92.14s (± 2.04%)102.32s (± 2.36%)🔻+10.17s (+11.04%)100.68s106.76sp=0.005 n=6
Emit Time0.34s (± 3.04%)0.35s (± 1.81%)~0.34s0.36sp=0.388 n=6
Total Time106.30s (± 1.77%)120.90s (± 2.05%)🔻+14.60s (+13.74%)119.28s125.42sp=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,258,1061,610,348🔻+352,242 (+28.00%)~~p=0.001 n=6
Types266,266296,415🔻+30,149 (+11.32%)~~p=0.001 n=6
Memory used3,146,094k (± 0.03%)4,641,312k (±11.71%)🔻+1,495,218k (+47.53%)3,530,774k4,865,086kp=0.005 n=6
Parse Time6.54s (± 0.52%)8.50s (± 1.69%)🔻+1.97s (+30.12%)8.31s8.62sp=0.005 n=6
Bind Time2.31s (± 0.71%)5.60s (± 2.24%)🔻+3.29s (+142.70%)5.42s5.80sp=0.005 n=6
Check Time43.00s (± 0.22%)56.82s (± 1.12%)🔻+13.82s (+32.14%)56.33s58.06sp=0.005 n=6
Emit Time3.55s (± 3.18%)4.96s (± 1.33%)🔻+1.41s (+39.62%)4.86s5.02sp=0.005 n=6
Total Time55.39s (± 0.27%)75.87s (± 0.78%)🔻+20.48s (+36.98%)75.39s77.02sp=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,258,1061,610,348🔻+352,242 (+28.00%)~~p=0.001 n=6
Types266,266296,415🔻+30,149 (+11.32%)~~p=0.001 n=6
Memory used2,619,344k (±11.20%)4,979,198k (± 0.03%)🔻+2,359,854k (+90.09%)4,977,693k4,981,846kp=0.005 n=6
Parse Time8.24s (± 2.21%)10.88s (± 0.35%)🔻+2.64s (+32.03%)10.83s10.93sp=0.005 n=6
Bind Time2.65s (± 2.01%)6.84s (± 1.86%)🔻+4.19s (+158.08%)6.71s7.04sp=0.005 n=6
Check Time53.55s (± 0.49%)69.86s (± 0.54%)🔻+16.31s (+30.46%)69.48s70.47sp=0.005 n=6
Emit Time4.47s (± 4.88%)6.23s (± 0.92%)🔻+1.76s (+39.30%)6.13s6.29sp=0.005 n=6
Total Time68.91s (± 0.69%)93.79s (± 0.50%)🔻+24.88s (+36.11%)93.35s94.60sp=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols261,785459,520🔻+197,735 (+75.53%)~~p=0.001 n=6
Types106,508128,536🔻+22,028 (+20.68%)~~p=0.001 n=6
Memory used438,764k (± 0.01%)762,686k (± 0.01%)🔻+323,923k (+73.83%)762,489k762,783kp=0.005 n=6
Parse Time3.56s (± 0.95%)4.37s (± 1.04%)🔻+0.81s (+22.74%)4.32s4.44sp=0.005 n=6
Bind Time1.30s (± 0.76%)2.82s (± 0.78%)🔻+1.52s (+117.20%)2.79s2.85sp=0.005 n=6
Check Time18.89s (± 0.61%)27.16s (± 0.39%)🔻+8.27s (+43.76%)27.03s27.31sp=0.005 n=6
Emit Time1.55s (± 1.11%)2.70s (± 0.79%)🔻+1.15s (+73.90%)2.67s2.73sp=0.005 n=6
Total Time25.31s (± 0.51%)37.05s (± 0.46%)🔻+11.74s (+46.40%)36.81s37.27sp=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors6868~~~p=1.000 n=6
Symbols225,919225,919~~~p=1.000 n=6
Types94,41594,415~~~p=1.000 n=6
Memory used371,064k (± 0.02%)508,685k (± 0.02%)🔻+137,621k (+37.09%)508,523k508,791kp=0.005 n=6
Parse Time2.89s (± 1.26%)3.32s (± 1.02%)🔻+0.43s (+14.99%)3.29s3.37sp=0.005 n=6
Bind Time1.60s (± 1.61%)2.88s (± 0.75%)🔻+1.27s (+79.42%)2.84s2.90sp=0.005 n=6
Check Time16.35s (± 0.40%)19.47s (± 0.25%)🔻+3.12s (+19.08%)19.39s19.52sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time20.85s (± 0.23%)25.67s (± 0.23%)🔻+4.82s (+23.14%)25.61s25.74sp=0.005 n=6
vscode - node (v18.15.0, x64)
Errors30~~~p=1.000 n=6+0
Symbols3,110,4780~~~p=1.000 n=6+0
Types1,072,2340~~~p=1.000 n=6+0
Memory used3,202,485k (± 0.01%)0k~~~p=1.000 n=6+0
Parse Time14.05s (± 0.61%)0s~~~p=1.000 n=6+0
Bind Time4.42s (± 0.33%)0s~~~p=1.000 n=6+0
Check Time87.16s (± 3.18%)0s~~~p=1.000 n=6+0
Emit Time24.81s (± 9.62%)0s~~~p=1.000 n=6+0
Total Time130.45s (± 2.74%)0s~~~p=1.000 n=6+0
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols279,484279,484~~~p=1.000 n=6
Types114,024114,024~~~p=1.000 n=6
Memory used429,358k (± 0.02%)554,686k (± 0.01%)🔻+125,328k (+29.19%)554,648k554,734kp=0.005 n=6
Parse Time3.32s (± 0.44%)3.74s (± 0.39%)🔻+0.42s (+12.66%)3.72s3.75sp=0.005 n=6
Bind Time1.45s (± 0.92%)2.03s (± 0.65%)🔻+0.58s (+40.05%)2.01s2.04sp=0.004 n=6
Check Time15.11s (± 0.36%)15.70s (± 0.23%)+0.59s (+ 3.93%)15.65s15.74sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time19.88s (± 0.23%)21.47s (± 0.25%)🔻+1.59s (+ 8.01%)21.39s21.53sp=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors33~~~p=1.000 n=6
Symbols540,560540,560~~~p=1.000 n=6
Types181,448181,448~~~p=1.000 n=6
Memory used484,314k (± 0.01%)587,285k (± 0.01%)🔻+102,971k (+21.26%)587,222k587,368kp=0.005 n=6
Parse Time3.36s (± 1.32%)3.82s (± 0.73%)🔻+0.46s (+13.73%)3.79s3.87sp=0.005 n=6
Bind Time1.18s (± 1.28%)2.20s (± 0.34%)🔻+1.02s (+86.83%)2.19s2.21sp=0.004 n=6
Check Time19.12s (± 0.15%)20.98s (± 0.24%)🔻+1.86s (+ 9.71%)20.90s21.05sp=0.005 n=6
Emit Time0.00s (±244.70%)0.00s~~~p=0.405 n=6
Total Time23.66s (± 0.29%)27.00s (± 0.14%)🔻+3.34s (+14.10%)26.96s27.05sp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

tsserver

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,309ms (± 0.45%)2,881ms (± 0.29%)🔻+571ms (+24.73%)2,871ms2,892msp=0.005 n=6
Req 2 - geterr5,288ms (± 0.22%)5,880ms (± 0.34%)🔻+593ms (+11.21%)5,846ms5,897msp=0.005 n=6
Req 3 - references268ms (± 1.02%)308ms (± 3.07%)🔻+40ms (+15.08%)302ms327msp=0.005 n=6
Req 4 - navto228ms (± 0.58%)273ms (± 1.75%)🔻+45ms (+19.58%)265ms277msp=0.005 n=6
Req 5 - completionInfo count1,3571,357~~~p=1.000 n=6
Req 5 - completionInfo82ms (± 7.15%)69ms (± 2.38%)🟩-13ms (-16.26%)68ms72msp=0.004 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,464ms (± 0.32%)3,110ms (± 0.48%)🔻+646ms (+26.24%)3,093ms3,127msp=0.005 n=6
Req 2 - geterr3,968ms (± 0.27%)4,356ms (± 0.14%)🔻+388ms (+ 9.77%)4,349ms4,364msp=0.005 n=6
Req 3 - references284ms (± 0.26%)347ms (± 0.26%)🔻+63ms (+22.11%)346ms348msp=0.004 n=6
Req 4 - navto227ms (± 0.18%)283ms (± 0.58%)🔻+56ms (+24.54%)281ms285msp=0.004 n=6
Req 5 - completionInfo count1,5191,519~~~p=1.000 n=6
Req 5 - completionInfo85ms (± 0.60%)77ms (± 1.33%)🟩-8ms (- 9.38%)76ms78msp=0.004 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen5,237ms (± 0.53%)6,712ms (± 0.47%)🔻+1,475ms (+28.16%)6,667ms6,749msp=0.005 n=6
Req 2 - geterr1,143ms (± 0.59%)1,116ms (± 0.45%)-28ms (- 2.42%)1,109ms1,122msp=0.005 n=6
Req 3 - references84ms (± 4.59%)93ms (± 1.58%)🔻+10ms (+11.58%)92ms95msp=0.004 n=6
Req 4 - navto450ms (± 0.73%)431ms (± 0.36%)🟩-19ms (- 4.22%)429ms433msp=0.005 n=6
Req 5 - completionInfo count3,4503,450~~~p=1.000 n=6
Req 5 - completionInfo851ms (± 1.91%)884ms (± 0.36%)+33ms (+ 3.86%)880ms889msp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

startup

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
tsc-startup - node (v18.15.0, x64)
Execution time159.87ms (± 0.20%)167.62ms (± 0.17%)🔻+7.74ms (+ 4.84%)166.63ms171.90msp=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time234.48ms (± 0.20%)258.88ms (± 0.13%)🔻+24.41ms (+10.41%)257.27ms262.96msp=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time229.55ms (± 0.15%)247.47ms (± 0.14%)🔻+17.92ms (+ 7.81%)245.93ms250.53msp=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time229.05ms (± 0.15%)246.89ms (± 0.15%)🔻+17.84ms (+ 7.79%)245.27ms250.95msp=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@rbuckton

Ron Buckton (rbuckton) commented Oct 2, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test

@typescript-bot

TypeScript Bot (typescript-bot) commented Oct 2, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test✅ Started👀 Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Oct 2, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3131~~~p=1.000 n=6
Symbols62,34062,340~~~p=1.000 n=6
Types50,37950,379~~~p=1.000 n=6
Memory used195,826k (± 0.72%)271,626k (± 0.75%)🔻+75,800k (+38.71%)268,746k273,113kp=0.005 n=6
Parse Time1.61s (± 0.94%)1.75s (± 1.32%)🔻+0.15s (+ 9.14%)1.72s1.78sp=0.005 n=6
Bind Time0.86s (± 2.14%)1.36s (± 1.38%)🔻+0.50s (+57.86%)1.33s1.38sp=0.005 n=6
Check Time11.70s (± 0.38%)12.33s (± 1.67%)🔻+0.63s (+ 5.37%)12.21s12.73sp=0.005 n=6
Emit Time3.34s (± 3.07%)4.35s (± 4.58%)🔻+1.01s (+30.29%)4.08s4.52sp=0.005 n=6
Total Time17.50s (± 0.45%)19.79s (± 1.59%)🔻+2.28s (+13.05%)19.38s20.33sp=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors3333~~~p=1.000 n=6
Symbols947,886947,886~~~p=1.000 n=6
Types410,840410,840~~~p=1.000 n=6
Memory used1,224,807k (± 0.00%)1,726,563k (± 0.01%)🔻+501,756k (+40.97%)1,726,397k1,726,658kp=0.005 n=6
Parse Time8.13s (± 0.42%)9.82s (± 0.40%)🔻+1.70s (+20.86%)9.77s9.87sp=0.005 n=6
Bind Time2.27s (± 0.56%)4.47s (± 0.70%)🔻+2.20s (+96.84%)4.44s4.52sp=0.005 n=6
Check Time37.83s (± 0.19%)38.56s (± 0.40%)+0.73s (+ 1.93%)38.41s38.80sp=0.005 n=6
Emit Time18.27s (± 0.71%)19.95s (± 1.08%)🔻+1.69s (+ 9.23%)19.59s20.22sp=0.005 n=6
Total Time66.49s (± 0.17%)72.80s (± 0.27%)🔻+6.31s (+ 9.49%)72.50s73.09sp=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols2,535,9732,535,973~~~p=1.000 n=6
Types935,535935,535~~~p=1.000 n=6
Memory used2,369,503k (± 0.01%)2,790,944k (± 0.01%)🔻+421,442k (+17.79%)2,790,735k2,791,202kp=0.005 n=6
Parse Time11.37s (± 0.39%)13.29s (± 0.64%)🔻+1.92s (+16.92%)13.19s13.40sp=0.005 n=6
Bind Time2.66s (± 0.57%)4.54s (± 0.27%)🔻+1.89s (+71.19%)4.54s4.57sp=0.004 n=6
Check Time91.84s (± 1.04%)98.91s (± 4.39%)🔻+7.08s (+ 7.70%)94.86s104.59sp=0.005 n=6
Emit Time0.35s (± 1.16%)0.35s (± 1.16%)~0.35s0.36sp=1.000 n=6
Total Time106.21s (± 0.86%)117.11s (± 3.70%)🔻+10.89s (+10.26%)112.95s122.71sp=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,258,1061,610,360🔻+352,254 (+28.00%)~~p=0.001 n=6
Types266,266296,415🔻+30,149 (+11.32%)~~p=0.001 n=6
Memory used3,026,235k (± 9.68%)4,422,498k (±15.51%)🔻+1,396,262k (+46.14%)3,535,287k4,866,950kp=0.005 n=6
Parse Time6.54s (± 1.04%)8.50s (± 2.14%)🔻+1.97s (+30.07%)8.27s8.66sp=0.005 n=6
Bind Time2.34s (± 1.24%)4.63s (± 1.18%)🔻+2.29s (+97.79%)4.57s4.69sp=0.005 n=6
Check Time42.98s (± 0.47%)49.38s (± 0.59%)🔻+6.41s (+14.91%)48.91s49.71sp=0.005 n=6
Emit Time3.48s (± 3.51%)4.85s (± 1.92%)🔻+1.37s (+39.51%)4.76s4.98sp=0.005 n=6
Total Time55.34s (± 0.66%)67.37s (± 0.60%)🔻+12.04s (+21.75%)66.82s67.73sp=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,258,1061,610,360🔻+352,254 (+28.00%)~~p=0.001 n=6
Types266,266296,415🔻+30,149 (+11.32%)~~p=0.001 n=6
Memory used2,645,072k (±10.90%)4,981,678k (± 0.02%)🔻+2,336,606k (+88.34%)4,980,514k4,983,208kp=0.005 n=6
Parse Time8.22s (± 2.45%)10.80s (± 0.46%)🔻+2.58s (+31.42%)10.73s10.86sp=0.005 n=6
Bind Time2.65s (± 2.00%)5.63s (± 1.13%)🔻+2.98s (+112.37%)5.51s5.68sp=0.005 n=6
Check Time53.28s (± 0.46%)61.07s (± 0.28%)🔻+7.79s (+14.62%)60.81s61.29sp=0.005 n=6
Emit Time4.58s (± 6.11%)6.03s (± 1.87%)🔻+1.44s (+31.44%)5.87s6.22sp=0.005 n=6
Total Time68.74s (± 0.58%)83.54s (± 0.28%)🔻+14.80s (+21.54%)83.26s83.88sp=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols261,785459,532🔻+197,747 (+75.54%)~~p=0.001 n=6
Types106,508128,536🔻+22,028 (+20.68%)~~p=0.001 n=6
Memory used438,761k (± 0.01%)762,267k (± 0.01%)🔻+323,506k (+73.73%)762,168k762,375kp=0.005 n=6
Parse Time4.38s (± 0.97%)5.43s (± 0.71%)🔻+1.05s (+24.08%)5.37s5.48sp=0.005 n=6
Bind Time1.63s (± 0.72%)2.99s (± 0.72%)🔻+1.37s (+83.83%)2.96s3.02sp=0.005 n=6
Check Time23.38s (± 0.35%)28.86s (± 0.42%)🔻+5.48s (+23.45%)28.66s28.99sp=0.005 n=6
Emit Time1.92s (± 0.72%)3.28s (± 1.43%)🔻+1.35s (+70.22%)3.20s3.33sp=0.005 n=6
Total Time31.31s (± 0.41%)40.55s (± 0.31%)🔻+9.24s (+29.53%)40.34s40.68sp=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors6868~~~p=1.000 n=6
Symbols225,919225,919~~~p=1.000 n=6
Types94,41594,415~~~p=1.000 n=6
Memory used371,158k (± 0.05%)508,568k (± 0.01%)🔻+137,410k (+37.02%)508,462k508,638kp=0.005 n=6
Parse Time2.91s (± 1.16%)3.33s (± 0.99%)🔻+0.43s (+14.62%)3.30s3.37sp=0.005 n=6
Bind Time1.59s (± 1.01%)2.42s (± 1.17%)🔻+0.83s (+52.36%)2.37s2.45sp=0.005 n=6
Check Time16.37s (± 0.30%)16.82s (± 0.35%)+0.45s (+ 2.73%)16.77s16.90sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time20.87s (± 0.26%)22.57s (± 0.12%)🔻+1.70s (+ 8.16%)22.54s22.61sp=0.005 n=6
vscode - node (v18.15.0, x64)
Errors30~~~p=1.000 n=6+0
Symbols3,111,1890~~~p=1.000 n=6+0
Types1,072,4520~~~p=1.000 n=6+0
Memory used3,203,322k (± 0.01%)0k~~~p=1.000 n=6+0
Parse Time14.05s (± 0.50%)0s~~~p=1.000 n=6+0
Bind Time4.44s (± 0.77%)0s~~~p=1.000 n=6+0
Check Time85.62s (± 1.57%)0s~~~p=1.000 n=6+0
Emit Time23.36s (± 8.15%)0s~~~p=1.000 n=6+0
Total Time127.47s (± 2.51%)0s~~~p=1.000 n=6+0
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols285,140285,140~~~p=1.000 n=6
Types115,774115,774~~~p=1.000 n=6
Memory used434,848k (± 0.02%)560,241k (± 0.01%)🔻+125,393k (+28.84%)560,202k560,293kp=0.005 n=6
Parse Time3.30s (± 0.49%)3.76s (± 0.15%)🔻+0.46s (+13.85%)3.75s3.76sp=0.004 n=6
Bind Time1.46s (± 0.57%)2.04s (± 0.37%)🔻+0.57s (+39.14%)2.03s2.05sp=0.004 n=6
Check Time15.28s (± 0.23%)15.89s (± 0.23%)🔻+0.61s (+ 4.00%)15.84s15.93sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time20.04s (± 0.19%)21.69s (± 0.19%)🔻+1.64s (+ 8.19%)21.63s21.72sp=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors33~~~p=1.000 n=6
Symbols540,222540,222~~~p=1.000 n=6
Types181,145181,145~~~p=1.000 n=6
Memory used483,858k (± 0.01%)586,623k (± 0.01%)🔻+102,766k (+21.24%)586,507k586,707kp=0.005 n=6
Parse Time3.36s (± 0.64%)3.78s (± 0.52%)🔻+0.42s (+12.35%)3.75s3.81sp=0.005 n=6
Bind Time1.17s (± 1.13%)1.96s (± 0.85%)🔻+0.79s (+67.28%)1.94s1.98sp=0.005 n=6
Check Time19.03s (± 0.47%)19.84s (± 0.52%)🔻+0.81s (+ 4.23%)19.68s19.95sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time23.57s (± 0.38%)25.57s (± 0.41%)🔻+2.00s (+ 8.50%)25.41s25.71sp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

tsserver

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,320ms (± 0.22%)2,879ms (± 0.26%)🔻+559ms (+24.08%)2,868ms2,888msp=0.005 n=6
Req 2 - geterr5,282ms (± 0.50%)5,865ms (± 0.42%)🔻+584ms (+11.05%)5,840ms5,898msp=0.005 n=6
Req 3 - references265ms (± 0.78%)314ms (± 3.59%)🔻+50ms (+18.78%)303ms326msp=0.005 n=6
Req 4 - navto229ms (± 0.66%)275ms (± 1.23%)🔻+46ms (+20.20%)269ms277msp=0.005 n=6
Req 5 - completionInfo count1,3571,357~~~p=1.000 n=6
Req 5 - completionInfo82ms (± 6.31%)69ms (± 2.38%)🟩-14ms (-16.43%)68ms72msp=0.004 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen2,460ms (± 0.82%)3,103ms (± 0.43%)🔻+643ms (+26.14%)3,085ms3,119msp=0.005 n=6
Req 2 - geterr3,972ms (± 0.42%)4,365ms (± 0.38%)🔻+393ms (+ 9.89%)4,345ms4,386msp=0.005 n=6
Req 3 - references283ms (± 1.09%)348ms (± 1.13%)🔻+65ms (+23.09%)345ms356msp=0.005 n=6
Req 4 - navto226ms (± 0.28%)279ms (± 2.48%)🔻+53ms (+23.53%)265ms282msp=0.003 n=6
Req 5 - completionInfo count1,5191,519~~~p=1.000 n=6
Req 5 - completionInfo84ms (± 5.60%)79ms (± 4.95%)~76ms87msp=0.285 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen5,221ms (± 0.26%)6,536ms (± 0.47%)🔻+1,316ms (+25.21%)6,492ms6,585msp=0.005 n=6
Req 2 - geterr1,151ms (± 1.20%)1,114ms (± 0.56%)🟩-37ms (- 3.17%)1,106ms1,124msp=0.005 n=6
Req 3 - references80ms (± 4.76%)93ms (± 0.44%)🔻+13ms (+15.56%)92ms93msp=0.003 n=6
Req 4 - navto449ms (± 0.36%)403ms (± 1.41%)🟩-46ms (-10.24%)397ms409msp=0.004 n=6
Req 5 - completionInfo count3,4503,450~~~p=1.000 n=6
Req 5 - completionInfo843ms (± 0.79%)896ms (± 1.58%)🔻+53ms (+ 6.27%)881ms910msp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

startup

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
tsc-startup - node (v18.15.0, x64)
Execution time160.06ms (± 0.17%)167.64ms (± 0.15%)🔻+7.58ms (+ 4.74%)166.68ms171.44msp=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time234.30ms (± 0.15%)258.73ms (± 0.13%)🔻+24.43ms (+10.43%)257.26ms264.12msp=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time229.16ms (± 0.15%)246.96ms (± 0.13%)🔻+17.80ms (+ 7.77%)245.35ms250.81msp=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time228.58ms (± 0.16%)246.43ms (± 0.16%)🔻+17.85ms (+ 7.81%)244.83ms250.49msp=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

@rbuckton

Ron Buckton (rbuckton) commented Oct 3, 2024

Copy link
Copy Markdown
ContributorAuthor

TypeScript Bot (@typescript-bot) perf test faster

@typescript-bot

TypeScript Bot (typescript-bot) commented Oct 3, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

CommandStatusResults
perf test faster✅ Started👀 Results

@typescript-bot

TypeScript Bot (typescript-bot) commented Oct 3, 2024

Copy link
Copy Markdown
Contributor

Ron Buckton (@rbuckton)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
MetricbaselineprDeltaBestWorstp-value
Compiler-Unions - node (v18.15.0, x64)
Errors3131~~~p=1.000 n=6
Symbols62,34062,340~~~p=1.000 n=6
Types50,37950,379~~~p=1.000 n=6
Memory used195,794k (± 0.71%)282,919k (± 0.05%)🔻+87,125k (+44.50%)282,686k283,088kp=0.005 n=6
Parse Time1.61s (± 1.20%)1.70s (± 2.44%)🔻+0.09s (+ 5.48%)1.65s1.76sp=0.005 n=6
Bind Time0.87s (± 1.03%)1.54s (± 0.89%)🔻+0.67s (+77.59%)1.53s1.57sp=0.005 n=6
Check Time11.76s (± 0.61%)13.21s (± 1.38%)🔻+1.45s (+12.36%)13.03s13.41sp=0.005 n=6
Emit Time3.36s (± 3.25%)4.49s (± 4.91%)🔻+1.13s (+33.68%)4.08s4.71sp=0.005 n=6
Total Time17.60s (± 0.40%)20.95s (± 1.59%)🔻+3.35s (+19.02%)20.35s21.32sp=0.005 n=6
angular-1 - node (v18.15.0, x64)
Errors3333~~~p=1.000 n=6
Symbols947,886947,886~~~p=1.000 n=6
Types410,840410,840~~~p=1.000 n=6
Memory used1,224,612k (± 0.00%)1,811,305k (± 0.00%)🔻+586,694k (+47.91%)1,811,260k1,811,340kp=0.005 n=6
Parse Time6.63s (± 0.49%)8.23s (± 0.77%)🔻+1.60s (+24.19%)8.17s8.31sp=0.005 n=6
Bind Time1.89s (± 0.29%)3.96s (± 0.84%)🔻+2.07s (+109.90%)3.92s4.01sp=0.004 n=6
Check Time31.76s (± 0.26%)36.03s (± 0.33%)🔻+4.26s (+13.42%)35.86s36.15sp=0.005 n=6
Emit Time15.23s (± 0.53%)16.38s (± 0.26%)🔻+1.15s (+ 7.54%)16.30s16.41sp=0.005 n=6
Total Time55.50s (± 0.24%)64.59s (± 0.23%)🔻+9.08s (+16.36%)64.35s64.81sp=0.005 n=6
mui-docs - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols2,537,3622,537,362~~~p=1.000 n=6
Types935,789935,789~~~p=1.000 n=6
Memory used2,371,723k (± 0.00%)3,073,324k (± 0.00%)🔻+701,601k (+29.58%)3,073,244k3,073,430kp=0.005 n=6
Parse Time9.49s (± 0.14%)11.08s (± 0.17%)🔻+1.60s (+16.81%)11.05s11.10sp=0.005 n=6
Bind Time2.19s (± 0.55%)4.09s (± 0.51%)🔻+1.90s (+86.70%)4.07s4.12sp=0.004 n=6
Check Time76.26s (± 0.52%)97.91s (± 0.92%)🔻+21.64s (+28.38%)97.08s99.65sp=0.005 n=6
Emit Time0.28s (± 2.88%)0.30s (± 1.70%)🔻+0.02s (+ 7.06%)0.30s0.31sp=0.004 n=6
Total Time88.23s (± 0.45%)113.39s (± 0.79%)🔻+25.16s (+28.51%)112.55s115.11sp=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,258,1061,646,954🔻+388,848 (+30.91%)~~p=0.001 n=6
Types266,266304,305🔻+38,039 (+14.29%)~~p=0.001 n=6
Memory used2,422,381k (± 0.02%)4,085,849k (± 9.23%)🔻+1,663,468k (+68.67%)3,841,962k4,573,342kp=0.005 n=6
Parse Time5.21s (± 0.81%)6.72s (± 0.82%)🔻+1.50s (+28.80%)6.64s6.78sp=0.005 n=6
Bind Time1.94s (± 0.42%)4.12s (± 0.79%)🔻+2.18s (+112.56%)4.07s4.16sp=0.005 n=6
Check Time35.57s (± 0.62%)45.03s (± 0.67%)🔻+9.46s (+26.59%)44.63s45.35sp=0.005 n=6
Emit Time3.02s (± 1.59%)4.60s (± 0.37%)🔻+1.58s (+52.21%)4.58s4.63sp=0.005 n=6
Total Time45.75s (± 0.38%)60.46s (± 0.49%)🔻+14.71s (+32.16%)60.09s60.80sp=0.005 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols1,258,1061,646,954🔻+388,848 (+30.91%)~~p=0.001 n=6
Types266,266304,305🔻+38,039 (+14.29%)~~p=0.001 n=6
Memory used2,495,992k (± 0.02%)4,031,833k (± 7.28%)🔻+1,535,841k (+61.53%)3,907,491k4,631,789kp=0.005 n=6
Parse Time5.36s (± 0.69%)6.87s (± 0.89%)🔻+1.51s (+28.20%)6.82s6.99sp=0.005 n=6
Bind Time1.74s (± 0.43%)4.08s (± 0.40%)🔻+2.34s (+134.90%)4.06s4.11sp=0.004 n=6
Check Time35.90s (± 0.15%)45.47s (± 0.50%)🔻+9.57s (+26.66%)45.08s45.66sp=0.005 n=6
Emit Time3.05s (± 0.83%)4.74s (± 2.64%)🔻+1.69s (+55.30%)4.60s4.91sp=0.005 n=6
Total Time46.06s (± 0.16%)61.16s (± 0.25%)🔻+15.10s (+32.80%)60.91s61.31sp=0.005 n=6
self-compiler - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols261,785485,416🔻+223,631 (+85.43%)~~p=0.001 n=6
Types106,508136,330🔻+29,822 (+28.00%)~~p=0.001 n=6
Memory used438,734k (± 0.01%)839,197k (± 0.02%)🔻+400,463k (+91.28%)838,963k839,416kp=0.005 n=6
Parse Time3.53s (± 1.18%)4.50s (± 0.75%)🔻+0.98s (+27.69%)4.46s4.56sp=0.005 n=6
Bind Time1.31s (± 1.15%)2.53s (± 0.82%)🔻+1.22s (+93.02%)2.50s2.56sp=0.005 n=6
Check Time18.88s (± 0.41%)27.55s (± 1.83%)🔻+8.67s (+45.91%)27.23s28.57sp=0.005 n=6
Emit Time1.53s (± 1.12%)3.31s (± 0.88%)🔻+1.78s (+116.21%)3.28s3.36sp=0.005 n=6
Total Time25.25s (± 0.39%)37.90s (± 1.30%)🔻+12.65s (+50.10%)37.55s38.89sp=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors6868~~~p=1.000 n=6
Symbols225,919225,919~~~p=1.000 n=6
Types94,41594,415~~~p=1.000 n=6
Memory used371,174k (± 0.03%)532,076k (± 0.01%)🔻+160,903k (+43.35%)531,999k532,198kp=0.005 n=6
Parse Time3.64s (± 0.64%)4.20s (± 1.10%)🔻+0.56s (+15.35%)4.12s4.26sp=0.005 n=6
Bind Time1.97s (± 1.26%)3.23s (± 0.46%)🔻+1.26s (+64.18%)3.21s3.25sp=0.005 n=6
Check Time20.35s (± 0.30%)22.46s (± 0.30%)🔻+2.11s (+10.36%)22.39s22.58sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time25.95s (± 0.32%)29.89s (± 0.32%)🔻+3.93s (+15.15%)29.80s30.06sp=0.005 n=6
vscode - node (v18.15.0, x64)
Errors30~~~p=1.000 n=6+0
Symbols3,110,9210~~~p=1.000 n=6+0
Types1,072,4860~~~p=1.000 n=6+0
Memory used3,203,828k (± 0.01%)0k~~~p=1.000 n=6+0
Parse Time14.03s (± 0.17%)0s~~~p=1.000 n=6+0
Bind Time4.52s (± 2.76%)0s~~~p=1.000 n=6+0
Check Time85.57s (± 1.51%)0s~~~p=1.000 n=6+0
Emit Time23.41s (± 7.49%)0s~~~p=1.000 n=6+0
Total Time127.53s (± 2.31%)0s~~~p=1.000 n=6+0
webpack - node (v18.15.0, x64)
Errors00~~~p=1.000 n=6
Symbols285,146285,146~~~p=1.000 n=6
Types115,776115,776~~~p=1.000 n=6
Memory used434,827k (± 0.01%)600,285k (± 0.01%)🔻+165,458k (+38.05%)600,237k600,353kp=0.005 n=6
Parse Time4.05s (± 0.46%)4.72s (± 0.56%)🔻+0.67s (+16.44%)4.70s4.77sp=0.005 n=6
Bind Time1.72s (± 1.66%)2.71s (± 1.13%)🔻+0.99s (+57.31%)2.67s2.75sp=0.005 n=6
Check Time18.27s (± 0.45%)20.22s (± 0.25%)🔻+1.94s (+10.63%)20.13s20.28sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time24.05s (± 0.44%)27.64s (± 0.27%)🔻+3.59s (+14.93%)27.57s27.78sp=0.005 n=6
xstate-main - node (v18.15.0, x64)
Errors33~~~p=1.000 n=6
Symbols540,222540,222~~~p=1.000 n=6
Types181,145181,145~~~p=1.000 n=6
Memory used483,897k (± 0.01%)629,406k (± 0.00%)🔻+145,509k (+30.07%)629,373k629,446kp=0.005 n=6
Parse Time4.17s (± 0.68%)4.73s (± 0.70%)🔻+0.56s (+13.35%)4.68s4.77sp=0.005 n=6
Bind Time1.46s (± 1.35%)2.75s (± 0.32%)🔻+1.29s (+88.79%)2.74s2.76sp=0.005 n=6
Check Time23.64s (± 0.36%)26.09s (± 0.22%)🔻+2.45s (+10.38%)25.99s26.16sp=0.005 n=6
Emit Time0.00s0.00s~~~p=1.000 n=6
Total Time29.26s (± 0.32%)33.56s (± 0.19%)🔻+4.30s (+14.70%)33.45s33.63sp=0.005 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
BenchmarkNameIterations
Currentpr6
Baselinebaseline6

Developer Information:

Download Benchmarks

Closed
@microsoftMicrosoft (microsoft) locked as resolved and limited conversation to collaborators Jun 23, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

Copilot code reviewCopilot
Copilot will automatically review once the pull request is marked ready for review

Assignees

@rbucktonRon Buckton (rbuckton)

Labels

Author: TeamFor Uncommitted BugPR for untriaged, rejected, closed or missing bug

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rbuckton@typescript-bot@RyanCavanaugh