Allow tsc to be run within a node v8 snapshot by jakebailey · Pull Request #55830 · microsoft/TypeScript

Jake Bailey (jakebailey) · 2023-09-22T15:15:01Z

This PR switches tsc.js to a wrapper which:

Hashes the contents of tsc.js and the Node version to choose a path a snapshot.
Checks to see if a cached snapshot is available.
If it is, execs that snapshot in a subprocess, passing stdio through.
If it isn't, creates a snapshot on a background thread, while running tsc in the foreground.

Startup speed of the wrapper (as of Feb 13, 2024):

Benchmark 1: node ./built/local/tsc.js --version
Time (mean ± σ): 55.4 ms ± 0.7 ms [User: 40.2 ms, System: 14.3 ms]
Range (min … max): 54.1 ms … 57.5 ms 53 runs
Benchmark 2: node ./built/local/tscReal.js --version
Time (mean ± σ): 88.9 ms ± 0.7 ms [User: 73.3 ms, System: 15.1 ms]
Range (min … max): 86.9 ms … 90.5 ms 33 runs
Benchmark 3: ./tscBun --version
Time (mean ± σ): 104.6 ms ± 1.0 ms [User: 96.4 ms, System: 10.2 ms]
Range (min … max): 102.6 ms … 106.8 ms 28 runs
Summary
'node ./built/local/tsc.js --version' ran
1.61 ± 0.02 times faster than 'node ./built/local/tscReal.js --version'
1.89 ± 0.03 times faster than './tscBun --version'

If we just run the blob directly without the wrapper, the speedup would be a lot more:

$ hyperfine 'node ./built/local/tsc.js --version' 'node --snapshot-blob snapshot.blob -- --version'
Benchmark 1: node ./built/local/tsc.js --version
Time (mean ± σ): 90.7 ms ± 1.6 ms [User: 78.9 ms, System: 11.5 ms]
Range (min … max): 87.1 ms … 94.1 ms 31 runs
Benchmark 2: node --snapshot-blob snapshot.blob -- --version
Time (mean ± σ): 29.4 ms ± 0.9 ms [User: 19.4 ms, System: 9.0 ms]
Range (min … max): 27.7 ms … 32.3 ms 98 runs
Summary
'node --snapshot-blob snapshot.blob -- --version' ran
3.09 ± 0.11 times faster than 'node ./built/local/tsc.js --version'

But there's no easy way to have Node do this.

Related: #25658

Jake Bailey (jakebailey) · 2023-09-30T19:53:10Z

I made an alternative entrypoint which is able to create a snapshot on the fly and use it for further executions. Even in the best case where I don't check that the file hasn't change, it doesn't really save all that much:

$ hyperfine -w 10 'node ./built/local/tsc.js --version' 'node ./snapshotify.js ./built/local/tsc.js --version'
Benchmark 1: node ./built/local/tsc.js --version
Time (mean ± σ): 89.0 ms ± 1.1 ms [User: 78.0 ms, System: 10.8 ms]
Range (min … max): 86.9 ms … 91.1 ms 33 runs
Benchmark 2: node ./snapshotify.js ./built/local/tsc.js --version
Time (mean ± σ): 49.4 ms ± 1.1 ms [User: 35.2 ms, System: 13.4 ms]
Range (min … max): 47.4 ms … 53.9 ms 59 runs
Summary
'node ./snapshotify.js ./built/local/tsc.js --version' ran
1.80 ± 0.05 times faster than 'node ./built/local/tsc.js --version'

$ hyperfine -w 10 'node ./built/local/tsc.js -p scripts' 'node ./snapshotify.js ./built/local/tsc.js -p scripts'
Benchmark 1: node ./built/local/tsc.js -p scripts
Time (mean ± σ): 1.951 s ± 0.041 s [User: 4.422 s, System: 0.179 s]
Range (min … max): 1.888 s … 2.012 s 10 runs
Benchmark 2: node ./snapshotify.js ./built/local/tsc.js -p scripts
Time (mean ± σ): 1.811 s ± 0.027 s [User: 4.257 s, System: 0.149 s]
Range (min … max): 1.780 s … 1.860 s 10 runs
Summary
'node ./snapshotify.js ./built/local/tsc.js -p scripts' ran
1.08 ± 0.03 times faster than 'node ./built/local/tsc.js -p scripts'

Hard to say whether or not this is worth it when we're not shipping the blobs themselves (which, we can't).

Jake Bailey (jakebailey) · 2024-01-09T23:07:51Z

TypeScript Bot (@typescript-bot) perf test

TypeScript Bot (typescript-bot) · 2024-01-09T23:07:54Z

Heya Jake Bailey (@jakebailey), I've started to run the regular perf test suite on this PR at 75c9404. You can monitor the build here.

Update: The results are in!

Jake Bailey (jakebailey) · 2024-01-09T23:23:50Z

TypeScript Bot (@typescript-bot) perf test

TypeScript Bot (typescript-bot) · 2024-01-09T23:23:53Z

Heya Jake Bailey (@jakebailey), I've started to run the regular perf test suite on this PR at e43ced9. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2024-01-09T23:23:54Z

Jake Bailey (@jakebailey), the perf run you requested failed. You can check the log here.

TypeScript Bot (typescript-bot) · 2024-01-09T23:44:10Z

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

startup

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{tsc-startup - node (v18.15.0, x64)}
_{Execution time}	_{153.95ms (± 0.21%)}	_{108.73ms (±13.06%)}	_{🟩-45.22ms (-29.38%)}	_105.72ms	_585.97ms	_{p=0.000 n=600}
_{tsserver-startup - node (v18.15.0, x64)}
_{Execution time}	_{229.16ms (± 0.16%)}	_{229.27ms (± 0.18%)}	_{+0.10ms (+ 0.04%)}	_227.91ms	_235.26ms	_{p=0.044 n=600}
_{tsserverlibrary-startup - node (v18.15.0, x64)}
_{Execution time}	_{230.86ms (± 0.16%)}	_{230.68ms (± 0.17%)}	_{-0.18ms (- 0.08%)}	_229.08ms	_233.46ms	_{p=0.000 n=600}
_{typescript-startup - node (v18.15.0, x64)}
_{Execution time}	_{231.39ms (± 0.17%)}	_{231.39ms (± 0.19%)}	_~	_229.89ms	_237.03ms	_{p=0.832 n=600}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

tsc-startup - node (v18.15.0, x64)
tsserver-startup - node (v18.15.0, x64)
tsserverlibrary-startup - node (v18.15.0, x64)
typescript-startup - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsc

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Angular - node (v18.15.0, x64)}
_{Memory used}	_{295,507k (± 0.01%)}	_{408,607k (±13.53%)}	_{🔻+113,100k (+38.27%)}	_295,670k	_431,590k	_{p=0.005 n=6}
_{Parse Time}	_{2.65s (± 0.24%)}	_{2.65s (± 4.90%)}	_~	_2.57s	_2.91s	_{p=0.060 n=6}
_{Bind Time}	_{0.82s (± 0.00%)}	_{0.82s (± 0.99%)}	_~	_0.81s	_0.83s	_{p=0.290 n=6}
_{Check Time}	_{8.16s (± 0.39%)}	_{8.16s (± 0.40%)}	_~	_8.12s	_8.20s	_{p=0.936 n=6}
_{Emit Time}	_{7.11s (± 0.44%)}	_{7.11s (± 0.21%)}	_~	_7.08s	_7.12s	_{p=1.000 n=6}
_{Total Time}	_{18.74s (± 0.20%)}	_{18.74s (± 0.71%)}	_~	_18.65s	_19.00s	_{p=0.228 n=6}
_{Compiler-Unions - node (v18.15.0, x64)}
_{Memory used}	_{192,427k (± 0.70%)}	_{251,505k (±12.21%)}	_{🔻+59,078k (+30.70%)}	_191,757k	_277,135k	_{p=0.013 n=6}
_{Parse Time}	_{1.35s (± 0.87%)}	_{1.39s (± 9.46%)}	_~	_1.33s	_1.66s	_{p=0.506 n=6}
_{Bind Time}	_{0.72s (± 0.00%)}	_{0.73s (± 0.87%)}	_{+0.01s (+ 1.39%)}	_0.72s	_0.74s	_{p=0.008 n=6}
_{Check Time}	_{9.37s (± 0.44%)}	_{9.24s (± 0.85%)}	_{-0.13s (- 1.39%)}	_9.18s	_9.39s	_{p=0.036 n=6}
_{Emit Time}	_{2.63s (± 0.39%)}	_{2.77s (± 4.17%)}	_~	_2.61s	_2.86s	_{p=0.120 n=6}
_{Total Time}	_{14.06s (± 0.39%)}	_{14.12s (± 1.15%)}	_~	_13.90s	_14.40s	_{p=0.377 n=6}
_{Monaco - node (v18.15.0, x64)}
_{Memory used}	_{347,424k (± 0.00%)}	_{428,209k (± 9.22%)}	_{🔻+80,786k (+23.25%)}	_347,548k	_444,397k	_{p=0.005 n=6}
_{Parse Time}	_{2.47s (± 0.47%)}	_{2.50s (± 5.49%)}	_~	_2.44s	_2.78s	_{p=0.084 n=6}
_{Bind Time}	_{0.93s (± 0.44%)}	_{0.92s (± 0.97%)}	_~	_0.91s	_0.93s	_{p=0.086 n=6}
_{Check Time}	_{6.90s (± 0.53%)}	_{6.87s (± 0.45%)}	_~	_6.84s	_6.92s	_{p=0.295 n=6}
_{Emit Time}	_{4.05s (± 0.44%)}	_{4.04s (± 0.49%)}	_~	_4.01s	_4.06s	_{p=0.871 n=6}
_{Total Time}	_{14.35s (± 0.29%)}	_{14.34s (± 0.99%)}	_~	_14.25s	_14.62s	_{p=0.148 n=6}
_{TFS - node (v18.15.0, x64)}
_{Memory used}	_{302,767k (± 0.01%)}	_{341,013k (± 5.70%)}	_{🔻+38,247k (+12.63%)}	_302,944k	_355,352k	_{p=0.005 n=6}
_{Parse Time}	_{2.01s (± 1.31%)}	_{2.07s (± 6.37%)}	_~	_2.00s	_2.34s	_{p=0.466 n=6}
_{Bind Time}	_{1.00s (± 0.83%)}	_{1.00s (± 0.51%)}	_~	_1.00s	_1.01s	_{p=0.923 n=6}
_{Check Time}	_{6.31s (± 0.32%)}	_{6.30s (± 0.30%)}	_~	_6.28s	_6.33s	_{p=0.570 n=6}
_{Emit Time}	_{3.57s (± 0.41%)}	_{3.58s (± 0.21%)}	_~	_3.57s	_3.59s	_{p=0.099 n=6}
_{Total Time}	_{12.89s (± 0.32%)}	_{12.96s (± 1.09%)}	_~	_12.86s	_13.24s	_{p=0.467 n=6}
_{material-ui - node (v18.15.0, x64)}
_{Memory used}	_{508,271k (± 0.00%)}	_{528,146k (± 1.97%)}	_{+19,875k (+ 3.91%)}	_508,460k	_540,028k	_{p=0.005 n=6}
_{Parse Time}	_{2.59s (± 0.62%)}	_{2.68s (± 4.45%)}	_{+0.09s (+ 3.48%)}	_2.61s	_2.92s	_{p=0.006 n=6}
_{Bind Time}	_{0.99s (± 0.64%)}	_{0.90s (± 5.56%)}	_{🟩-0.09s (- 9.26%)}	_0.87s	_1.00s	_{p=0.046 n=6}
_{Check Time}	_{17.08s (± 0.29%)}	_{17.46s (± 0.79%)}	_{+0.38s (+ 2.23%)}	_17.21s	_17.59s	_{p=0.005 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{20.66s (± 0.26%)}	_{21.04s (± 0.40%)}	_{+0.38s (+ 1.85%)}	_20.96s	_21.14s	_{p=0.005 n=6}
_{xstate - node (v18.15.0, x64)}
_{Memory used}	_{512,980k (± 0.01%)}	_{571,323k (± 4.62%)}	_{🔻+58,343k (+11.37%)}	_517,526k	_584,054k	_{p=0.005 n=6}
_{Parse Time}	_{3.28s (± 0.33%)}	_{3.40s (± 3.34%)}	_{+0.12s (+ 3.61%)}	_3.34s	_3.63s	_{p=0.005 n=6}
_{Bind Time}	_{1.54s (± 0.34%)}	_{1.43s (± 0.69%)}	_{🟩-0.11s (- 6.83%)}	_1.42s	_1.45s	_{p=0.004 n=6}
_{Check Time}	_{2.84s (± 0.62%)}	_{2.82s (± 0.66%)}	_~	_2.80s	_2.85s	_{p=0.373 n=6}
_{Emit Time}	_{0.07s (± 0.00%)}	_{0.07s (± 5.69%)}	_~	_0.07s	_0.08s	_{p=0.405 n=6}
_{Total Time}	_{7.72s (± 0.19%)}	_{7.73s (± 1.75%)}	_~	_7.65s	_8.01s	_{p=0.065 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Angular - node (v18.15.0, x64)
Compiler-Unions - node (v18.15.0, x64)
Monaco - node (v18.15.0, x64)
TFS - node (v18.15.0, x64)
material-ui - node (v18.15.0, x64)
xstate - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsserver

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-UnionsTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,349ms (± 0.48%)}	_{2,344ms (± 0.59%)}	_~	_2,324ms	_2,359ms	_{p=0.520 n=6}
_{Req 2 - geterr}	_{5,513ms (± 1.43%)}	_{5,492ms (± 1.16%)}	_~	_5,413ms	_5,562ms	_{p=0.575 n=6}
_{Req 3 - references}	_{323ms (± 0.36%)}	_{323ms (± 0.41%)}	_~	_322ms	_325ms	_{p=0.739 n=6}
_{Req 4 - navto}	_{273ms (± 1.22%)}	_{274ms (± 1.40%)}	_~	_270ms	_278ms	_{p=0.615 n=6}
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_~	_1,356	_1,356	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{87ms (± 8.83%)}	_{88ms (± 6.98%)}	_~	_79ms	_93ms	_{p=1.000 n=6}
_{CompilerTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,480ms (± 0.92%)}	_{2,477ms (± 0.63%)}	_~	_2,463ms	_2,504ms	_{p=0.689 n=6}
_{Req 2 - geterr}	_{4,141ms (± 1.80%)}	_{4,165ms (± 1.96%)}	_~	_4,076ms	_4,241ms	_{p=0.810 n=6}
_{Req 3 - references}	_{336ms (± 1.76%)}	_{334ms (± 1.10%)}	_~	_331ms	_341ms	_{p=0.871 n=6}
_{Req 4 - navto}	_{285ms (± 1.07%)}	_{285ms (± 1.09%)}	_~	_282ms	_291ms	_{p=0.933 n=6}
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_~	_1,518	_1,518	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{83ms (± 8.93%)}	_{83ms (± 6.18%)}	_~	_78ms	_90ms	_{p=1.000 n=6}
_{xstateTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,596ms (± 0.77%)}	_{2,598ms (± 0.89%)}	_~	_2,562ms	_2,617ms	_{p=0.936 n=6}
_{Req 2 - geterr}	_{1,699ms (± 1.77%)}	_{1,711ms (± 2.58%)}	_~	_1,647ms	_1,762ms	_{p=0.470 n=6}
_{Req 3 - references}	_{116ms (± 9.63%)}	_{123ms (± 6.65%)}	_~	_106ms	_126ms	_{p=0.346 n=6}
_{Req 4 - navto}	_{365ms (± 0.22%)}	_{364ms (± 0.86%)}	_~	_358ms	_366ms	_{p=0.796 n=6}
_{Req 5 - completionInfo count}	_{2,073 (± 0.00%)}	_{2,073 (± 0.00%)}	_~	_2,073	_2,073	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{305ms (± 1.90%)}	_{309ms (± 1.89%)}	_~	_300ms	_315ms	_{p=0.376 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

CompilerTSServer - node (v18.15.0, x64)
Compiler-UnionsTSServer - node (v18.15.0, x64)
xstateTSServer - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

Jake Bailey (jakebailey) · 2024-01-10T06:00:39Z

So far:

Benchmark 1: node ./built/local/tsc.js --version
Time (mean ± σ): 55.4 ms ± 0.7 ms [User: 40.2 ms, System: 14.3 ms]
Range (min … max): 54.1 ms … 57.5 ms 53 runs
Benchmark 2: node ./built/local/tscReal.js --version
Time (mean ± σ): 88.9 ms ± 0.7 ms [User: 73.3 ms, System: 15.1 ms]
Range (min … max): 86.9 ms … 90.5 ms 33 runs
Benchmark 3: ./tscBun --version
Time (mean ± σ): 104.6 ms ± 1.0 ms [User: 96.4 ms, System: 10.2 ms]
Range (min … max): 102.6 ms … 106.8 ms 28 runs
Summary
'node ./built/local/tsc.js --version' ran
1.61 ± 0.02 times faster than 'node ./built/local/tscReal.js --version'
1.89 ± 0.03 times faster than './tscBun --version'

The perf benchmarker does not do warmup runs, so it's a little unhappy. It fares worse than my machine at only a 30% startup boost.

Jake Bailey (jakebailey) · 2024-03-29T22:22:59Z

For those following this, native automatic caching in Node is able to achieve better perf than what I was able to do: nodejs/node#47472 (comment)

$ hyperfine -w 10 -r 100 'NODE_COMPILER_CACHE= $HOME/work/node/out/Release/node ./built/local/tsc.js --version''NODE_COMPILER_CACHE=./tmp $HOME/work/node/out/Release/node ./built/local/tsc.js --version'Benchmark 1: NODE_COMPILER_CACHE= $HOME/work/node/out/Release/node ./built/local/tsc.js --version Time (mean ± σ): 89.9 ms ± 1.4 ms [User: 77.9 ms, System: 11.1 ms] Range (min … max): 88.0 ms … 97.7 ms 100 runsBenchmark 2: NODE_COMPILER_CACHE=./tmp $HOME/work/node/out/Release/node ./built/local/tsc.js --version Time (mean ± σ): 39.9 ms ± 0.6 ms [User: 29.9 ms, System: 9.6 ms] Range (min … max): 38.7 ms … 42.6 ms 100 runsSummary 'NODE_COMPILER_CACHE=./tmp $HOME/work/node/out/Release/node ./built/local/tsc.js --version' ran 2.25 ± 0.05 times faster than 'NODE_COMPILER_CACHE= $HOME/work/node/out/Release/node ./built/local/tsc.js --version'

Jake Bailey (jakebailey) · 2024-04-16T03:59:03Z

TypeScript Bot (@typescript-bot) perf test

TypeScript Bot (typescript-bot) · 2024-04-16T03:59:05Z

Starting jobs; this comment will be updated as builds start and complete.

Command	Status	Results
`perf test`	✅ Started	👀 Results

TypeScript Bot (typescript-bot) · 2024-04-16T04:35:41Z

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Angular - node (v18.15.0, x64)}
_{Memory used}	_{297,008k (± 0.01%)}	_{437,891k (± 1.51%)}	_{🔻+140,883k (+47.43%)}	_431,920k	_446,430k	_{p=0.005 n=6}
_{Parse Time}	_{4.04s (± 0.49%)}	_{3.98s (± 0.25%)}	_{-0.06s (- 1.44%)}	_3.97s	_3.99s	_{p=0.005 n=6}
_{Bind Time}	_{1.21s (± 0.69%)}	_{1.21s (± 0.69%)}	_~	_1.20s	_1.22s	_{p=0.796 n=6}
_{Check Time}	_{12.10s (± 0.33%)}	_{12.12s (± 0.31%)}	_~	_12.07s	_12.16s	_{p=0.687 n=6}
_{Emit Time}	_{10.50s (± 0.33%)}	_{10.59s (± 0.63%)}	_{+0.09s (+ 0.89%)}	_10.50s	_10.70s	_{p=0.020 n=6}
_{Total Time}	_{27.86s (± 0.19%)}	_{27.91s (± 0.29%)}	_~	_27.82s	_28.04s	_{p=0.334 n=6}
_{Compiler-Unions - node (v18.15.0, x64)}
_{Memory used}	_{192,378k (± 0.77%)}	_{263,078k (± 0.32%)}	_{🔻+70,700k (+36.75%)}	_262,019k	_264,139k	_{p=0.005 n=6}
_{Parse Time}	_{2.02s (± 0.37%)}	_{2.00s (± 1.72%)}	_~	_1.97s	_2.05s	_{p=0.373 n=6}
_{Bind Time}	_{1.06s (± 1.03%)}	_{1.08s (± 0.78%)}	_{+0.02s (+ 1.42%)}	_1.06s	_1.08s	_{p=0.042 n=6}
_{Check Time}	_{14.06s (± 0.34%)}	_{13.87s (± 0.62%)}	_{-0.19s (- 1.35%)}	_13.72s	_13.95s	_{p=0.005 n=6}
_{Emit Time}	_{3.85s (± 1.28%)}	_{4.17s (± 0.88%)}	_{🔻+0.33s (+ 8.50%)}	_4.11s	_4.22s	_{p=0.005 n=6}
_{Total Time}	_{20.98s (± 0.38%)}	_{21.11s (± 0.53%)}	_~	_20.93s	_21.25s	_{p=0.066 n=6}
_{Monaco - node (v18.15.0, x64)}
_{Memory used}	_{348,868k (± 0.01%)}	_{454,605k (± 0.03%)}	_{🔻+105,737k (+30.31%)}	_454,485k	_454,816k	_{p=0.005 n=6}
_{Parse Time}	_{3.84s (± 0.71%)}	_{3.81s (± 0.64%)}	_~	_3.76s	_3.83s	_{p=0.076 n=6}
_{Bind Time}	_{1.32s (± 0.68%)}	_{1.31s (± 0.62%)}	_{-0.01s (- 1.01%)}	_1.30s	_1.32s	_{p=0.038 n=6}
_{Check Time}	_{10.28s (± 0.38%)}	_{10.29s (± 0.10%)}	_~	_10.27s	_10.30s	_{p=0.808 n=6}
_{Emit Time}	_{6.00s (± 0.37%)}	_{5.99s (± 0.49%)}	_~	_5.95s	_6.04s	_{p=0.514 n=6}
_{Total Time}	_{21.44s (± 0.28%)}	_{21.39s (± 0.14%)}	_~	_21.36s	_21.44s	_{p=0.142 n=6}
_{TFS - node (v18.15.0, x64)}
_{Memory used}	_{302,964k (± 0.01%)}	_{360,945k (± 1.45%)}	_{🔻+57,981k (+19.14%)}	_350,223k	_363,178k	_{p=0.005 n=6}
_{Parse Time}	_{2.02s (± 0.60%)}	_{2.03s (± 0.98%)}	_~	_2.00s	_2.05s	_{p=0.615 n=6}
_{Bind Time}	_{0.97s (± 0.53%)}	_{0.98s (± 0.91%)}	_~	_0.97s	_0.99s	_{p=0.190 n=6}
_{Check Time}	_{6.43s (± 0.25%)}	_{6.45s (± 0.45%)}	_~	_6.41s	_6.49s	_{p=0.193 n=6}
_{Emit Time}	_{3.58s (± 0.18%)}	_{3.59s (± 0.42%)}	_~	_3.57s	_3.61s	_{p=0.131 n=6}
_{Total Time}	_{13.01s (± 0.21%)}	_{13.06s (± 0.18%)}	_{+0.05s (+ 0.40%)}	_13.02s	_13.09s	_{p=0.019 n=6}
_{material-ui - node (v18.15.0, x64)}
_{Memory used}	_{511,635k (± 0.00%)}	_{533,713k (± 0.93%)}	_{🔻+22,079k (+ 4.32%)}	_523,840k	_537,013k	_{p=0.005 n=6}
_{Parse Time}	_{3.25s (± 0.61%)}	_{3.34s (± 0.54%)}	_{+0.09s (+ 2.61%)}	_3.32s	_3.37s	_{p=0.005 n=6}
_{Bind Time}	_{1.19s (± 1.39%)}	_{1.04s (± 0.53%)}	_{🟩-0.15s (-12.66%)}	_1.03s	_1.04s	_{p=0.004 n=6}
_{Check Time}	_{20.55s (± 0.37%)}	_{21.02s (± 0.45%)}	_{+0.47s (+ 2.30%)}	_20.89s	_21.14s	_{p=0.005 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{24.99s (± 0.34%)}	_{25.39s (± 0.40%)}	_{+0.40s (+ 1.61%)}	_25.26s	_25.52s	_{p=0.005 n=6}
_{mui-docs - node (v18.15.0, x64)}
_{Memory used}	_{1,748,707k (± 0.00%)}	_{1,796,018k (± 1.42%)}	_{+47,311k (+ 2.71%)}	_1,783,786k	_1,848,029k	_{p=0.005 n=6}
_{Parse Time}	_{8.10s (± 0.37%)}	_{8.43s (± 0.88%)}	_{🔻+0.33s (+ 4.05%)}	_8.28s	_8.48s	_{p=0.005 n=6}
_{Bind Time}	_{2.74s (± 0.59%)}	_{2.24s (± 0.73%)}	_{🟩-0.50s (-18.23%)}	_2.22s	_2.27s	_{p=0.004 n=6}
_{Check Time}	_{66.77s (± 0.37%)}	_{68.72s (± 0.51%)}	_{+1.95s (+ 2.93%)}	_68.31s	_69.34s	_{p=0.005 n=6}
_{Emit Time}	_{0.17s (± 6.19%)}	_{0.17s (± 6.19%)}	_~	_0.16s	_0.18s	_{p=1.000 n=6}
_{Total Time}	_{77.78s (± 0.35%)}	_{79.56s (± 0.35%)}	_{+1.79s (+ 2.30%)}	_79.21s	_80.00s	_{p=0.005 n=6}
_{self-build-src - node (v18.15.0, x64)}
_{Memory used}	_{2,306,767k (± 0.05%)}	_{5,365,257k (±14.61%)}	_{🔻+3,058,491k (+132.59%)}	_4,802,148k	_6,670,784k	_{p=0.005 n=6}
_{Parse Time}	_{7.39s (± 0.45%)}	_{7.55s (± 0.68%)}	_{+0.16s (+ 2.12%)}	_7.48s	_7.63s	_{p=0.005 n=6}
_{Bind Time}	_{2.76s (± 0.82%)}	_{2.32s (± 1.60%)}	_{🟩-0.44s (-15.85%)}	_2.28s	_2.38s	_{p=0.005 n=6}
_{Check Time}	_{49.39s (± 0.50%)}	_{45.50s (± 0.59%)}	_{🟩-3.89s (- 7.88%)}	_45.09s	_45.81s	_{p=0.005 n=6}
_{Emit Time}	_{3.94s (± 1.58%)}	_{3.25s (± 2.00%)}	_{🟩-0.68s (-17.36%)}	_3.14s	_3.33s	_{p=0.005 n=6}
_{Total Time}	_{63.48s (± 0.47%)}	_{58.63s (± 0.43%)}	_{🟩-4.84s (- 7.63%)}	_58.18s	_58.92s	_{p=0.005 n=6}
_{self-build-src-public-api - node (v18.15.0, x64)}
_{Memory used}	_{2,381,487k (± 0.05%)}	_{2,382,787k (± 0.03%)}	_~	_2,381,369k	_2,383,747k	_{p=0.093 n=6}
_{Parse Time}	_{7.65s (± 0.42%)}	_{7.59s (± 1.13%)}	_~	_7.43s	_7.67s	_{p=0.199 n=6}
_{Bind Time}	_{2.52s (± 0.71%)}	_{2.52s (± 0.82%)}	_~	_2.48s	_2.54s	_{p=0.871 n=6}
_{Check Time}	_{49.72s (± 0.67%)}	_{49.83s (± 0.62%)}	_~	_49.49s	_50.20s	_{p=0.810 n=6}
_{Emit Time}	_{3.90s (± 1.97%)}	_{3.95s (± 3.37%)}	_~	_3.81s	_4.17s	_{p=0.630 n=6}
_{Total Time}	_{63.79s (± 0.53%)}	_{63.88s (± 0.58%)}	_~	_63.46s	_64.39s	_{p=0.936 n=6}
_{self-compiler - node (v18.15.0, x64)}
_{Memory used}	_{419,093k (± 0.01%)}	_{476,250k (± 1.27%)}	_{🔻+57,157k (+13.64%)}	_470,629k	_486,500k	_{p=0.005 n=6}
_{Parse Time}	_{3.39s (± 0.52%)}	_{3.34s (± 0.52%)}	_{-0.06s (- 1.67%)}	_3.32s	_3.37s	_{p=0.006 n=6}
_{Bind Time}	_{1.31s (± 0.89%)}	_{1.29s (± 0.76%)}	_{-0.02s (- 1.52%)}	_1.28s	_1.30s	_{p=0.022 n=6}
_{Check Time}	_{18.02s (± 0.35%)}	_{18.08s (± 0.36%)}	_~	_17.99s	_18.17s	_{p=0.295 n=6}
_{Emit Time}	_{1.37s (± 1.76%)}	_{1.38s (± 1.09%)}	_~	_1.37s	_1.41s	_{p=0.628 n=6}
_{Total Time}	_{24.10s (± 0.17%)}	_{24.09s (± 0.29%)}	_~	_24.00s	_24.16s	_{p=1.000 n=6}
_{vscode - node (v18.15.0, x64)}
_{Memory used}	_{2,913,896k (± 0.00%)}	_{3,001,919k (± 1.83%)}	_{+88,023k (+ 3.02%)}	_2,958,643k	_3,110,939k	_{p=0.005 n=6}
_{Parse Time}	_{11.24s (± 0.21%)}	_{11.22s (± 0.47%)}	_~	_11.13s	_11.29s	_{p=0.808 n=6}
_{Bind Time}	_{3.41s (± 0.44%)}	_{3.40s (± 0.24%)}	_{-0.02s (- 0.49%)}	_3.39s	_3.41s	_{p=0.039 n=6}
_{Check Time}	_{62.64s (± 0.51%)}	_{62.29s (± 0.45%)}	_~	_61.85s	_62.58s	_{p=0.149 n=6}
_{Emit Time}	_{16.50s (± 0.56%)}	_{19.91s (± 0.75%)}	_{🔻+3.41s (+20.65%)}	_19.65s	_20.08s	_{p=0.005 n=6}
_{Total Time}	_{93.78s (± 0.38%)}	_{96.82s (± 0.44%)}	_{+3.04s (+ 3.24%)}	_96.02s	_97.13s	_{p=0.005 n=6}
_{webpack - node (v18.15.0, x64)}
_{Memory used}	_{409,469k (± 0.01%)}	_{461,662k (± 0.99%)}	_{🔻+52,193k (+12.75%)}	_452,358k	_464,503k	_{p=0.005 n=6}
_{Parse Time}	_{4.80s (± 0.92%)}	_{4.84s (± 1.43%)}	_~	_4.75s	_4.95s	_{p=0.228 n=6}
_{Bind Time}	_{2.03s (± 0.48%)}	_{2.01s (± 0.98%)}	_~	_2.00s	_2.05s	_{p=0.058 n=6}
_{Check Time}	_{21.00s (± 0.19%)}	_{20.96s (± 0.27%)}	_~	_20.88s	_21.04s	_{p=0.260 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{27.83s (± 0.13%)}	_{27.82s (± 0.23%)}	_~	_27.75s	_27.93s	_{p=0.369 n=6}
_{xstate - node (v18.15.0, x64)}
_{Memory used}	_{671,924k (± 0.01%)}	_{760,483k (± 0.05%)}	_{🔻+88,558k (+13.18%)}	_760,169k	_761,089k	_{p=0.005 n=6}
_{Parse Time}	_{3.40s (± 0.40%)}	_{3.48s (± 0.16%)}	_{+0.07s (+ 2.06%)}	_3.47s	_3.48s	_{p=0.005 n=6}
_{Bind Time}	_{1.57s (± 0.00%)}	_{1.43s (± 0.53%)}	_{🟩-0.14s (- 8.81%)}	_1.42s	_1.44s	_{p=0.003 n=6}
_{Check Time}	_{2.90s (± 0.49%)}	_{2.90s (± 0.81%)}	_~	_2.86s	_2.92s	_{p=1.000 n=6}
_{Emit Time}	_{0.02s (± 0.00%)}	_{0.02s (± 0.00%)}	_~	_0.02s	_0.02s	_{p=1.000 n=6}
_{Total Time}	_{7.91s (± 0.24%)}	_{7.83s (± 0.23%)}	_{-0.08s (- 0.97%)}	_7.80s	_7.85s	_{p=0.005 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Angular - node (v18.15.0, x64)
Compiler-Unions - node (v18.15.0, x64)
Monaco - node (v18.15.0, x64)
TFS - node (v18.15.0, x64)
material-ui - node (v18.15.0, x64)
mui-docs - node (v18.15.0, x64)
self-build-src - node (v18.15.0, x64)
self-build-src-public-api - node (v18.15.0, x64)
self-compiler - node (v18.15.0, x64)
vscode - node (v18.15.0, x64)
webpack - node (v18.15.0, x64)
xstate - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsserver

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-UnionsTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{3,499ms (± 0.34%)}	_{3,491ms (± 0.85%)}	_~	_3,460ms	_3,532ms	_{p=0.575 n=6}
_{Req 2 - geterr}	_{7,585ms (± 1.91%)}	_{7,668ms (± 1.92%)}	_~	_7,465ms	_7,831ms	_{p=0.378 n=6}
_{Req 3 - references}	_{426ms (± 0.48%)}	_{426ms (± 0.99%)}	_~	_421ms	_431ms	_{p=0.936 n=6}
_{Req 4 - navto}	_{337ms (± 0.79%)}	_{336ms (± 0.95%)}	_~	_333ms	_340ms	_{p=0.625 n=6}
_{Req 5 - completionInfo count}	_{1,357 (± 0.00%)}	_{1,357 (± 0.00%)}	_~	_1,357	_1,357	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{117ms (± 7.58%)}	_{122ms (± 8.16%)}	_~	_111ms	_135ms	_{p=0.375 n=6}
_{CompilerTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,516ms (± 0.86%)}	_{2,510ms (± 0.92%)}	_~	_2,475ms	_2,539ms	_{p=0.575 n=6}
_{Req 2 - geterr}	_{3,835ms (± 0.23%)}	_{3,909ms (± 2.07%)}	_~	_3,831ms	_3,993ms	_{p=0.173 n=6}
_{Req 3 - references}	_{307ms (± 0.82%)}	_{303ms (± 1.77%)}	_~	_297ms	_310ms	_{p=0.146 n=6}
_{Req 4 - navto}	_{228ms (± 0.28%)}	_{226ms (± 0.91%)}	_~	_223ms	_228ms	_{p=0.129 n=6}
_{Req 5 - completionInfo count}	_{1,519 (± 0.00%)}	_{1,519 (± 0.00%)}	_~	_1,519	_1,519	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{84ms (± 0.62%)}	_{79ms (± 6.05%)}	_~	_72ms	_84ms	_{p=0.103 n=6}
_{xstateTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{3,167ms (± 0.28%)}	_{3,191ms (± 0.87%)}	_~	_3,149ms	_3,230ms	_{p=0.093 n=6}
_{Req 2 - geterr}	_{1,888ms (±12.20%)}	_{1,752ms (± 1.13%)}	_~	_1,732ms	_1,790ms	_{p=0.872 n=6}
_{Req 3 - references}	_{177ms (± 7.49%)}	_{154ms (± 0.86%)}	_{🟩-23ms (-12.90%)}	_153ms	_156ms	_{p=0.012 n=6}
_{Req 4 - navto}	_{496ms (±10.62%)}	_{430ms (± 0.67%)}	_~	_426ms	_434ms	_{p=0.170 n=6}
_{Req 5 - completionInfo count}	_{2,079 (± 0.00%)}	_{2,079 (± 0.00%)}	_~	_2,079	_2,079	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{420ms (±10.94%)}	_{363ms (± 1.46%)}	_~	_357ms	_372ms	_{p=0.108 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

CompilerTSServer - node (v18.15.0, x64)
Compiler-UnionsTSServer - node (v18.15.0, x64)
xstateTSServer - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

startup

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{tsc-startup - node (v18.15.0, x64)}
_{Execution time}	_{153.92ms (± 0.16%)}	_{106.50ms (± 0.20%)}	_{🟩-47.42ms (-30.81%)}	_105.79ms	_108.44ms	_{p=0.000 n=600}
_{tsserver-startup - node (v18.15.0, x64)}
_{Execution time}	_{227.73ms (± 0.14%)}	_{227.77ms (± 0.16%)}	_~	_226.32ms	_232.80ms	_{p=0.480 n=600}
_{tsserverlibrary-startup - node (v18.15.0, x64)}
_{Execution time}	_{224.14ms (± 0.15%)}	_{224.14ms (± 0.18%)}	_~	_222.64ms	_228.67ms	_{p=0.356 n=600}
_{typescript-startup - node (v18.15.0, x64)}
_{Execution time}	_{225.48ms (± 0.18%)}	_{225.52ms (± 0.15%)}	_~	_223.83ms	_228.55ms	_{p=0.188 n=600}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

tsc-startup - node (v18.15.0, x64)
tsserver-startup - node (v18.15.0, x64)
tsserverlibrary-startup - node (v18.15.0, x64)
typescript-startup - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

Jake Bailey (jakebailey) · 2024-06-03T21:01:18Z

Per #55830 (comment), I'm going to close this. I think the future of caching is going to be via Node directly.

Allow tsc to be run within a node v8 snapshot
a4f19b0

TypeScript Bot (typescript-bot) assigned Jake Bailey (jakebailey)Sep 22, 2023

TypeScript Bot (typescript-bot) added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Sep 22, 2023

Another TODO
502961d

Jake Bailey (jakebailey) commented Sep 22, 2023
View reviewed changes

Comment threadsrc/tsc/tsc.ts Outdated

Jake Bailey (jakebailey) commented Sep 22, 2023
View reviewed changes

Comment threadsrc/tsc/tsc.ts Outdated

Jack Works (Jack-Works) reviewed Sep 23, 2023
View reviewed changes

Comment threadsrc/tsc/tsconfig.json

Merge branch 'main' into node-snapshotting
a8e7b8b

Jake Bailey (jakebailey) added 3 commits September 30, 2023 23:28

Add snapshotify script for testing
c6c6377

Merge branch 'main' into node-snapshotting
88f31b9

Fix in bun whose api is present but throws
351b2d7

Daniel Rosenwasser (DanielRosenwasser) mentioned this pull request Jan 4, 2024
TypeScript 5.4 Iteration Plan #56948
Closed

Jake Bailey (jakebailey) added 8 commits January 8, 2024 21:28

Merge branch 'main' into node-snapshotting
92f29f0

Hashing is ok
69c81ad

Cleanup, fix
13720a0

Leave todo about fallback
6d14cce

Push this into src
944b399

Detect non-v8 snapshotable systems
94bb856

Remove unused
14e36f0

Revert gitignore
75c9404

Fix smoke test
e43ced9

Update src/tsc/tsc.ts
0b20495

Jake Bailey (jakebailey) mentioned this pull request Jan 10, 2024
Add warmup benchmarks microsoft/typescript-benchmarking#11
Closed

Daniel Rosenwasser (DanielRosenwasser) mentioned this pull request Feb 22, 2024
TypeScript 5.5 Iteration Plan #57475
Closed

Jake Bailey (jakebailey) added 2 commits March 19, 2024 12:58

Merge branch 'main' into node-snapshotting
69fcecb

reduce any
27c10eb

Jake Bailey (jakebailey) mentioned this pull request Mar 29, 2024
Support for file-system based persistent code cache in user-land module loaders nodejs/node#47472
Closed

Jake Bailey (jakebailey) closed this Jun 3, 2024

Jake Bailey (jakebailey) mentioned this pull request Jun 3, 2024
Ideas for faster cold compiler start-up #25658
Open

Microsoft (microsoft) locked as resolved and limited conversation to collaborators Oct 16, 2025

Uh oh!

Conversation

Jake Bailey (jakebailey) commented Sep 22, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jake Bailey (jakebailey) commented Sep 30, 2023

Uh oh!

Jake Bailey (jakebailey) commented Jan 9, 2024

Uh oh!

TypeScript Bot (typescript-bot) commented Jan 9, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jake Bailey (jakebailey) commented Jan 9, 2024

Uh oh!

TypeScript Bot (typescript-bot) commented Jan 9, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Jan 9, 2024

Uh oh!

TypeScript Bot (typescript-bot) commented Jan 9, 2024

startup

tsc

tsserver

Uh oh!

Jake Bailey (jakebailey) commented Jan 10, 2024

Uh oh!

Jake Bailey (jakebailey) commented Mar 29, 2024

Uh oh!

Jake Bailey (jakebailey) commented Apr 16, 2024

Uh oh!

TypeScript Bot (typescript-bot) commented Apr 16, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Apr 16, 2024

tsc

tsserver

startup

Uh oh!

Jake Bailey (jakebailey) commented Jun 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Jake Bailey (jakebailey) commented Sep 22, 2023•
edited
Loading

TypeScript Bot (typescript-bot) commented Jan 9, 2024•
edited
Loading

TypeScript Bot (typescript-bot) commented Jan 9, 2024•
edited
Loading

TypeScript Bot (typescript-bot) commented Apr 16, 2024•
edited
Loading