Set stackTraceLimit to 0 in fileSystemEntryExists

[Brandon Cheng (gluxon)]

The exception thrown by Node.js's fs.statSync function contains a stack trace that can be expensive to compute. Since this exception isn't used by fileSystemEntryExists, we can safely set Error.stackTraceLimit to 0 without a change in behavior.

A significant performance improvement was noticed with this change while profiling tsserver on packages within a proprietary monorepo. Specifically, my team saw high self time percentages for Node.js's uvException and handleErrorFromBinding internal functions. These functions are executed within fs.statSync when it fails to find the given path.

fs.statSync: https://github.com/nodejs/node/blob/v14.4.0/lib/fs.js#L1030-L1037
handleErrorFromBinding: https://github.com/nodejs/node/blob/v14.4.0/lib/internal/fs/utils.js#L254-L269
uvException: https://github.com/nodejs/node/blob/v14.4.0/lib/internal/errors.js#L390-L443

Measurements

After adding Error.stackTraceLimit = 0, we saw:

• For a large configured project with 12,565 files, tsserver reached the projectLoadingFinish event 48.78% faster. (~46.786s vs ~31.447s)
• For a medium project with 7,064 files, tsserver was 25.75% faster. (~20.897s vs ~16.618s)
• For a small project with 796 files, tsserver was only a negligible 3.00% faster. (~3.545s vs ~3.442)

Measurements were taken on macOS 10.15.6, Node.js 14.4.0, and a recent master commit of TypeScript (610fa28d). The average of 3 runs before and after this change were taken.

I would normally include .cpuprofile and isolate-*-*-*.log files, but can't post them publicly in this case. If there's any other summaries the TypeScript team would be curious about I can report them.

fs.statSync Misses

Within our monorepo, the fs.statSync misses were mostly searches for alternative file extensions of module imports.

• For node_modules imports, a lot of .ts/.tsx lookups failed until the .d.ts file was found.
• Within projects with a lot of JSX files, .ts files were looked for before finding the .tsx version.
• In the medium scale project mentioned above, a total of 38,515 non-existent files were queried during createProgram.

[Brandon Cheng (gluxon)]

Tagging Andrew Casey (@amcasey) due to the relation with PR #36190.

Please verify that:

• There is an associated issue in the Backlog milestone (required)
• Code is up-to-date with the master branch
• You've successfully run gulp runtests locally
• There are new or updated unit tests validating the change

New tests may be a bit difficult with this specific change. Open to any testing suggestions if anyone has them. Also not sure what the associated backlog issue for this should be.

[Sheetal Nandi (sheetalkamat)]

Another option is to just return using existsSync and ignore directory or file and make our code more tolerant (it might already be) for finding file instead of directory or vice versa. That could give us better perf in both cases.

[Brandon Cheng (gluxon)]

Definitely. I was also wondering how necessary it was to check for file versus folder existence.

I'm guessing the current behavior of this function is this way so folders ending with .ts get ignored. Is that an edge case where a behavior change would be acceptable?

[Sheetal Nandi (sheetalkamat)]

If you make a change we can see what user tests break and see what happens

[Brandon Cheng (gluxon)]

Ended up not doing this and favoring Error.stackTraceLimit = 0 instead. I assume we wanted to try using existsSync entirely since we were worried about perf in the path exists case. Setting Error.stackTraceLimit = 0 should improve performance when files don't exist with no change to checking files that do exist.

[Andrew Casey (amcasey)]

Nice!

[Brandon Cheng (gluxon)]

I ended up removing the !fs.existsSync check favoring Error.stackTraceLimit = 0 instead. This is a pattern Node.js core uses as well: https://github.com/nodejs/node/blob/v14.4.0/lib/internal/errors.js#L414-L423 I reran some timings and this solution is comparable to the original in terms of performance gains.

Since the happy path (when the path exists) no longer calls both fs.existsSync and fs.statSync, I also removed the "Other Notes" caveat from the PR description.

[Andrew Casey (amcasey)]

Brandon Cheng (@gluxon), it's precisely the file/dir distinction that prevented me from rolling out existsSync more broadly. It's possible we can add try-catches for the rare cases where the wrong type is found, but it seemed likely to be a pervasive change.

As an alternative, I've proposed a change to node itself: nodejs/node#33716

[Andrew Casey (amcasey)]

Without having re-reviewed the code, I'm also somewhat concerned that the exists/non-exists balance might be different in long-running watch or language service scenarios than it is during batch compilation.

[Brandon Cheng (gluxon)]

As an alternative, I've proposed a change to node itself: nodejs/node#33716

That's a great proposed change. I was quite frustrated that there's no fast way to stat in node. Thanks for opening that.

Without having re-reviewed the code, I'm also somewhat concerned that the exists/non-exists balance might be different in long-running watch or language service scenarios than it is during batch compilation.

I ended up using the Error.stackTraceLimit = 0 trick mentioned in your link and isolated it to this specific hotspot. This change should be insensitive to the exists/non-exists balance.

[Brandon Cheng (gluxon)]

It looks like the node12 and node14 tests timed out cloning this PR. Pushing an amended commit to restart CI.

[Andrew Casey (amcasey)]

Love the tailored mitigation!

[Brandon Cheng (gluxon)]

Love the tailored mitigation!

Awesome. Thanks to you and Sheetal Nandi (@sheetalkamat) for the fast reviews!

[Sheetal Nandi (sheetalkamat)]

TypeScript Bot (@typescript-bot) perf test this

[TypeScript Bot (typescript-bot)]

Heya Sheetal Nandi (@sheetalkamat), I've started to run the perf test suite on this PR at 930b81c. You can monitor the build here.

Update: The results are in!

[TypeScript Bot (typescript-bot)]

Sheetal Nandi (@sheetalkamat)
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..40043

Metric	master	40043	Delta	Best	Worst
Angular - node (v10.16.3, x64)
Memory used	343,662k (± 0.01%)	343,182k (± 0.02%)	-479k (- 0.14%)	343,011k	343,328k
Parse Time	2.01s (± 0.54%)	2.00s (± 0.65%)	-0.01s (- 0.65%)	1.97s	2.04s
Bind Time	0.82s (± 0.60%)	0.82s (± 0.93%)	-0.00s (- 0.00%)	0.80s	0.84s
Check Time	4.79s (± 0.55%)	4.77s (± 0.57%)	-0.02s (- 0.33%)	4.70s	4.85s
Emit Time	5.21s (± 0.87%)	5.17s (± 0.67%)	-0.03s (- 0.63%)	5.08s	5.25s
Total Time	12.82s (± 0.49%)	12.76s (± 0.46%)	-0.06s (- 0.47%)	12.62s	12.87s
Monaco - node (v10.16.3, x64)
Memory used	339,295k (± 0.03%)	339,322k (± 0.04%)	+27k (+ 0.01%)	339,104k	339,536k
Parse Time	1.58s (± 0.82%)	1.55s (± 0.75%)	-0.03s (- 1.78%)	1.51s	1.57s
Bind Time	0.71s (± 0.91%)	0.72s (± 1.25%)	+0.00s (+ 0.42%)	0.70s	0.74s
Check Time	4.93s (± 0.48%)	4.92s (± 0.57%)	-0.01s (- 0.20%)	4.86s	4.99s
Emit Time	2.75s (± 0.78%)	2.75s (± 0.49%)	0.00s ( 0.00%)	2.72s	2.79s
Total Time	9.98s (± 0.39%)	9.94s (± 0.30%)	-0.04s (- 0.39%)	9.89s	10.00s
TFS - node (v10.16.3, x64)
Memory used	302,095k (± 0.09%)	302,034k (± 0.02%)	-61k (- 0.02%)	301,849k	302,144k
Parse Time	1.21s (± 0.84%)	1.21s (± 0.77%)	+0.01s (+ 0.50%)	1.20s	1.23s
Bind Time	0.67s (± 1.05%)	0.66s (± 1.03%)	-0.00s (- 0.75%)	0.64s	0.67s
Check Time	4.41s (± 0.37%)	4.41s (± 0.60%)	+0.00s (+ 0.09%)	4.37s	4.47s
Emit Time	2.90s (± 1.21%)	2.89s (± 0.53%)	-0.01s (- 0.28%)	2.85s	2.92s
Total Time	9.18s (± 0.54%)	9.18s (± 0.31%)	-0.00s (- 0.01%)	9.13s	9.23s
material-ui - node (v10.16.3, x64)
Memory used	458,720k (± 0.02%)	458,427k (± 0.02%)	-293k (- 0.06%)	458,333k	458,708k
Parse Time	2.04s (± 0.37%)	1.96s (± 0.78%)	🟩-0.09s (- 4.21%)	1.94s	2.01s
Bind Time	0.66s (± 1.46%)	0.65s (± 1.02%)	-0.01s (- 1.21%)	0.64s	0.66s
Check Time	12.89s (± 0.66%)	12.87s (± 0.53%)	-0.03s (- 0.20%)	12.73s	12.99s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	15.59s (± 0.58%)	15.47s (± 0.43%)	-0.12s (- 0.78%)	15.33s	15.59s
Angular - node (v12.1.0, x64)
Memory used	320,833k (± 0.03%)	320,456k (± 0.02%)	-377k (- 0.12%)	320,334k	320,653k
Parse Time	1.99s (± 0.62%)	1.97s (± 0.48%)	-0.03s (- 1.35%)	1.95s	1.99s
Bind Time	0.80s (± 0.91%)	0.81s (± 0.61%)	+0.00s (+ 0.50%)	0.80s	0.82s
Check Time	4.64s (± 0.51%)	4.64s (± 0.40%)	+0.00s (+ 0.04%)	4.61s	4.68s
Emit Time	5.33s (± 0.57%)	5.30s (± 0.37%)	-0.03s (- 0.56%)	5.26s	5.34s
Total Time	12.77s (± 0.36%)	12.72s (± 0.30%)	-0.05s (- 0.38%)	12.65s	12.81s
Monaco - node (v12.1.0, x64)
Memory used	321,541k (± 0.02%)	321,532k (± 0.02%)	-10k (- 0.00%)	321,370k	321,679k
Parse Time	1.55s (± 0.81%)	1.53s (± 0.95%)	-0.01s (- 0.71%)	1.51s	1.57s
Bind Time	0.69s (± 0.52%)	0.69s (± 0.75%)	-0.00s (- 0.43%)	0.68s	0.70s
Check Time	4.72s (± 0.54%)	4.69s (± 0.31%)	-0.03s (- 0.66%)	4.66s	4.72s
Emit Time	2.82s (± 1.23%)	2.80s (± 0.58%)	-0.02s (- 0.85%)	2.76s	2.83s
Total Time	9.79s (± 0.61%)	9.71s (± 0.23%)	-0.07s (- 0.75%)	9.67s	9.75s
TFS - node (v12.1.0, x64)
Memory used	286,576k (± 0.02%)	286,531k (± 0.03%)	-45k (- 0.02%)	286,392k	286,734k
Parse Time	1.22s (± 0.53%)	1.23s (± 0.65%)	+0.01s (+ 0.49%)	1.21s	1.24s
Bind Time	0.64s (± 1.04%)	0.64s (± 0.97%)	-0.00s (- 0.62%)	0.63s	0.66s
Check Time	4.32s (± 0.48%)	4.32s (± 0.36%)	-0.01s (- 0.14%)	4.29s	4.36s
Emit Time	2.92s (± 0.83%)	2.92s (± 0.54%)	+0.00s (+ 0.07%)	2.90s	2.97s
Total Time	9.11s (± 0.44%)	9.11s (± 0.27%)	+0.00s (+ 0.03%)	9.05s	9.17s
material-ui - node (v12.1.0, x64)
Memory used	436,816k (± 0.07%)	436,750k (± 0.04%)	-66k (- 0.02%)	436,006k	436,994k
Parse Time	2.02s (± 0.60%)	1.96s (± 0.51%)	-0.06s (- 2.96%)	1.94s	1.98s
Bind Time	0.63s (± 0.94%)	0.62s (± 1.09%)	-0.01s (- 1.58%)	0.61s	0.64s
Check Time	11.69s (± 1.14%)	11.53s (± 0.51%)	-0.15s (- 1.32%)	11.40s	11.70s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	14.34s (± 0.99%)	14.12s (± 0.45%)	-0.23s (- 1.57%)	13.99s	14.29s
Angular - node (v8.9.0, x64)
Memory used	340,175k (± 0.01%)	339,817k (± 0.02%)	-359k (- 0.11%)	339,691k	339,990k
Parse Time	2.54s (± 0.14%)	2.53s (± 0.46%)	-0.02s (- 0.59%)	2.50s	2.55s
Bind Time	0.86s (± 0.80%)	0.85s (± 0.76%)	-0.01s (- 0.93%)	0.84s	0.86s
Check Time	5.39s (± 0.70%)	5.39s (± 0.50%)	-0.00s (- 0.09%)	5.34s	5.46s
Emit Time	5.96s (± 0.63%)	5.82s (± 1.30%)	-0.14s (- 2.38%)	5.67s	5.93s
Total Time	14.76s (± 0.47%)	14.59s (± 0.57%)	-0.17s (- 1.15%)	14.42s	14.77s
Monaco - node (v8.9.0, x64)
Memory used	340,538k (± 0.01%)	340,538k (± 0.02%)	+0k (+ 0.00%)	340,347k	340,656k
Parse Time	1.88s (± 0.50%)	1.87s (± 0.36%)	-0.00s (- 0.16%)	1.86s	1.89s
Bind Time	0.89s (± 0.82%)	0.88s (± 0.67%)	-0.00s (- 0.45%)	0.87s	0.90s
Check Time	5.47s (± 0.65%)	5.44s (± 0.43%)	-0.02s (- 0.44%)	5.40s	5.51s
Emit Time	3.25s (± 1.27%)	3.24s (± 0.86%)	-0.01s (- 0.22%)	3.18s	3.33s
Total Time	11.48s (± 0.70%)	11.44s (± 0.31%)	-0.04s (- 0.32%)	11.37s	11.53s
TFS - node (v8.9.0, x64)
Memory used	303,825k (± 0.01%)	303,819k (± 0.02%)	-6k (- 0.00%)	303,702k	303,982k
Parse Time	1.55s (± 0.66%)	1.54s (± 0.47%)	-0.01s (- 0.58%)	1.53s	1.56s
Bind Time	0.68s (± 0.54%)	0.68s (± 1.01%)	-0.00s (- 0.15%)	0.66s	0.69s
Check Time	5.06s (± 1.50%)	5.10s (± 1.25%)	+0.03s (+ 0.67%)	4.92s	5.23s
Emit Time	3.01s (± 3.32%)	3.01s (± 2.68%)	+0.00s (+ 0.03%)	2.91s	3.21s
Total Time	10.30s (± 0.48%)	10.32s (± 0.44%)	+0.02s (+ 0.22%)	10.24s	10.41s
material-ui - node (v8.9.0, x64)
Memory used	462,986k (± 0.02%)	462,725k (± 0.01%)	-261k (- 0.06%)	462,625k	462,933k
Parse Time	2.40s (± 0.52%)	2.39s (± 0.46%)	-0.01s (- 0.46%)	2.37s	2.42s
Bind Time	0.79s (± 1.04%)	0.78s (± 0.93%)	-0.01s (- 1.51%)	0.77s	0.80s
Check Time	17.13s (± 0.99%)	17.13s (± 1.35%)	-0.00s (- 0.00%)	16.70s	17.63s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	20.33s (± 0.86%)	20.30s (± 1.14%)	-0.03s (- 0.16%)	19.90s	20.79s
Angular - node (v8.9.0, x86)
Memory used	195,186k (± 0.02%)	195,041k (± 0.04%)	-146k (- 0.07%)	194,853k	195,187k
Parse Time	2.46s (± 0.70%)	2.45s (± 0.71%)	-0.02s (- 0.69%)	2.43s	2.50s
Bind Time	0.98s (± 1.19%)	0.98s (± 0.69%)	+0.00s (+ 0.31%)	0.96s	0.99s
Check Time	4.88s (± 0.58%)	4.83s (± 0.53%)	-0.05s (- 1.00%)	4.79s	4.90s
Emit Time	5.91s (± 0.60%)	5.89s (± 0.95%)	-0.02s (- 0.37%)	5.73s	6.01s
Total Time	14.23s (± 0.34%)	14.15s (± 0.59%)	-0.09s (- 0.61%)	13.95s	14.38s
Monaco - node (v8.9.0, x86)
Memory used	193,576k (± 0.02%)	193,564k (± 0.03%)	-12k (- 0.01%)	193,460k	193,726k
Parse Time	1.91s (± 0.78%)	1.91s (± 1.02%)	-0.00s (- 0.21%)	1.87s	1.96s
Bind Time	0.70s (± 0.95%)	0.70s (± 0.95%)	-0.00s (- 0.43%)	0.69s	0.72s
Check Time	5.54s (± 0.62%)	5.56s (± 0.80%)	+0.02s (+ 0.38%)	5.46s	5.66s
Emit Time	2.67s (± 0.69%)	2.67s (± 0.82%)	+0.00s (+ 0.04%)	2.63s	2.72s
Total Time	10.82s (± 0.57%)	10.83s (± 0.48%)	+0.01s (+ 0.14%)	10.75s	10.97s
TFS - node (v8.9.0, x86)
Memory used	173,811k (± 0.04%)	173,787k (± 0.02%)	-24k (- 0.01%)	173,684k	173,867k
Parse Time	1.59s (± 1.13%)	1.59s (± 0.76%)	-0.00s (- 0.13%)	1.57s	1.63s
Bind Time	0.64s (± 0.74%)	0.65s (± 1.12%)	+0.01s (+ 0.78%)	0.64s	0.67s
Check Time	4.66s (± 0.59%)	4.67s (± 0.70%)	+0.01s (+ 0.26%)	4.58s	4.74s
Emit Time	2.80s (± 0.73%)	2.78s (± 1.04%)	-0.02s (- 0.64%)	2.71s	2.85s
Total Time	9.70s (± 0.51%)	9.69s (± 0.57%)	-0.01s (- 0.06%)	9.57s	9.78s
material-ui - node (v8.9.0, x86)
Memory used	262,229k (± 0.02%)	262,096k (± 0.01%)	-133k (- 0.05%)	262,011k	262,175k
Parse Time	2.44s (± 0.51%)	2.44s (± 0.61%)	+0.00s (+ 0.12%)	2.42s	2.47s
Bind Time	0.69s (± 2.62%)	0.68s (± 1.58%)	-0.01s (- 1.59%)	0.66s	0.70s
Check Time	15.66s (± 0.97%)	15.69s (± 0.83%)	+0.03s (+ 0.19%)	15.47s	16.15s
Emit Time	0.00s (± 0.00%)	0.00s (± 0.00%)	0.00s ( NaN%)	0.00s	0.00s
Total Time	18.79s (± 0.88%)	18.81s (± 0.71%)	+0.02s (+ 0.11%)	18.57s	19.29s

System

Machine Name	ts-ci-ubuntu
Platform	linux 4.4.0-166-generic
Architecture	x64
Available Memory	16 GB
Available Memory	1 GB
CPUs	4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

Hosts

• node (v10.16.3, x64)
• node (v12.1.0, x64)
• node (v8.9.0, x64)
• node (v8.9.0, x86)

Scenarios

• Angular - node (v10.16.3, x64)
• Angular - node (v12.1.0, x64)
• Angular - node (v8.9.0, x64)
• Angular - node (v8.9.0, x86)
• Monaco - node (v10.16.3, x64)
• Monaco - node (v12.1.0, x64)
• Monaco - node (v8.9.0, x64)
• Monaco - node (v8.9.0, x86)
• TFS - node (v10.16.3, x64)
• TFS - node (v12.1.0, x64)
• TFS - node (v8.9.0, x64)
• TFS - node (v8.9.0, x86)
• material-ui - node (v10.16.3, x64)
• material-ui - node (v12.1.0, x64)
• material-ui - node (v8.9.0, x64)
• material-ui - node (v8.9.0, x86)

Benchmark	Name	Iterations
Current	40043	10
Baseline	master	10

[Eric Jeney (ericjeney)]

Would it be possible to backport this to the 4.0 branch as well, once merged? My team is eager to get this performance improvement as soon as possible.

[Andrew Casey (amcasey)]

Eric Jeney (@ericjeney) What kind of improvement are you seeing? When I was investigating (before this PR), it looked like it was in the 5% range, which is certainly very nice, but is not hold-the-release. Having data from real customers would make it easier to argue for a late change.

[Eric Jeney (ericjeney)]

Brandon Cheng (@gluxon) and I are on the same team, so my results are similar to his. We work on a code base of ~1,500,000 lines of TypeScript spread across ~500 packages.

In addition to the benchmarks he noted above, I tested on a patched version of tsserver.js in VS Code and timed the "Initializing JS/TS Language Features..." prompt after selecting the Restart TS Server action. I noted a ~20% improvement for several packages, going from a consistent 38s => 28-30s for one of them.

If not in 4.0, is this the sort of change you would ship in a patch release? Or are patch releases reserved for regressions?

[Andrew Casey (amcasey)]

Eric Jeney (@ericjeney) We'll have to discuss the potential risks, but those numbers are pretty exciting. I'll post more when I know more.

[Eric Jeney (ericjeney)]

Thank you!

I just confirmed with another coworker as well, who works on a separate monorepo, and saw a >25% improvement on one of their packages (17.5s => 13s), so the improvement seems to not be isolated to only the codebase that I work on.

[Andrew Casey (amcasey)]

Eric Jeney (@ericjeney) If things look good in nightly builds, this will be a strong candidate for inclusion in a patch release. No promises though.

[Brandon Cheng (gluxon)]

Thanks for the update Andrew Casey (@amcasey)! Is there anything I still need to do on our side before this can merge?

[Wesley Wigham (weswigham)]

If it matters this much, TBH, we should consider doing this for all the falliable FS operations in sys.ts that we just eat the error on - realpath, getModifiedTime/setModifiedTime, deleteFile, createDirectory, getFileSize, writeFile, getAccessibleFileSystemEntries, fileSystemEntryExists, and watchPresentFileSystemEntry (all of them combined will probably also represent measurable savings under --build or --watch)

Simultaneously, a gripe: Why isn't Error.prototype.stack a lazier getter 😦 I feel like this could also be "fixed" in node.

[Andrew Casey (amcasey)]

Brandon Cheng (@gluxon), nope, just wanted to hear from Sheetal Nandi (@sheetalkamat). 😄 Thanks again!

[Eric Anderson (ericanderson)]

Thanks Andrew Casey (@amcasey) / Wesley Wigham (@weswigham) / Sheetal Nandi (@sheetalkamat) !

[Andrew Casey (amcasey)]

TypeScript Bot (@typescript-bot) cherry-pick this to release-4.0

[TypeScript Bot (typescript-bot)]

Heya Andrew Casey (@amcasey), I've started to run the task to cherry-pick this into release-4.0 on this PR at 930b81c. You can monitor the build here.

[TypeScript Bot (typescript-bot)]

Hey Andrew Casey (@amcasey), I couldn't open a PR with the cherry-pick. (You can check the log here). You may need to squash and pick this PR into release-4.0 manually.