Share stackTable, frameTable, funcTable, resourceTable and nativeSymbols between threads by mstange · Pull Request #5482 · firefox-devtools/profiler

mstange · 2025-06-04T23:34:21Z

Production | Deploy preview

Sharing the string table happened in #5481.
This PR makes us share the rest, and completes issue #3918.

Only samples and markers are still per-thread. I don't expect this to change.
This includes allocation samples.

To do:

Fix type check and test failures
Upgrade the URL when upgrading the profile, to remap function indexes etc. in the URL
When compacting during upload, apply translation tables to redux state (e.g. update focus function indexes in URL)
Do something to address perf impact for computing the derived thread (transforms and call node table computation happen per thread and take longer if the tables are bigger because they now contain information for all threads, should be able to reuse more of the computed outputs in the thread derivation pipeline)
Memoize the computation of the implementation filtered thread across threads
Adjust indexes in transforms when merging two different profiles
Add a paragraph to CHANGELOG-formats.md for the new version.
Add a test for merging profiles with applied transforms where we actually check the filtered call tree

Potential follow-ups:

Add a test checking that IndexIntoSourceTable indexes in the redux state are updated after profile sanitizing
Add a test checking that collapse-resource transforms aren't broken if profile sanitizing adds elements to the funcTable

vlasakm · 2025-08-10T16:16:25Z

This is really great! Sharing this data across threads works especially well for applications where there is a large number of threads, but just a few "thread pools" whose threads are all executing the same code.

Using this allowed me to employ better caching when creating a firefox profile, save on the profile size, etc.

It also made me greedy, and I attempted to visualize a rather large profile (1602 threads, 577108 stacks, 29760 frames). Previously my experience was that Firefox Profiler would choke on say more than 300+ threads, and I hoped this change with sharing could improve the situation. Now it loads, and the UI is still quite responsive, but the load of the profile is taking long 2 minutes 15 seconds (and Firefox prompts to kill the script, etc.).

90 % of the load time is spent is in computeCallNodeTable, which I believe may be called repeatedly for each thread (and the merged thread which includes all of them), even though all these calls would process the same stack table and frame table, now that they are shared.

I know very little about React/Redux, but I see that there's some concept of memoization and thread selectors, that is still used for call nodes, even though after this change to share the stack and frame table. Very naively, would it perhaps be possible to just compute and memoize the call node information once by not using the thread selector mechanism here? But there's a great chance I am missing all the needed context.

FWIW I was and am still using this branch for quite some time. I am not sure if I am able to contribute much (I even failed to rebase it on top of more recent changes), but it would be great to see this, perhaps after all the big work-in-progress changes land (like the Typescript migration).

mstange · 2025-08-10T16:35:58Z

Yes indeed, sharing the computed call node table across threads while properly handling per thread transforms is the next thing I want to work on when I get back to this task. Thanks for the encouragement and it's great to hear that you're seeing real improvements from it!

codecov · 2026-01-29T23:27:20Z

Codecov Report

❌ Patch coverage is 85.69092% with 175 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.36%. Comparing base (19aa5e9) to head (850e4ff).
⚠️ Report is 11 commits behind head on main.

Files with missing lines	Patch %	Lines
src/app-logic/url-handling.ts	12.50%	63 Missing ⚠️
src/profile-logic/transforms.ts	29.41%	48 Missing ⚠️
src/profile-logic/profile-compacting.ts	87.96%	25 Missing and 1 partial ⚠️
src/profile-logic/sanitize.ts	88.60%	9 Missing ⚠️
src/reducers/code.ts	50.00%	6 Missing ⚠️
src/profile-logic/merge-compare.ts	96.66%	5 Missing ⚠️
src/profile-logic/profile-data.ts	94.73%	5 Missing ⚠️
src/reducers/url-state.ts	80.00%	5 Missing ⚠️
src/profile-logic/processed-profile-versioning.ts	97.87%	3 Missing ⚠️
src/profile-logic/index-translation.ts	90.47%	2 Missing ⚠️
... and 2 more

Additional details and impacted files

@@ Coverage Diff @@## main #5482 +/- ##
==========================================
- Coverage 85.56% 85.36% -0.20% 
==========================================
Files 319 320 +1 Lines 31499 31916 +417 Branches 8697 8780 +83 ==========================================
+ Hits 26953 27246 +293 - Misses 4115 4239 +124 
Partials 431 431

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Small things I noticed while working on #5482.

mstange · 2026-02-02T20:00:08Z

This is now ready for review.

More small stuff that makes #5482 a tiny bit easier.

canova

I promised myself that I would be done with reviewing this PR today, but I underestimated the amount of changes here 😅 I'm done-ish with the first commit, which is the main beast. So pushing the review comments now even though I still need to review the others.

canova

I'm done reviewing the rest of the PR! (I saw the github's unicorn error message quite a few times while doing so 😄) It was a lot easier to review the other commits. I think it looks good to me with the issues I raised yesterday fixed. I would prefer to take a look at the PR once more once these are fixed though. But I assume that review will be a lot more quicker.

canova · 2026-02-27T12:09:09Z

Ah another question, I see that there are 2 items unchecked in the list. Are they done? I think we can also do those as follow-ups as they don't look critical.

mstange · 2026-02-28T18:06:25Z

Ah another question, I see that there are 2 items unchecked in the list. Are they done? I think we can also do those as follow-ups as they don't look critical.

Ah yes, I took a brief look at this and it didn't look straightforward to add those tests, so I'll leave those two items for potential follow-ups. Edited the PR description.

canova

Thanks, looks great! Let's land it before it gets more bitrotted 😄

…Symbols across threads. This makes func indexes etc valid globally instead of scoped per threads. It also allows for smaller profiles because data doesn't have to be duplicated. Symbolication is probably faster. Selecting multiple threads, i.e. displaying a "merged thread", is now easier because we no longer need to compute merged funcTables etc for the merged thread; the global tables are already "merged". This also also means that func index mapping during symbolication now works even if multiple threads are selected. Fixesfirefox-devtools#5703. This profile version comes both with a profile upgrader and with a URL upgrader, because we have to adjust any func and resource indexes for transforms in the URL.

We also need to adjust any indexes in the redux state after compacting, for example funcTable or resourceTable indexes used by transforms. This patch also makes us adjust the source index in the URL after compaction. Strictly speaking this fixes an existing bug, because the source table was already being compacted. It also fixes the bug where we weren't keeping track of the oldFuncCount properly when splitting functions into private-window and non-private-window instances, and weren't updating collapse-resource transforms correctly. Fixesfirefox-devtools#5793.

We compute the call node table based on the filtered thread, per thread. But now that every thread contains the full stackTable with entries used by all threads, this computation has become much more expensive. But most of the time we are passing the same tables from the shared profile info, so we can just do the work once and share it across threads. This only fixes the case where none of the threads have a transform applied. They'll end up passing the tables from the shared data and hit the memoization cache. If a thread has a transform, it'll pass different tables and not hit the cache. We don't do anything to share work across two threads that happen to have the same transforms. This also doesn't fix the case where there's an implementation filter applied. Here's how loading the profile https://share.firefox.dev/4bxDqu8 compares: - Before sharing the tables: https://share.firefox.dev/4r0imRO (1.1 seconds) - After sharing the tables, no global memoization: https://share.firefox.dev/4qNGp6m (1.8 seconds) - After sharing the tables, with global memoization: https://share.firefox.dev/4rwkwso (1.3 seconds)

…threads. Here's how loading the profile https://share.firefox.dev/4kbkjrU compares: - Before sharing the tables: https://share.firefox.dev/4kh5i84 (1.1 seconds) - After sharing the tables, no global memoization: https://share.firefox.dev/3NLn1IK (1.6 seconds) - After sharing the tables, with global memoization: https://share.firefox.dev/3LNNuot (1.3 seconds)

Here's how loading the profile https://share.firefox.dev/4a0b88Y compares: - Before sharing the tables: https://share.firefox.dev/4qeiVGu (1.1 seconds) - After sharing the tables, no global memoization: https://share.firefox.dev/3NUEi27 (2.0 seconds) - After sharing the tables, with global memoization: https://share.firefox.dev/4qcDQty (1.3 seconds)

mstange · 2026-03-02T16:02:40Z

Thanks for the review!

Changes: [fatadel] Fix crash when nativeSymbol index is out of bounds in assembly view (#5850) [depfu[bot]] Update all Yarn dependencies (2026-02-25) (#5859) [Nazım Can Altınova] Fix the color of dark mode back arrow svg (#5863) [fatadel] Force canvas redraw when system theme changes (#5861) [Nazım Can Altınova] Fix unhandled promise rejection in setupInitialUrlState (#5864) [fatadel] Persist selected marker in URL and show sticky tooltip on load (#5847) [Markus Stange] Implement the "collapse resource" transform with the help of the "collapse direct recursion" transform. (#5824) [Markus Stange] Bump rollup from 2.79.2 to 2.80.0 (#5868) [Markus Stange] Remove async attribute from module script tag. (#5870) [Nazım Can Altınova] Update the docsify package that's used in the user documentation (#5872) [Markus Stange] Share stackTable, frameTable, funcTable, resourceTable and nativeSymbols between threads (#5482) [Nazım Can Altınova] Escape CSS URLs that are coming from profiles (#5874) [fatadel] Update home page message for the other browser case (#5866) [fatadel] Add support for ternaries in marker labels (#5857) [Markus Stange] Reduce allocations for getStackLineInfo + getStackAddressInfo (#5761) And special thanks to our localizers: de: Ger fy-NL: Fjoerfoks it: Francesco Lodolo [:flod] nl: Fjoerfoks ru: berry ru: Valery Ledovskoy zh-TW: Pin-guang Chen

mstange force-pushed the share-everything branch from 6f1959d to 4e19f9eCompare January 8, 2026 19:26

mstange force-pushed the share-everything branch 4 times, most recently from 24a4604 to 8d957f0Compare January 29, 2026 23:25

mstange force-pushed the share-everything branch 4 times, most recently from a5b1248 to 4b18e81Compare January 30, 2026 18:40

This was referenced Jan 30, 2026
Two small test fixes #5801
Merged
Some fixes to the profile merging code #5802
Merged

mstange force-pushed the share-everything branch from 4b18e81 to ff1d089Compare January 30, 2026 22:43

canova added a commit that referenced this pull request Feb 2, 2026
Two small test fixes (#5801)
45bfe68
Small things I noticed while working on #5482.

mstange force-pushed the share-everything branch from ff1d089 to 76f35faCompare February 2, 2026 19:59

mstange marked this pull request as ready for review February 2, 2026 20:00

mstange force-pushed the share-everything branch 2 times, most recently from ce546d0 to 0d1cc61Compare February 2, 2026 21:18

mstange added a commit that referenced this pull request Feb 3, 2026
Some fixes to the profile merging code (#5802)
c08fe8a
More small stuff that makes #5482 a tiny bit easier.

mstange requested a review from canova February 3, 2026 15:35

mstange force-pushed the share-everything branch 3 times, most recently from d1916d3 to cf780f4Compare February 3, 2026 17:04

canova reviewed Feb 26, 2026
View reviewed changes

canova reviewed Feb 27, 2026
View reviewed changes

mstange force-pushed the share-everything branch from cf780f4 to 52e7600Compare February 28, 2026 18:00

mstange force-pushed the share-everything branch 2 times, most recently from 31327b6 to 02779d1Compare March 2, 2026 03:21

canova approved these changes Mar 2, 2026
View reviewed changes

mstangeand others added 9 commits March 2, 2026 11:02

Update snapshots.
fc4b1a6

Use Int32Array and oldXToNewXPlusOne convention in merge-compare.ts.
aa28987

Correctly adjust transforms when merging profiles.
a938f3b

Reserve functions for collapsed resources globally.
fd648e6

mstange force-pushed the share-everything branch from 02779d1 to 850e4ffCompare March 2, 2026 16:02

mstange enabled auto-merge March 2, 2026 16:02

mstange merged commit 78d0ebb into firefox-devtools:mainMar 2, 2026
19 checks passed

canova mentioned this pull request Mar 3, 2026
Deploy March 3, 2026 #5877
Merged

mstange mentioned this pull request May 20, 2026
dhat importer no longer works #6035
Closed

Uh oh!

Conversation

mstange commented Jun 4, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vlasakm commented Aug 10, 2025

Uh oh!

mstange commented Aug 10, 2025

Uh oh!

codecovBot commented Jan 29, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mstange commented Feb 2, 2026

Uh oh!

canova left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

canova left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

canova commented Feb 27, 2026

Uh oh!

mstange commented Feb 28, 2026

Uh oh!

canova left a comment

Choose a reason for hiding this comment

Uh oh!

mstange commented Mar 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mstange commented Jun 4, 2025•
edited
Loading

codecovBot commented Jan 29, 2026•
edited
Loading

canova left a comment •
edited
Loading