Uh oh!
There was an error while loading. Please reload this page.
perf(queriesObserver): fix O(n²) performance issue in batch updates - #9467
Closed
joseph0926 wants to merge 1 commit into
Closed
perf(queriesObserver): fix O(n²) performance issue in batch updates#9467joseph0926 wants to merge 1 commit into
joseph0926 wants to merge 1 commit into
Conversation
View your CI Pipeline Execution ↗ for commit 5f51b4a
☁️ Nx Cloud last updated this comment at |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@## main #9467 +/- ##
===========================================
+ Coverage 45.30% 59.60% +14.29%
===========================================
Files 208 137 -71 Lines 8283 5525 -2758 Branches 1869 1477 -392 ===========================================
- Hits 3753 3293 -460 + Misses 4085 1930 -2155 + Partials 445 302 -143 🚀 New features to boost your workflow:
|
TkDodo
reviewed
Jul 31, 2025
Comment on lines
-255
to
+262
| const index = this.#observers.indexOf(observer) | ||
| if (index !== -1) { | ||
| const index = this.#indexMap.get(observer) | ||
| if (index !== undefined) { |
Collaborator
There was a problem hiding this comment.
why don’t we just pass the index into onUpdate instead of passing the whole observer and then trying to find the index again? For example:
diff --git a/packages/query-core/src/queriesObserver.ts b/packages/query-core/src/queriesObserver.ts
index 853e490ab..84defa874 100644
--- a/packages/query-core/src/queriesObserver.ts+++ b/packages/query-core/src/queriesObserver.ts@@ -63,9 +63,9 @@ export class QueriesObserver<
protected onSubscribe(): void {
if (this.listeners.size === 1) {
- this.#observers.forEach((observer) => {+ this.#observers.forEach((observer, index) => {
observer.subscribe((result) => {
- this.#onUpdate(observer, result)+ this.#onUpdate(index, result)
})
})
}
@@ -137,9 +137,9 @@ export class QueriesObserver<
observer.destroy()
})
- difference(newObservers, prevObservers).forEach((observer) => {+ difference(newObservers, prevObservers).forEach((observer, index) => {
observer.subscribe((result) => {
- this.#onUpdate(observer, result)+ this.#onUpdate(index, result)
})
})
@@ -251,12 +251,9 @@ export class QueriesObserver<
return observers
}
- #onUpdate(observer: QueryObserver, result: QueryObserverResult): void {- const index = this.#observers.indexOf(observer)- if (index !== -1) {- this.#result = replaceAt(this.#result, index, result)- this.#notify()- }+ #onUpdate(index: number, result: QueryObserverResult): void {+ this.#result = replaceAt(this.#result, index, result)+ this.#notify()
}
#notify(): void {ContributorAuthor
There was a problem hiding this comment.
Thank you for the suggestion
I considered passing the index directly, but noticed these potential issues
When setQueries() reorders observers, existing subscriptions keep their old captured indices
// Initial: observers = [A, B, C]// A subscribes with index=0 in onSubscribe()// After setQueries(): observers = [D, A, B, C] // A still uses index=0, but should use index=1The index from difference().forEach() is not the actual position in this.#observers
difference(newObservers,prevObservers).forEach((observer,index)=>{// If difference returns [D, E], index is 0,1// But their actual positions in observers might be [A, D, B, E, C] -> 1,3observer.subscribe((result)=>{this.#onUpdate(index,result)// <- I think there is a problem here.})})What do you think about these concerns?
Collaborator
There was a problem hiding this comment.
Right, can you then please add a test case where the implementation I suggested would fail.
ContributorAuthor
There was a problem hiding this comment.
import{afterEach,beforeEach,describe,expect,test,vi}from'vitest'import{queryKey}from'@tanstack/query-test-utils'import{QueriesObserver,QueryClient}from'..'importtype{QueryObserverResult}from'..'describe('queriesObserver - index tracking issue',()=>{letqueryClient: QueryClientbeforeEach(()=>{vi.useFakeTimers()queryClient=newQueryClient()queryClient.mount()})afterEach(()=>{queryClient.clear()vi.useRealTimers()})test('should fail when using forEach index with dynamic query changes',async()=>{constkey1=queryKey()constkey2=queryKey()constkey3=queryKey()constkey4=queryKey()constqueryFn1=vi.fn().mockReturnValue('data1')constqueryFn2=vi.fn().mockReturnValue('data2')constqueryFn3=vi.fn().mockReturnValue('data3')constqueryFn4=vi.fn().mockReturnValue('data4')constobserver=newQueriesObserver(queryClient,[{queryKey: key1,queryFn: queryFn1},{queryKey: key2,queryFn: queryFn2},])constresults: Array<Array<QueryObserverResult>>=[]constunsubscribe=observer.subscribe((result)=>{results.push([...result])})awaitvi.advanceTimersByTimeAsync(0)results.length=0observer.setQueries([{queryKey: key3,queryFn: queryFn3},{queryKey: key1,queryFn: queryFn1},{queryKey: key4,queryFn: queryFn4},{queryKey: key2,queryFn: queryFn2},])awaitvi.advanceTimersByTimeAsync(0)queryClient.setQueryData(key3,'updated3')queryClient.setQueryData(key4,'updated4')queryClient.setQueryData(key1,'updated1')unsubscribe()constfinalResult=results[results.length-1]expect(finalResult).toHaveLength(4)expect(finalResult?.[0]).toMatchObject({data: 'updated3'})expect(finalResult?.[1]).toMatchObject({data: 'updated1'})expect(finalResult?.[2]).toMatchObject({data: 'updated4'})expect(finalResult?.[3]).toMatchObject({data: 'data2'})})test('should fail when reordering queries with existing subscriptions',async()=>{constkey1=queryKey()constkey2=queryKey()constkey3=queryKey()letupdateCount=0constqueryFn1=vi.fn().mockImplementation(()=>`data1-${++updateCount}`)constqueryFn2=vi.fn().mockImplementation(()=>`data2-${++updateCount}`)constqueryFn3=vi.fn().mockImplementation(()=>`data3-${++updateCount}`)constobserver=newQueriesObserver(queryClient,[{queryKey: key1,queryFn: queryFn1},{queryKey: key2,queryFn: queryFn2},{queryKey: key3,queryFn: queryFn3},])constresults: Array<Array<QueryObserverResult>>=[]constunsubscribe=observer.subscribe((result)=>{results.push([...result])})awaitvi.advanceTimersByTimeAsync(0)results.length=0observer.setQueries([{queryKey: key3,queryFn: queryFn3},{queryKey: key1,queryFn: queryFn1},{queryKey: key2,queryFn: queryFn2},])awaitqueryClient.invalidateQueries({queryKey: key1})awaitvi.advanceTimersByTimeAsync(0)constresultAfterInvalidate=results[results.length-1]expect(resultAfterInvalidate?.[1]?.data).toMatch(/data1-\d+/)expect(resultAfterInvalidate?.[0]?.data).toBe('data3-3')expect(resultAfterInvalidate?.[2]?.data).toBe('data2-2')unsubscribe()})})
the failure only occurs with the version that captures the index in the subscription callback. When I run the same tests against either the current main branch’s queriesObserver.ts or the queriesObserver.ts in my PR, all of the tests pass without any issues.
Collaborator
There was a problem hiding this comment.
I believe you, but then please add the test case to the PR.
ContributorAuthor
There was a problem hiding this comment.
Following your suggestion, I've created and run various performance tests, and I'd like to share the results. While theoretically replacing indexOf O(n) with WeakMap O(1) should improve performance, the actual test results were different from what I expected.
- Even in extreme cases with 10,000 queries, there was no meaningful performance difference
- In some cases, the WeakMap version was actually slightly slower
It seems that other optimizations like using Set in the difference function and caching in findMatchingObservers have already resolved the main O(n²) issues.
I apologize for taking up your valuable time reviewing a PR that ultimately doesn't provide substantial improvements. I would appreciate it if you could close the PR.
This has been a great learning experience, and I'm grateful for your time and feedback. Thank you for the opportunity to contribute to the project.
ContributorAuthor
There was a problem hiding this comment.
For reference, I am sharing one of the codes I tested.
import{afterEach,beforeEach,describe,expect,test}from'vitest'import{QueriesObserver,QueryClient}from'..'importtype{QueryObserverOptions}from'..'describe('QueriesObserver extreme cases - Real-time Dashboard',()=>{letqueryClient: QueryClientbeforeEach(()=>{queryClient=newQueryClient({defaultOptions: {queries: {retry: false,},},})queryClient.mount()})afterEach(()=>{queryClient.clear()})test('should handle real-time dashboard with 10,000+ queries where last ones update frequently',async()=>{constTOTAL_QUERIES=10000constACTIVE_QUERIES=100constUPDATE_ROUNDS=10console.log(`\n Real-time Dashboard Simulation:`)console.log(`Total queries: ${TOTAL_QUERIES}`)console.log(`Active queries (frequently updating): ${ACTIVE_QUERIES}`)console.log(`Update rounds: ${UPDATE_ROUNDS}\n`)constcreateQueries=(round: number=0): Array<QueryObserverOptions>=>{returnArray.from({length: TOTAL_QUERIES},(_,i)=>{constisActive=i>=TOTAL_QUERIES-ACTIVE_QUERIESreturn{queryKey: ['dashboard',i,round],queryFn: async()=>{awaitnewPromise((resolve)=>setTimeout(resolve,0))return{id: i,data: `${round===0 ? 'initial' : `update-${round}`}-${i}`,timestamp: Date.now(),
isActive,}},staleTime: isActive ? 0 : Infinity,}})}conststartSetup=performance.now()constobserver=newQueriesObserver(queryClient,createQueries(0))lettotalUpdates=0constupdateTimes: Array<number>=[]letlastUpdateTime=performance.now()constupdateMetrics: Array<{round: numberupdateCount: numberduration: numberavgTimePerUpdate: number}>=[]constunsubscribe=observer.subscribe(()=>{constnow=performance.now()updateTimes.push(now-lastUpdateTime)lastUpdateTime=nowtotalUpdates++})constsetupTime=performance.now()-startSetupconsole.log(`Setup time: ${setupTime.toFixed(1)}ms`)console.log('\n Initial load...')constinitialLoadStart=performance.now()awaitnewPromise((resolve)=>setTimeout(resolve,100))constinitialLoadTime=performance.now()-initialLoadStartconstinitialUpdateCount=totalUpdatesconsole.log(`Initial load completed: ${initialLoadTime.toFixed(1)}ms, ${initialUpdateCount} updates`,)console.log('\n Starting frequent updates on active queries...')for(letround=1;round<=UPDATE_ROUNDS;round++){constroundStart=performance.now()constupdateCountBefore=totalUpdatesconstupdatedQueries=Array.from({length: TOTAL_QUERIES},(_,i)=>{constisActive=i>=TOTAL_QUERIES-ACTIVE_QUERIESif(isActive){return{queryKey: ['dashboard',i,round],queryFn: async()=>{awaitnewPromise((resolve)=>setTimeout(resolve,0))return{id: i,data: `update-${round}-${i}`,timestamp: Date.now(),isActive: true,}},staleTime: 0,}}else{return{queryKey: ['dashboard',i,0],queryFn: async()=>{return{id: i,data: `initial-${i}`,timestamp: Date.now(),isActive: false,}},staleTime: Infinity,}}})observer.setQueries(updatedQueries)awaitnewPromise((resolve)=>setTimeout(resolve,50))constroundDuration=performance.now()-roundStartconstroundUpdates=totalUpdates-updateCountBeforeupdateMetrics.push({
round,updateCount: roundUpdates,duration: roundDuration,avgTimePerUpdate: roundDuration/ACTIVE_QUERIES,})}console.log('\n Performance Analysis:')console.log('Round | Duration | Updates | Avg/Update')console.log('------|----------|---------|------------')updateMetrics.forEach(({ round, duration, updateCount, avgTimePerUpdate })=>{console.log(`${round.toString().padEnd(5)} | ${duration.toFixed(1).padEnd(8)}ms | ${updateCount.toString().padEnd(7)} | ${avgTimePerUpdate.toFixed(2)}ms`,)},)constvalidMetrics=updateMetrics.filter((m)=>m.updateCount>0)if(validMetrics.length>=2){constfirstRoundAvg=validMetrics[0].avgTimePerUpdateconstlastRoundAvg=validMetrics[validMetrics.length-1].avgTimePerUpdateconstdegradation=lastRoundAvg/firstRoundAvgconsole.log(`\n Performance degradation: ${degradation.toFixed(2)}x`)console.log(`Total updates processed: ${totalUpdates}`)if(updateTimes.length>100){constsortedTimes=[...updateTimes].sort((a,b)=>a-b)constp50=sortedTimes[Math.floor(sortedTimes.length*0.5)]constp95=sortedTimes[Math.floor(sortedTimes.length*0.95)]constp99=sortedTimes[Math.floor(sortedTimes.length*0.99)]console.log('\n Update time distribution:')console.log(`P50: ${p50?.toFixed(2)}ms`)console.log(`P95: ${p95?.toFixed(2)}ms`)console.log(`P99: ${p99?.toFixed(2)}ms`)}expect(degradation).toBeLessThan(3.0)}unsubscribe()})test('should demonstrate O(n) vs O(n²) behavior with increasing observer counts',async()=>{consttestSizes=[1000,2000,4000,8000]constresults: Array<{size: numbersetupTime: numberupdateTime: numbertimePerQuery: number}>=[]console.log('\n Scalability Analysis:')for(constsizeoftestSizes){constactiveCount=Math.min(100,size/10)constqueries=Array.from({length: size},(_,i)=>({queryKey: ['scale-test',size,i],queryFn: async()=>({id: i,data: `result-${i}`}),staleTime: i>=size-activeCount ? 0 : Infinity,}))constsetupStart=performance.now()constobserver=newQueriesObserver(queryClient,queries)letupdateCount=0constunsubscribe=observer.subscribe(()=>{updateCount++})constsetupTime=performance.now()-setupStartawaitnewPromise((resolve)=>setTimeout(resolve,50))constupdateStart=performance.now()constupdatedQueries=queries.map((q,i)=>{if(i>=size-activeCount){return{
...q,queryKey: ['scale-test',size,i,'updated'],}}returnq})observer.setQueries(updatedQueries)awaitnewPromise((resolve)=>setTimeout(resolve,50))constupdateTime=performance.now()-updateStartresults.push({
size,
setupTime,
updateTime,timePerQuery: updateTime/activeCount,})unsubscribe()awaitqueryClient.clear()}console.log('Size | Setup Time | Update Time | Time/Query | Growth')console.log('------|------------|-------------|------------|--------')results.forEach((result,i)=>{constgrowth=i>0
? (result.timePerQuery/results[0].timePerQuery).toFixed(2)
: '1.00'console.log(`${result.size.toString().padEnd(5)} | ${result.setupTime.toFixed(1).padEnd(10)}ms | ${result.updateTime.toFixed(1).padEnd(11)}ms | ${result.timePerQuery.toFixed(2).padEnd(10)}ms | ${growth}x`,)})consttimePerQueryRatios=results.map((r)=>r.timePerQuery)constmaxRatio=Math.max(...timePerQueryRatios)constminRatio=Math.min(...timePerQueryRatios)constvarianceRatio=maxRatio/minRatioconsole.log(`\nTime per query variance: ${varianceRatio.toFixed(2)}x (should be close to 1.0 for O(n))`,)expect(varianceRatio).toBeLessThan(3.0)})})Collaborator
There was a problem hiding this comment.
alright, let’s close this then. Thanks for working on this 🙏
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is an additional workaround attempt for an issue I've been challenged to solve before.
Previous issue
First Resolution Attempt PR
Background
This PR addresses a long-standing performance issue first reported in #8295, where using
useQuerieswith batch data fetching patterns (like DataLoader) causes severe performance degradation due to O(n²) complexity.The Problem
When multiple queries resolve simultaneously (common with DataLoader pattern), the performance degrades quadratically:
The issue occurs because:
extra notes
i found two remaining pain‑points:
If 100 observers change only 1 member, we still perform
100 × 100 = 10 000comparisons.Missing early‑return / direct access
Concrete mental model
Real-world impact
Previous Improvements
Over the past months, several optimizations have been made:
1. Caching
findMatchingObserversresults (#8304)I previously added
observerMatchesto cache results instead of recalculating:2. Optimizing
differencefunction (O(n²) → O(n))3. Optimizing
findMatchingObserverswith MapThe Remaining Issue
Despite these improvements, one critical O(n) operation remains in
#onUpdate:With batch updates, this creates:
This PR's Solution
This PR introduces a
WeakMapto track observer indices, eliminating the O(n) search:Why WeakMap?
I chose
WeakMapover regularMapto address the concern raised in #8686 about storing observers in both a Map and an Array:WeakMapsolves this elegantly:This approach maintains the simplicity of the original design while achieving O(1) performance.
Performance Results
With this change, batch updates now scale linearly:
Summary
This completes the optimization journey for
QueriesObserver:findMatchingObserverscaching (my previous PR)differencefunction: O(n²) → O(n)findMatchingObservers: O(n) → O(1) with Map#onUpdate: O(n) → O(1) with WeakMap (this PR)The combination of these improvements transforms what was once a quadratic bottleneck into efficient linear scaling, making
useQueriesviable for large-scale batch operations.