Skip to content

feat: implement useLiveInfiniteQuery hook for React - #666

Merged
KyleAMathews merged 3 commits into
samwillis/load-more-trackingfrom
use-live-infinite-query
Oct 15, 2025
Merged

feat: implement useLiveInfiniteQuery hook for React#666
KyleAMathews merged 3 commits into
samwillis/load-more-trackingfrom
use-live-infinite-query

Conversation

@KyleAMathews

@KyleAMathewsKyleAMathews commented Oct 9, 2025

Copy link
Copy Markdown
Collaborator

stacked on #669

Summary

Implements useLiveInfiniteQuery hook for React, providing infinite scrolling/pagination with live updates and async loading state tracking.

This PR adds a new useLiveInfiniteQuery hook that combines TanStack Query's infinite query pattern with TanStack DB's live query reactivity. The hook provides a familiar API for pagination while automatically updating pages as underlying data changes, with full async loading state management.

What's Included

  • New useLiveInfiniteQuery hook with TanStack Query-compatible API

    • pages: Array of page arrays
    • data: Flattened array of all loaded items
    • pageParams: Array of page parameters used
    • fetchNextPage(): Load the next page
    • hasNextPage: Boolean indicating if more pages are available
    • isFetchingNextPage: Boolean tracking async loading state from setWindow
    • All useLiveQuery properties (status, isLoading, isReady, collection, etc.)
  • Automatic live updates: Pages update reactively when underlying data changes

  • Async loading state tracking: Properly tracks when setWindow is loading additional data

  • Type-safe utils checking: Custom type guard for LiveQueryCollectionUtils using TypeScript's is keyword

  • Dependency tracking: Resets pagination when query dependencies change

  • Intelligent next page detection: Automatically detects when new data becomes available

  • Concurrent fetch prevention: Prevents multiple simultaneous page loads

  • Comprehensive test suite: 17 tests covering pagination, live updates, async loading, sync detection, and edge cases

Implementation Details

Efficient Window Management:

  • Uses liveQueryCollection.utils.setWindow() to dynamically adjust pagination window
  • No query recreation on page changes - just window adjustments
  • Peek-ahead mechanism: Fetches N pages + 1 item to detect if more pages exist
  • Works with ordered queries (requires .orderBy())

Async Loading State Management:

  • setWindow returns true when data is immediately available (synchronous)
  • setWindow returns Promise<void> when loading additional data (asynchronous)
  • isFetchingNextPage state tracks promise resolution
  • Prevents concurrent page fetches while loading is in progress

Live Reactivity:

  • Automatically detects when new items are synced to the collection
  • Updates hasNextPage when data availability changes
  • Pages re-render with updated data without manual intervention

Type Safety:

  • Custom isLiveQueryCollectionUtils type guard ensures proper typing
  • Uses TypeScript's is predicate for accurate type narrowing
  • Full type inference for query results and page data

Memory Efficient:

  • Only fetches the data needed for loaded pages plus one peek-ahead item
  • Dynamically expands the window as users load more pages
  • Leverages collection's built-in ordering and indexing

Example Usage

const{ data, pages, fetchNextPage, hasNextPage, isFetchingNextPage,
isLoading }=useLiveInfiniteQuery((q)=>q.from({posts: postsCollection}).orderBy(({ posts })=>posts.createdAt,'desc'),{pageSize: 20,getNextPageParam: (lastPage)=>lastPage.length===20 ? lastPage.length : undefined})// Use in your component{pages.map((page,i)=>(<divkey={i}>{page.map(post=><PostCardkey={post.id}post={post}/>)}</div>))}{hasNextPage&&(<buttononClick={fetchNextPage}disabled={isFetchingNextPage}>{isFetchingNextPage ? 'Loading...' : 'Load More'}</button>)}

Test Coverage

All 17 tests passing:

  • ✅ Initial page loading
  • ✅ Multiple page fetching
  • ✅ No more pages detection
  • ✅ Empty results handling
  • ✅ Live updates when data changes
  • ✅ Deletions across pages
  • ✅ Where clause filtering
  • ✅ Dependency change handling
  • ✅ Page params tracking
  • ✅ Exact page size boundaries
  • ✅ Concurrent fetch prevention
  • ✅ No fetch when hasNextPage is false
  • ✅ Custom initialPageParam support
  • ✅ Dynamic hasNextPage detection on sync
  • isFetchingNextPage with immediate data
  • isFetchingNextPage with async loading
  • ✅ Prevent fetching while isFetchingNextPage is true

95% statement coverage on useLiveInfiniteQuery.

Closes#613

@changeset-bot

changeset-botBot commented Oct 9, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 6f7161c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
NameType
@tanstack/react-dbPatch
@tanstack/db-example-react-todoPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-newBot commented Oct 9, 2025

Copy link
Copy Markdown
More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@666

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@666

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@666

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@666

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@666

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@666

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@666

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@666

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@666

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@666

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@666

commit: 6f7161c

@github-actions

github-actionsBot commented Oct 9, 2025

Copy link
Copy Markdown
Contributor

Size Change: 0 B

Total Size: 83.6 kB

ℹ️ View Unchanged
FilenameSize
./packages/db/dist/esm/collection/change-events.js963 B
./packages/db/dist/esm/collection/changes.js1.01 kB
./packages/db/dist/esm/collection/events.js413 B
./packages/db/dist/esm/collection/index.js3.23 kB
./packages/db/dist/esm/collection/indexes.js1.16 kB
./packages/db/dist/esm/collection/lifecycle.js1.8 kB
./packages/db/dist/esm/collection/mutations.js2.52 kB
./packages/db/dist/esm/collection/state.js3.79 kB
./packages/db/dist/esm/collection/subscription.js2.2 kB
./packages/db/dist/esm/collection/sync.js2.2 kB
./packages/db/dist/esm/deferred.js230 B
./packages/db/dist/esm/errors.js3.57 kB
./packages/db/dist/esm/event-emitter.js798 B
./packages/db/dist/esm/index.js1.65 kB
./packages/db/dist/esm/indexes/auto-index.js794 B
./packages/db/dist/esm/indexes/base-index.js835 B
./packages/db/dist/esm/indexes/btree-index.js2 kB
./packages/db/dist/esm/indexes/lazy-index.js1.21 kB
./packages/db/dist/esm/indexes/reverse-index.js577 B
./packages/db/dist/esm/local-only.js967 B
./packages/db/dist/esm/local-storage.js2.33 kB
./packages/db/dist/esm/optimistic-action.js294 B
./packages/db/dist/esm/proxy.js3.86 kB
./packages/db/dist/esm/query/builder/functions.js615 B
./packages/db/dist/esm/query/builder/index.js4.04 kB
./packages/db/dist/esm/query/builder/ref-proxy.js938 B
./packages/db/dist/esm/query/compiler/evaluators.js1.55 kB
./packages/db/dist/esm/query/compiler/expressions.js760 B
./packages/db/dist/esm/query/compiler/group-by.js2.04 kB
./packages/db/dist/esm/query/compiler/index.js2.21 kB
./packages/db/dist/esm/query/compiler/joins.js2.65 kB
./packages/db/dist/esm/query/compiler/order-by.js1.43 kB
./packages/db/dist/esm/query/compiler/select.js1.28 kB
./packages/db/dist/esm/query/ir.js785 B
./packages/db/dist/esm/query/live-query-collection.js404 B
./packages/db/dist/esm/query/live/collection-config-builder.js5.49 kB
./packages/db/dist/esm/query/live/collection-registry.js233 B
./packages/db/dist/esm/query/live/collection-subscriber.js2.11 kB
./packages/db/dist/esm/query/optimizer.js3.26 kB
./packages/db/dist/esm/scheduler.js1.29 kB
./packages/db/dist/esm/SortedMap.js1.24 kB
./packages/db/dist/esm/transactions.js3.05 kB
./packages/db/dist/esm/utils.js1.01 kB
./packages/db/dist/esm/utils/browser-polyfills.js365 B
./packages/db/dist/esm/utils/btree.js6.01 kB
./packages/db/dist/esm/utils/comparison.js754 B
./packages/db/dist/esm/utils/index-optimization.js1.73 kB

compressed-size-action::db-package-size

@github-actions

github-actionsBot commented Oct 9, 2025

Copy link
Copy Markdown
Contributor

Size Change: 0 B

Total Size: 1.46 kB

ℹ️ View Unchanged
FilenameSize
./packages/react-db/dist/esm/index.js152 B
./packages/react-db/dist/esm/useLiveQuery.js1.31 kB

compressed-size-action::react-db-package-size

@KyleAMathews
KyleAMathewsforce-pushed the use-live-infinite-query branch from cf96419 to 248bdf4CompareOctober 9, 2025 21:40
@KyleAMathews
KyleAMathews changed the base branch from main to kevin/pred-pushdown-query-collOctober 9, 2025 21:41
@samwillis
samwillisforce-pushed the kevin/pred-pushdown-query-coll branch 2 times, most recently from 652af67 to 58d7af8CompareOctober 14, 2025 13:14
@samwillis
samwillisforce-pushed the use-live-infinite-query branch from 248bdf4 to 42a9dc6CompareOctober 15, 2025 10:17
@samwillis
samwillis changed the base branch from kevin/pred-pushdown-query-coll to mainOctober 15, 2025 10:17

@samwillissamwillis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KyleAMathews I rebased this on main after merging the utils.setWindow() pr #663, then updated this to use that + added a couple of extra tests.

I think it's good to go, but want to check if you want to do anything docs wise?

I will add the isLoadingMore as part of #669

KyleAMathewsand others added 2 commits October 15, 2025 13:04
improve types
add test that checks that we detect new pages on more rows syncing
changeset
tweaks
@samwillis
samwillisforce-pushed the use-live-infinite-query branch from a29d881 to 9146c76CompareOctober 15, 2025 12:27
@samwillis
samwillis changed the base branch from main to samwillis/load-more-trackingOctober 15, 2025 12:27
@samwillis
samwillisforce-pushed the use-live-infinite-query branch from 9146c76 to 6f7161cCompareOctober 15, 2025 12:29
@samwillis

Copy link
Copy Markdown
Collaborator

I've changed this to be stacked on #669 and use the promise from setWindow to handle tracking of the loading state.

@KyleAMathews
KyleAMathews merged commit f31a67e into samwillis/load-more-trackingOct 15, 2025
6 checks passed
@KyleAMathews
KyleAMathews deleted the use-live-infinite-query branch October 15, 2025 17:48
samwillis added a commit that referenced this pull request Oct 15, 2025
* wip
* refactor so both CollectionEventsManager and CollectionSubscription subclass the same event emiiter implimetation
* changeset
* rename loadMore to loadSubset
* feed the subscription object through to the loadSubset call, and add an unsunbscribed event to it
* feed subscription through to the loadSubset callback, add unsubscribe event to the subscription, fix types
* add sync mode to base colleciton
* loadSubset fn return promise or true
* add comment on setting is loading
* address review
* remove public trackLoadPromise
* setWindow returns a promise when it triggers loading subset
* feat: implement useLiveInfiniteQuery hook for React (#666)
* feat: implement useLiveInfiniteQuery hook for React
* use the new utils.setWindow to page through the results
improve types
add test that checks that we detect new pages on more rows syncing
changeset
tweaks
* isFetchingNextPage set by promise from setWindow
---------
Co-authored-by: Sam Willis <sam.willis@gmail.com>
---------
Co-authored-by: Kyle Mathews <mathews.kyle@gmail.com>
@devcsrj

Copy link
Copy Markdown
Contributor

Is a port for tanstack/svelte-db already in the works somewhere?

@KyleAMathews

Copy link
Copy Markdown
CollaboratorAuthor

@devcsrj no so feel to take it on!

@devcsrj

Copy link
Copy Markdown
Contributor

Hey @KyleAMathews ! Gave it a try here #1447

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

useLiveInfiniteQuery

3 participants

@KyleAMathews@samwillis@devcsrj