Uh oh!
There was an error while loading. Please reload this page.
Query Driven Sync with On-Demand Collection Loading - #763
Merged
Conversation
🦋 Changeset detectedLatest commit: 471fa8f The changes in this PR will be included in the next version bump. This PR includes changesets to release 13 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
More templates@tanstack/angular-db@tanstack/db@tanstack/db-ivm@tanstack/electric-db-collection@tanstack/offline-transactions@tanstack/powersync-db-collection@tanstack/query-db-collection@tanstack/react-db@tanstack/rxdb-db-collection@tanstack/solid-db@tanstack/svelte-db@tanstack/trailbase-db-collection@tanstack/vue-dbcommit: |
Contributor
Size Change: +5.99 kB (+7.49%) 🔍 Total Size: 86 kB
ℹ️ View Unchanged
|
Contributor
Size Change: 0 B Total Size: 3.34 kB ℹ️ View Unchanged
|
samwillisforce-pushed
the
query-driven-sync
branch
from
November 6, 2025 17:26
4e77602 to
685af96Compare33 tasks
7 tasks
This was referenced Nov 10, 2025
Closed
) * Add unit tests that reproduce the problems with moveWindow on infinite limits * Handle Infinity limit in move * Changeset * Add support for orderBy and limit options in currentStateAsChanges * Unit tests for currentStateAsChanges * changeset * fix for eslint * Move helper functions to the end of the file * add predicate utils * optimise in when all primatives * optimisations * changeset * minusWherePredicates * Add unit test for OR in both subset and superset that shows bug with OR handling * Reorder OR handling to fix bug and handle AND similarly * change chageset * dedupe code * add createDeduplicatedLoadSubset function * convert deduper to a class, with reset, and dedupe inflight * Simplify matching of inflight requests * Improve minusWherePredicates such that it can handle common conditions in a conjunction * Modify DeduplicatedLoadSubset.loadSubset such that it only loads the missing data * Remove unused predicate helper functions * Fix eslint issue * Prettier * Unify handling of IN and OR * Handle subset that is false * Fix isLimitSubset to correctly handle unlimited subsets --------- Co-authored-by: Kevin De Porre <kevin@electric-sql.com>
* Callback that informs about deduplicated loadSubset calls * Unit tests for onDeduplicate callback + move dedupe test files to the right folder * changeset * Object params * Updated lockfile * fix lock file --------- Co-authored-by: Sam Willis <sam.willis@gmail.com>
* Handle pushed down predicates in query collection Co-authored-by: Kevin De Porre <kevin@electric-sql.com> Co-authored-by: Sam Willis <sam.willis@gmail.com> * Deduplicate loadSubset requests in Query collection * Unit test for deduplicating limited ordered queries in query collection * GC tanstack query when subscription is unsubscribed --------- Co-authored-by: Sam Willis <sam.willis@gmail.com> Co-authored-by: Kevin De Porre <kevin@electric-sql.com>
* Handle pushed down predicates in Electric collection Co-authored-by: Kevin De Porre <kevin@electric-sql.com> Co-authored-by: Sam Willis <sam.willis@gmail.com> * use the subsetDuduper for electric * Leave fixme * fix DeduplicatedLoadSubset call * fix tests --------- Co-authored-by: Sam Willis <sam.willis@gmail.com>
#762) * Add optional compareOptions to the Collection Config which act as collection-wide config and allow queries to override those. * Fix import * changeset * Ran prettier * update docs * Live queries inherit compare options from collection in the FROM clause * Explicit compare options for live queries * Rename compareOptions to defaultStringCollation in the collection config and live query collection config * Docs update --------- Co-authored-by: Sam Willis <sam.willis@gmail.com>
…#786) * feat: add expression helpers for parsing LoadSubsetOptions in queryFn Add reusable expression parsing utilities to help developers translate TanStack DB predicates (where, orderBy, limit) into their API's format. - Add expression-helpers.ts with generic parsing utilities - parseWhereExpression: Parse where clauses with custom handlers - parseOrderByExpression: Parse order by into simple array - extractSimpleComparisons: Extract simple AND-ed filters - parseLoadSubsetOptions: Convenience function for all options - walkExpression, extractFieldPath, extractValue: Lower-level helpers - Export helpers from query-db-collection package - Add comprehensive documentation to query-collection.md - How LoadSubsetOptions are passed via ctx.meta - Expression helper usage with REST and GraphQL examples - API reference for all helper functions - Tips and best practices This makes it much easier to implement query collections with predicate push-down without having to manually parse expression AST trees. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * test: add comprehensive tests for expression helpers - 35 test cases covering all helper functions - Tests for parseWhereExpression with various operators - Tests for parseOrderByExpression with sorting options - Tests for extractSimpleComparisons - Tests for parseLoadSubsetOptions - Tests for low-level helpers (extractFieldPath, extractValue, walkExpression) - Integration test for complex real-world query scenarios All tests passing with 93.65% coverage of expression-helpers.ts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: add changeset for expression helpers 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: make GraphQL example more generic Remove Hasura-specific emphasis in GraphQL example section. The underscore-prefixed operators are common GraphQL conventions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: clarify GraphQL format is not prescriptive Change 'the GraphQL format' to 'a GraphQL format' to avoid implying this is the only way to structure GraphQL queries. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: remove Electric DB reference from tips Remove implementation-specific reference to Electric DB Collection from the tips section. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: make loadSubsetOptions always an object - Add default empty object parameter to createQueryFromOpts - Remove ?? {} pattern from all documentation - Update changeset example - Update PR description This makes the API cleaner - users can now access ctx.meta.loadSubsetOptions directly without null coalescing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: add explicit type annotations to avoid implicit any - Import BasicExpression and OrderBy from IR namespace - Add type annotations to all lambda parameters - Remove unnecessary type checks that were always true - Fix variable shadowing in walkExpression visitor parameter - Fixes build type errors and eslint warnings in expression-helpers.ts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * style: fix prettier formatting in changeset 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: correct TypeScript types in expression-helpers tests * feat: address Sam's review feedback - Add OperatorName type and operators constant to db package - Improve ParseWhereOptions handlers type to know IR operators - Support array indices in FieldPath (string | number) - Add locale/string collation options to ParsedOrderBy - Make extractSimpleComparisons throw on unsupported operations (or, not, etc.) - Update tests to expect throws instead of silent skipping * fix: remove invalid sensitivity/locale fields from test mocks CompareOptions uses StringCollationConfig which has stringSort, locale, and localeOptions fields, not sensitivity and locale at the top level. * fix: handle discriminated union for CompareOptions properly Use 'in' operator to check for optional fields (locale, localeOptions) that only exist when stringSort is 'locale' * move helpers to main db package --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Sam Willis <sam.willis@gmail.com>
samwillisforce-pushed
the
query-driven-sync
branch
from
November 12, 2025 00:02
f7b3988 to
55ea32eCompare* Implement end-to-end tests for sync feature Implement end-to-end tests for sync feature Implement end-to-end tests for sync feature chore: revert pnpm-lock.yaml to match query-driven-sync chore: update pnpm-lock.yaml with e2e test pg dependencies bump pnpm/action-setup version fix pnpmlockfile format only run normal tests on test command tidy and fix types fixes ci fix tidy checkpoint cleanup between each test add cleanup between tests all e2e tests pass! cleanup live queries fixes for ci add propper mutations to e2e tests fix ci timouts move the badly named accumulated data tests fix ci flake working query e2e tests fix lint fix assertions Fix race condition between test and logical replication stream Handle isNull and isUndefined in expression helpers Ran prettier. fix ci update docs add suport for not to extractSimpleComparisons and get tests to pass try and fix ci race * increase timeout
Contributor
🎉 This PR has been released!
Thank you for your contribution! |
This was referenced Jul 13, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a catch all branch/PR for all PR's implementing RFC: On-Demand Collection Loading via loadSubset
Must be merged to main with a rebase to capture all the commits in this branch
To merge: