Prune values across the query executor instead of on the calling thread - #19513
Open
xiangfu0 wants to merge 2 commits into
Open
Prune values across the query executor instead of on the calling thread#19513xiangfu0 wants to merge 2 commits into
xiangfu0 wants to merge 2 commits into
Conversation
xiangfu0
force-pushed
the
xiangfu0/data-3221-12-parallel-value-pruning
branch
from
September 9, 2026 01:56
8c6e534 to
785926d
Compare
xiangfu0
force-pushed
the
xiangfu0/data-3221-11-metadata-only-pruning
branch
from
September 9, 2026 01:56
6f1c691 to
9a69689
Compare
Use the query executor for value pruning when the segment count exceeds the parallel threshold, with task-local caches and bounded concurrency. Keep the serial path for small inputs or when no executor is supplied. Stop interrupted workers before visiting another segment so cancellation can finish while the deadline helper waits for workers to release segments. Cover executor use, fallback, deadlines, worker failures and interruption.
xiangfu0
changed the base branch from
xiangfu0/data-3221-11-metadata-only-pruning
to
master
September 10, 2026 23:02
xiangfu0
force-pushed
the
xiangfu0/data-3221-12-parallel-value-pruning
branch
from
September 10, 2026 23:02
785926d to
c375b71
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Value-based segment pruning uses the query executor for lists larger than 10 segments, bounded by the query's execution-thread limit. Small lists and calls without an executor retain serial execution.
ValueBasedSegmentPrunerowns the shared serial/parallel loops, task-local caches, deadline handling, interruption checks, and per-segment fetch-context acquisition/release.BloomFilterSegmentPrunerdelegates to that implementation while retaining its prefetch planning and cleanup. Duplicate execution code and stale threshold comments are removed. Parallel results may have a different order.Validation
ColumnValueSegmentPrunerTest,BloomFilterSegmentPrunerTest,SegmentPrunerServiceTest, andQueryMultiThreadingUtilsTest.pinot-core.ZstandardDecompressor.java:51because the Zstandard dependency references an unavailable JetBrainsNotNullannotation. Isolated compilation reproduced this on the exact master source; normal compilation succeeds, and supplying the annotation jar also makes strict compilation succeed.