fix: null-aware anti join could emit rows after a sibling probe partition saw NULL - #25076
Conversation
| .unwrap() | ||
| } | ||
|
|
||
| /// Drains the probe partitions of `join` one after another in the given |
There was a problem hiding this comment.
These partitions are drained sequentially, so the old implementation also passes both orders: the NULL flag is always written before the final partition examines it. The tests would also remain green if the counter ordering changed back to Relaxed.
Could we force the failing overlap with a synchronization hook where the non-NULL partition reads false, the sibling records NULL, and the non-NULL partition then becomes last or add a model-checked atomic test? This wrong-results race otherwise has no regression coverage.
There was a problem hiding this comment.
I add the loom test
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #25076 +/- ##
==========================================
+ Coverage 81.89% 81.90% +0.01%
==========================================
Files 1131 1132 +1
Lines 419945 420155 +210
Branches 419945 420155 +210
==========================================
+ Hits 343901 344121 +220
+ Misses 55763 55744 -19
- Partials 20281 20290 +9 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
kosiew
left a comment
There was a problem hiding this comment.
@jayzhan211, thanks for working on this. The synchronization change looks good to me. Encapsulating the probe-side state behind the final summary makes the ordering requirement much clearer, and the new two-partition tests cover the race in both completion orders. I have one non-blocking suggestion for some additional dictionary-key coverage.
|
|
||
| /// Builds a two-partition probe side for the cross-partition null-aware | ||
| /// tests: partition 0 holds the only NULL key, partition 1 holds none. | ||
| fn build_two_partition_probe_with_null_in_partition_0() -> Arc<dyn ExecutionPlan> { |
There was a problem hiding this comment.
Could we also add a two-partition LeftAnti case where the NULL-bearing probe partition uses a dictionary key containing only a logical NULL? The existing dictionary test is single-partition, while this new helper covers physical Int32 NULLs. I think that would be useful for exercising the logical_null_count() path through the new cross-partition summary. This is non-blocking.
kumarUjjawal
left a comment
There was a problem hiding this comment.
Thanks @jayzhan211
Looks good!
|
🚀 |
…sion Merging main's chunked emission of final build rows split the final stage into `prepare_unmatched_build_rows` and `emit_unmatched_build_rows`, leaving the null-aware post-processing in the second one while `probe_summary` was still bound in the first. `report_probe_completed` hands the summary out exactly once, so capture it alongside the bitmap snapshot in `EmitUnmatchedBuildRowsState` and read it back per chunk.
|
One of my CI failure is |
…n-null-aware-probe-race
`loom` was a plain dev-dependency, so it joined every `cargo test
--workspace` resolve. It is not inert there: loom pulls in
tracing-subscriber, whose env-filter turns on `regex-automata`'s
`dfa-build`/`dfa-search` features for the whole workspace build. Those
change which engine the shared `regex` selects, and the new path costs
enough extra stack to abort unrelated deep-recursion tests --
`sql::unparser::test_tpch_unparser_roundtrip` overflowed its stack in CI
on every run since the dependency was added.
Measured on `core_integration`, varying only whether loom is in the
resolve (`RUST_MIN_STACK` at which the test passes):
without loom 480K ok 1M ok 2M ok 4M ok
with loom 480K ok 1M FAIL 2M FAIL 4M ok
Declaring loom under `cfg(datafusion_loom)` leaves the ordinary build
byte-identical to one that never mentioned it. The cfg is namespaced
because a bare `loom` is also read by tokio, which would switch tokio
into its own loom build and drop `tokio::fs`.
Run the model checks with:
RUSTFLAGS="--cfg datafusion_loom" \
cargo test -p datafusion-physical-plan --lib loom_tests
Which issue does this PR close?
Rationale for this change
x NOT IN (subquery)is planned as a null-awareLeftAntihash join inCollectLeftmode with several probe partitions. If any probe partition seesa NULL key, the predicate is UNKNOWN for every build row and the join must
return nothing.
The final stage of
HashJoinStreamread the shared "probe side saw NULL"flag before decrementing the probe-partition counter. That allowed this
interleaving:
false;decrements the counter;
the unmatched build rows.
The result then contains rows that
NOT INmust suppress. The counter wasalso decremented with
Ordering::Relaxed, so nothing ordered the flag storesof a finishing partition before the reads of the last one, even when the
reads happened after the decrement.
What changes are included in this PR?
JoinLeftData::report_probe_completednow decrements withAcqRelandreturns
Option<ProbeSideSummary>:Someonly for the last partition, withthe shared flags read after the decrement. The two
AtomicBoolflags areprivate; the probe phase stores through
record_probe_batchand the earlyexit reads a
probe_side_has_null_hintthat is documented as best-effort.The final stage therefore cannot read the flags before its own decrement,
which guards the fix structurally rather than by a test that would have to
force a few-instruction race window.
process_unmatched_build_batchdecrements first and hands the summary tothe null-aware helpers. The NULL-probe rule of
LeftAntimoved intonull_aware_left_anti_final_indices, so all its final-stage rules are inone place.
AcqRelorderingon the counter.
What is the testing strategy for this PR?
Two new tests in
hash_join/exec.rs,test_null_aware_anti_join_probe_null_in_other_partitionandtest_null_aware_left_mark_probe_null_in_other_partition, run aCollectLeftjoin with a two-partition probe side where only partition 0 has a NULL key,
draining the partitions in both orders. They cover the cross-partition flag
propagation that had no test. The race itself cannot be reproduced
deterministically without a test hook, which is why the fix makes the wrong
call order unrepresentable instead. Existing null-aware join tests and the
join unit test suite pass unchanged.
Are there any user-facing changes?
Wrong results from
NOT INunder this interleaving are fixed. No API changes;all touched items are
pub(super).