Uh oh!
There was an error while loading. Please reload this page.
Remove ListingTable and FileScanConfig Unbounded (#8540) - #8573
Conversation
| } | ||
| #[tokio::test] | ||
| async fn test_with_lost_ordering_unbounded() -> Result<()> { |
There was a problem hiding this comment.
I couldn't see a reason to keep these as they effectively duplicate the above
There was a problem hiding this comment.
These tests makes sure that when sources are unbounded, we try to preserve existing ordering as much as possible. I will re-add corresponding tests that enforces this functionality. For this PR, I just ignore these tests.
| " CsvExec: file_groups={1 group: [[file_path]]}, projection=[a, c, d], output_ordering=[a@0 ASC NULLS LAST], has_header=true", | ||
| ]; | ||
| assert_optimized!(expected_input, expected_optimized, physical_plan); | ||
| assert_optimized!(expected_input, expected_optimized, physical_plan, true); |
There was a problem hiding this comment.
#8572 tracks making this the default behaviour
metesynnada
left a comment
There was a problem hiding this comment.
Overall LGTM. Just a minor issue on tests.
There was a problem hiding this comment.
I think test logic may depend on infinite sources. So, we can use an unbounded executor like StreamingTableExec to preserve the test logic.
There was a problem hiding this comment.
It needs either infinite sources or prefer_existing_sort, I switched it to the latter
| .await?; | ||
| let path = format!("{testdata}/csv/aggregate_test_100.csv"); | ||
| match infinite { |
mustafasrepo
left a comment
There was a problem hiding this comment.
LGTM!. Thanks @tustvold for this cleanup.
Which issue does this PR close?
Closes#8540
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?