Skip to content

[fix](load) fix broker load silently loaded only the first file when parsing multiple files path - #62969

Merged
liaoxin01 merged 1 commit into
apache:masterfrom
sollhui:fix_load_multiple_files
May 7, 2026
Merged

[fix](load) fix broker load silently loaded only the first file when parsing multiple files path#62969
liaoxin01 merged 1 commit into
apache:masterfrom
sollhui:fix_load_multiple_files

Conversation

@sollhui

@sollhuisollhui commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

Fix Nereids parsing for Broker Load statements that use multiple file paths in a single DATA INFILE clause, for example:

DATA INFILE("f1", "f2", "f3")

Previously, only the first file path was preserved. Later paths were parsed into a different ANTLR label and were not consumed by LogicalPlanBuilder, so Broker Load silently loaded only the first file.

This PR:

  • Uses one ANTLR list label for all DATA INFILE paths.

  • Updates LogicalPlanBuilder to build file paths from the collected list.

  • Adds FE unit coverage for parsed file paths.

  • Adds a regression case for S3 broker load with multiple paths in one DATA INFILE.

@hello-stephen

hello-stephen commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@sollhui

sollhui commented Apr 30, 2026

Copy link
Copy Markdown
ContributorAuthor

run buildall

liaoxin01
liaoxin01 approved these changes Apr 30, 2026
View reviewed changes

@liaoxin01liaoxin01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Apr 30, 2026
@github-actions

github-actionsBot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@sollhui

sollhui commented May 6, 2026

Copy link
Copy Markdown
ContributorAuthor

run buildall

1 similar comment
@liaoxin01

liaoxin01 commented May 6, 2026

Copy link
Copy Markdown
Contributor

run buildall

@liaoxin01
liaoxin01 merged commit ff2157b into apache:masterMay 7, 2026
33 of 34 checks passed
github-actionsBot pushed a commit that referenced this pull request May 7, 2026
@sollhuiYour Name
[fix](load) fix broker load silently loaded only the first file when …
8d39913
…parsing multiple files path (#62969)
### Proposed changes
Fix Nereids parsing for Broker Load statements that use multiple file
paths in a single DATA INFILE clause, for example:
```
DATA INFILE("f1", "f2", "f3")
```
Previously, only the first file path was preserved. Later paths were
parsed into a different ANTLR label and were not consumed by
LogicalPlanBuilder, so Broker Load silently loaded only the first file.
This PR:
- Uses one ANTLR list label for all DATA INFILE paths.
- Updates LogicalPlanBuilder to build file paths from the collected
list.
- Adds FE unit coverage for parsed file paths.
- Adds a regression case for S3 broker load with multiple paths in one
DATA INFILE.
@github-actionsgithub-actionsBot mentioned this pull request May 7, 2026
Merged
github-actionsBot pushed a commit that referenced this pull request May 7, 2026
@sollhuiYour Name
[fix](load) fix broker load silently loaded only the first file when …
17ddb18
…parsing multiple files path (#62969)
### Proposed changes
Fix Nereids parsing for Broker Load statements that use multiple file
paths in a single DATA INFILE clause, for example:
```
DATA INFILE("f1", "f2", "f3")
```
Previously, only the first file path was preserved. Later paths were
parsed into a different ANTLR label and were not consumed by
LogicalPlanBuilder, so Broker Load silently loaded only the first file.
This PR:
- Uses one ANTLR list label for all DATA INFILE paths.
- Updates LogicalPlanBuilder to build file paths from the collected
list.
- Adds FE unit coverage for parsed file paths.
- Adds a regression case for S3 broker load with multiple paths in one
DATA INFILE.
@github-actionsgithub-actionsBot mentioned this pull request May 7, 2026
Merged
yiguolei pushed a commit that referenced this pull request May 7, 2026
@github-actions@sollhui
branch-4.1: [fix](load) fix broker load silently loaded only the firs…
fd9bd87
…t file when parsing multiple files path #62969 (#63042)
Cherry-picked from #62969
Co-authored-by: hui lai <laihui@selectdb.com>
yiguolei pushed a commit that referenced this pull request May 7, 2026
@github-actions@sollhui
branch-4.0: [fix](load) fix broker load silently loaded only the firs…
2369600
…t file when parsing multiple files path #62969 (#63041)
Cherry-picked from #62969
Co-authored-by: hui lai <laihui@selectdb.com>
@yiguoleiyiguolei mentioned this pull request May 20, 2026
Closed
@morningmanmorningman mentioned this pull request Jun 3, 2026
Closed
zhaorongsheng pushed a commit to zhaorongsheng/doris that referenced this pull request Jun 4, 2026
@sollhuizhaorongsheng
[fix](load) fix broker load silently loaded only the first file when …
327393c
…parsing multiple files path (apache#62969)
### Proposed changes
Fix Nereids parsing for Broker Load statements that use multiple file
paths in a single DATA INFILE clause, for example:
```
DATA INFILE("f1", "f2", "f3")
```
Previously, only the first file path was preserved. Later paths were
parsed into a different ANTLR label and were not consumed by
LogicalPlanBuilder, so Broker Load silently loaded only the first file.
This PR:
- Uses one ANTLR list label for all DATA INFILE paths.
- Updates LogicalPlanBuilder to build file paths from the collected
list.
- Adds FE unit coverage for parsed file paths.
- Adds a regression case for S3 broker load with multiple paths in one
DATA INFILE.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchougavinchougavinchou approved these changes
@liaoxin01liaoxin01liaoxin01 approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/4.0.6-mergeddev/4.1.1-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@sollhui@hello-stephen@liaoxin01@gavinchou@yiguolei