[feature](struct-type) support csv format stream load for struct type by xy720 · Pull Request #17143 · apache/doris

xy720 · 2023-02-26T11:27:00Z

Proposed changes

Issue Number: close#16547

Refactor from_string method in data_type_struct.cpp to support csv format stream load for struct type.

Problem summary

Describe your changes.

Checklist(Required)

Does it affect the original behavior
Has unit tests been added
Has document been added or modified
Does it need to update dependencies
Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

github-actions · 2023-02-26T11:30:29Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2023-02-26T11:37:48Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2023-02-26T11:42:23Z

clang-tidy review says "All clean, LGTM! 👍"

xy720 · 2023-02-26T11:48:07Z

run buildall

xy720 · 2023-02-26T11:48:14Z

run p0

github-actions · 2023-02-26T11:48:45Z

clang-tidy review says "All clean, LGTM! 👍"

xy720 · 2023-02-26T12:10:46Z

run p0

hello-stephen · 2023-02-26T12:11:02Z

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 33.52 seconds
stream load tsv: 461 seconds loaded 74807831229 Bytes, about 154 MB/s
stream load json: 36 seconds loaded 2358488459 Bytes, about 62 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230301033930_clickbench_pr_106098.html

xy720 · 2023-02-26T12:30:20Z

run p0

github-actions · 2023-02-27T05:22:21Z

clang-tidy review says "All clean, LGTM! 👍"

xy720 · 2023-02-27T05:31:35Z

run buildall

xy720 · 2023-02-27T09:22:50Z

run buildall

xy720 · 2023-02-27T09:22:57Z

run p0

github-actions · 2023-02-27T09:25:16Z

clang-tidy review says "All clean, LGTM! 👍"

xy720 · 2023-02-27T09:36:06Z

run buildall

github-actions · 2023-02-27T09:37:23Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2023-02-27T09:38:43Z

clang-tidy review says "All clean, LGTM! 👍"

cambyzju

LGTM

github-actions · 2023-02-27T10:06:50Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-02-27T10:06:53Z

PR approved by anyone and no changes requested.

xy720 · 2023-02-27T14:16:43Z

run p0

github-actions · 2023-02-28T02:28:01Z

clang-tidy review says "All clean, LGTM! 👍"

xy720 · 2023-02-28T02:46:15Z

run buildall

xy720 · 2023-02-28T08:07:00Z

run buildall

github-actions · 2023-02-28T08:10:16Z

clang-tidy review says "All clean, LGTM! 👍"

xy720 · 2023-02-28T14:20:31Z

xxx

xy720 · 2023-02-28T14:20:43Z

run buildall

github-actions · 2023-02-28T14:22:53Z

clang-tidy review says "All clean, LGTM! 👍"

xy720 · 2023-03-01T02:41:15Z

run buildall

github-actions · 2023-03-01T02:43:52Z

clang-tidy review says "All clean, LGTM! 👍"

cambyzju

LGTM

github-actions · 2023-03-01T06:43:03Z

PR approved by at least one committer and no changes requested.

…apache#17143) Refactor from_string method in data_type_struct.cpp to support csv format stream load for struct type.

github-actionsBot added area/vectorization kind/test labels Feb 26, 2023

xy720 added area/stream-load struct-type Issues or PRs related to struct type labels Feb 26, 2023

xy720 mentioned this pull request Feb 26, 2023
[Roadmap] Support Struct+Map Type #16547
Closed
88 tasks

xy720 force-pushed the csv-load-for-struct branch from e3e3d0a to dacc9eeCompare February 27, 2023 05:18

cambyzju previously approved these changes Feb 27, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 27, 2023

github-actionsBot added the reviewed label Feb 27, 2023

xy720 dismissed cambyzju’s stale review via adf2fafFebruary 28, 2023 02:24

xy720 force-pushed the csv-load-for-struct branch from 69eeb7b to adf2fafCompare February 28, 2023 02:24

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Feb 28, 2023

xy720 force-pushed the csv-load-for-struct branch from adf2faf to b190c50Compare February 28, 2023 08:06

xy720 added 7 commits February 28, 2023 22:19

save
7e55acb

format
1d7ab87

format
fe8fc19

format
5ad8038

save
6e365dd

save
234046b

save
b22c713

xy720 force-pushed the csv-load-for-struct branch from b190c50 to b22c713Compare February 28, 2023 14:19

save
8e269bc

cambyzju approved these changes Mar 1, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Mar 1, 2023

xy720 merged commit 0732eb5 into apache:masterMar 1, 2023

yagagagaga pushed a commit to yagagagaga/doris that referenced this pull request Mar 9, 2023
[feature](struct-type) support csv format stream load for struct type (…
c3dd8e9
…apache#17143) Refactor from_string method in data_type_struct.cpp to support csv format stream load for struct type.

Uh oh!

Conversation

xy720 commented Feb 26, 2023

Proposed changes

Problem summary

Checklist(Required)

Further comments

Uh oh!

github-actionsBot commented Feb 26, 2023

Uh oh!

github-actionsBot commented Feb 26, 2023

Uh oh!

github-actionsBot commented Feb 26, 2023

Uh oh!

xy720 commented Feb 26, 2023

Uh oh!

xy720 commented Feb 26, 2023

Uh oh!

github-actionsBot commented Feb 26, 2023

Uh oh!

xy720 commented Feb 26, 2023

Uh oh!

hello-stephen commented Feb 26, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xy720 commented Feb 26, 2023

Uh oh!

github-actionsBot commented Feb 27, 2023

Uh oh!

xy720 commented Feb 27, 2023

Uh oh!

xy720 commented Feb 27, 2023

Uh oh!

xy720 commented Feb 27, 2023

Uh oh!

github-actionsBot commented Feb 27, 2023

Uh oh!

xy720 commented Feb 27, 2023

Uh oh!

github-actionsBot commented Feb 27, 2023

Uh oh!

github-actionsBot commented Feb 27, 2023

Uh oh!

cambyzju left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Feb 27, 2023

Uh oh!

github-actionsBot commented Feb 27, 2023

Uh oh!

xy720 commented Feb 27, 2023

Uh oh!

github-actionsBot commented Feb 28, 2023

Uh oh!

xy720 commented Feb 28, 2023

Uh oh!

xy720 commented Feb 28, 2023

Uh oh!

github-actionsBot commented Feb 28, 2023

Uh oh!

xy720 commented Feb 28, 2023

Uh oh!

xy720 commented Feb 28, 2023

Uh oh!

github-actionsBot commented Feb 28, 2023

Uh oh!

xy720 commented Mar 1, 2023

Uh oh!

github-actionsBot commented Mar 1, 2023

Uh oh!

cambyzju left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Mar 1, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

hello-stephen commented Feb 26, 2023•
edited
Loading