[FIX](collectiontype) fix shrink char column in map/struct by amorynan · Pull Request #25725 · apache/doris

amorynan · 2023-10-22T08:55:32Z

Proposed changes

Issue Number: close #xxx
fix shrink char column in map/struct
before we has char with specific length defined in map or struct field
we select map or struct , the char column in which one has been padding if we just insert less than specific length chars
but in mysql here just show inserted chars not padding specific length chars
like :

after

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

github-actions · 2023-10-22T09:04:10Z

clang-tidy review says "All clean, LGTM! 👍"

yiguolei

LGTM

yiguolei · 2023-10-23T02:53:00Z

run buildall

github-actions · 2023-10-23T03:02:58Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2023-10-23T03:14:32Z

TeamCity be ut coverage result:
Function Coverage: 37.06% (8307/22415)
Line Coverage: 29.20% (66693/228399)
Region Coverage: 27.81% (34639/124574)
Branch Coverage: 24.42% (17596/72046)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c6c9084868b5f60f184df8839829d62c7437cdf1_c6c9084868b5f60f184df8839829d62c7437cdf1/report/index.html

amorynan · 2023-10-23T07:12:41Z

run buildall

github-actions

clang-tidy made some suggestions

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

amorynan · 2023-10-23T08:01:32Z

run buildall

github-actions · 2023-10-23T08:11:28Z

clang-tidy review says "All clean, LGTM! 👍"

amorynan · 2023-10-23T08:19:19Z

run beut

amorynan · 2023-10-23T08:19:27Z

run clickbench

amorynan · 2023-10-23T08:35:42Z

run buildall

amorynan · 2023-10-23T08:43:12Z

run buildall

doris-robot · 2023-10-23T09:12:45Z

TeamCity be ut coverage result:
Function Coverage: 37.02% (8299/22417)
Line Coverage: 29.16% (66620/228474)
Region Coverage: 27.74% (34604/124754)
Branch Coverage: 24.40% (17582/72056)
Coverage Report: http://coverage.selectdb-in.cc/coverage/64a4b8349063a29519c1b2df74d0d97fe2f3b349_64a4b8349063a29519c1b2df74d0d97fe2f3b349/report/index.html

amorynan · 2023-10-23T09:15:48Z

run buildall

doris-robot · 2023-10-23T09:38:17Z

TeamCity be ut coverage result:
Function Coverage: 37.01% (8297/22417)
Line Coverage: 29.15% (66609/228466)
Region Coverage: 27.74% (34606/124740)
Branch Coverage: 24.40% (17581/72050)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ac91d7e86c5d70fb5f76a0810311258822798426_ac91d7e86c5d70fb5f76a0810311258822798426/report/index.html

doris-robot · 2023-10-23T10:51:40Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.81 seconds
stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17162004047 Bytes

amorynan · 2023-10-24T03:18:00Z

run buildall

amorynan · 2023-10-24T05:48:50Z

run p0

amorynan · 2023-10-24T06:14:34Z

run p0

amorynan · 2023-10-24T10:02:22Z

run buildall

eldenmoon

LGTM

github-actions · 2023-10-24T10:05:26Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-10-24T10:05:28Z

PR approved by anyone and no changes requested.

doris-robot · 2023-10-24T11:41:25Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.84 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162347658 Bytes

amorynan · 2023-10-25T08:08:19Z

run p0

yiguolei

LGTM

) fix shrink char column in map/struct before we has char with specific length defined in map or struct field we select map or struct , the char column in which one has been padding if we just insert less than specific length chars but in mysql here just show inserted chars not padding specific length chars --------- Co-authored-by: yiguolei <676222867@qq.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix shrink char column in map/struct
db3086d

xiaokang added p0_w dev/2.0.3 labels Oct 22, 2023

amorynan changed the title ~~[FIX](map/struct) fix shrink char column in map/struct~~[FIX](collectiontype) fix shrink char column in map/structOct 23, 2023

yiguolei previously approved these changes Oct 23, 2023
View reviewed changes

Merge branch 'master' into fix-shrink-char-in-collection-type
c6c9084

fix map shrink and add regress tests
e322477

amorynan dismissed yiguolei’s stale review via e322477October 23, 2023 07:12

github-actionsBot reviewed Oct 23, 2023
View reviewed changes

Comment threadbe/src/vec/columns/column_struct.h

Update be/src/vec/columns/column_struct.h
922f491
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix compile
64a4b83

fix out and compile
ac91d7e

fix case out and field functions
38ef574

fix case
fc39b2a

eldenmoon approved these changes Oct 24, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 24, 2023

github-actionsBot added the reviewed label Oct 24, 2023

amorynan requested a review from yiguolei October 25, 2023 08:54

yiguolei approved these changes Oct 25, 2023
View reviewed changes

yiguolei merged commit 8a436d8 into apache:masterOct 25, 2023

xiaokang added merge_conflict dev/2.0.3-merged dev/2.0.3 and removed dev/2.0.3 dev/2.0.3-merged labels Oct 25, 2023

xiaokang added dev/2.0.3-merged and removed dev/2.0.3 labels Oct 31, 2023

xiaokang mentioned this pull request Dec 4, 2023
Release Note 2.0.3 #27909
Closed

Uh oh!

Conversation

amorynan commented Oct 22, 2023

Proposed changes

Further comments

Uh oh!

github-actionsBot commented Oct 22, 2023

Uh oh!

yiguolei left a comment

Choose a reason for hiding this comment

Uh oh!

yiguolei commented Oct 23, 2023

Uh oh!

github-actionsBot commented Oct 23, 2023

Uh oh!

doris-robot commented Oct 23, 2023

Uh oh!

amorynan commented Oct 23, 2023

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

amorynan commented Oct 23, 2023

Uh oh!

github-actionsBot commented Oct 23, 2023

Uh oh!

amorynan commented Oct 23, 2023

Uh oh!

amorynan commented Oct 23, 2023

Uh oh!

amorynan commented Oct 23, 2023

Uh oh!

amorynan commented Oct 23, 2023

Uh oh!

doris-robot commented Oct 23, 2023

Uh oh!

amorynan commented Oct 23, 2023

Uh oh!

doris-robot commented Oct 23, 2023

Uh oh!

doris-robot commented Oct 23, 2023

Uh oh!

amorynan commented Oct 24, 2023

Uh oh!

amorynan commented Oct 24, 2023

Uh oh!

amorynan commented Oct 24, 2023

Uh oh!

amorynan commented Oct 24, 2023

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Oct 24, 2023

Uh oh!

github-actionsBot commented Oct 24, 2023

Uh oh!

doris-robot commented Oct 24, 2023

Uh oh!

amorynan commented Oct 25, 2023

Uh oh!

yiguolei left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants