[cloud](Variant) limit column size in MetaService and support handle rowset schema in backend side - #41482

Merged

xiaokang merged 2 commits into

apache:masterfrom

eldenmoon:cloud-lm-ar

Nov 1, 2024

eldenmoon commented Sep 29, 2024•
edited
Loading

Member

in local mode we limit column size in commit_txn phase, in cloud mode we limit it in write_schema_dict which in commit_rowset phase
add config to decide whether fill schema in BE side or MS side

doris-robot commented Sep 29, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

eldenmoon commented Sep 29, 2024

MemberAuthor

run buildall

eldenmoon force-pushed the cloud-lm-ar branch from 4b900cf to 262f7ebCompare

September 29, 2024 17:53

eldenmoon commented Sep 29, 2024

MemberAuthor

run buildall

wm1581066 requested a review from gavinchou

September 30, 2024 02:07

wm1581066 added the usercase label

gavinchou reviewed

View reviewed changes

cloud/src/meta-service/meta_service_schema.cpp

gavinchou requested changes

View reviewed changes

cloud/src/common/config.h Outdated

[cloud](Variant) limit column size in MetaService

acda4df

in local mode we limit column size in commit_txn phase, in cloud mode we limit it in `write_schema_dict` which in commit_rowset phase

eldenmoon force-pushed the cloud-lm-ar branch 2 times, most recently from b1f8c04 to acda4dfCompare

October 11, 2024 07:11

github-actionsBot commented Oct 11, 2024

Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment

github-actionsBot commented Oct 11, 2024

Contributor

clang-tidy review says "All clean, LGTM! 👍"

eldenmoon commented Oct 21, 2024

MemberAuthor

run buildall

github-actionsBot commented Oct 21, 2024

Contributor

clang-tidy review says "All clean, LGTM! 👍"

eldenmoon force-pushed the cloud-lm-ar branch from 366aa9d to c26c607Compare

October 21, 2024 08:05

github-actionsBot commented Oct 21, 2024

Contributor

clang-tidy review says "All clean, LGTM! 👍"

doris-robot commented Oct 21, 2024

TeamCity be ut coverage result:
Function Coverage: 37.48% (9708/25903)
Line Coverage: 28.76% (80650/280471)
Region Coverage: 28.19% (41730/148016)
Branch Coverage: 24.77% (21214/85650)
Coverage Report: http://coverage.selectdb-in.cc/coverage/366aa9d61e917e19e0bc1d478018ec51553aaa01_366aa9d61e917e19e0bc1d478018ec51553aaa01/report/index.html

optimize get_rowset fill dict columns with variant

9ee812c

eldenmoon force-pushed the cloud-lm-ar branch from c26c607 to 9ee812cCompare

October 22, 2024 13:22

eldenmoon commented Oct 22, 2024

MemberAuthor

run buildall

github-actionsBot commented Oct 22, 2024

Contributor

clang-tidy review says "All clean, LGTM! 👍"

eldenmoon requested a review from gavinchou

October 22, 2024 13:35

doris-robot commented Oct 22, 2024

TeamCity be ut coverage result:
Function Coverage: 37.47% (9708/25911)
Line Coverage: 28.73% (80534/280308)
Region Coverage: 28.15% (41638/147916)
Branch Coverage: 24.72% (21156/85578)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9ee812c2b11891b938ea73e548b998cafd85d9e8_9ee812c2b11891b938ea73e548b998cafd85d9e8/report/index.html

eldenmoon changed the title ~~[cloud](Variant) limit column size in MetaService~~[cloud](Variant) limit column size in MetaService and support handle rowset schema in backend side

eldenmoon added the dev/3.0.x label

xiaokang reviewed

View reviewed changes

be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp

be/src/cloud/pb_convert.cpp

cloud/src/common/config.h

cloud/src/meta-service/meta_service_schema.cpp

gavinchou approved these changes

View reviewed changes

gavinchou commented Oct 31, 2024

Contributor

LGTM

github-actionsBot added the approved label

github-actionsBot commented Oct 31, 2024

Contributor

PR approved by at least one committer and no changes requested.

github-actionsBot added the reviewed label

github-actionsBot commented Oct 31, 2024

Contributor

PR approved by anyone and no changes requested.

eldenmoon added dev/3.0.x and removed dev/3.0.x labels

xiaokang approved these changes

View reviewed changes

xiaokang left a comment

Contributor

LGTM

xiaokang merged commit 2092b44 into apache:master

github-actionsBot pushed a commit that referenced this pull request

[cloud](Variant) limit column size in MetaService and support handle …

f85ade1

…rowset schema in backend side (#41482)
1. in local mode we limit column size in commit_txn phase, in cloud mode
we limit it in `write_schema_dict` which in commit_rowset phase
2. add config to decide whether fill schema in BE side or MS side

github-actionsBot mentioned this pull request

[cloud](Variant) limit column size in MetaService and support handle rowset schema in backend side(#41482) #43093

Merged

eldenmoon deleted the cloud-lm-ar branch

November 4, 2024 02:11

eldenmoon pushed a commit that referenced this pull request

[cloud](Variant) limit column size in MetaService and support handle …

d9f270a

…rowset schema in backend side(#41482) (#43093)
PR Body: 1. in local mode we limit column size in commit_txn phase, in
cloud mode we limit it in `write_schema_dict` which in commit_rowset
phase
2. add config to decide whether fill schema in BE side or MS side
Cherry-picked from #41482

eldenmoon added dev/3.0.3-merged and removed dev/3.0.x labels

gavinchou added the cir label

gavinchou mentioned this pull request

Release Note 3.0.3 #44522

Closed

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request

[cloud](Variant) limit column size in MetaService and support handle …

1174240

…rowset schema in backend side (apache#41482)
1. in local mode we limit column size in commit_txn phase, in cloud mode
we limit it in `write_schema_dict` which in commit_rowset phase
2. add config to decide whether fill schema in BE side or MS side

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment