Skip to content

[Improvement](segment iterator) Optimize column row reservation to reduce overhead - #42060

Merged
airborne12 merged 3 commits into
apache:masterfrom
airborne12:improve-perf
Oct 22, 2024
Merged

[Improvement](segment iterator) Optimize column row reservation to reduce overhead#42060
airborne12 merged 3 commits into
apache:masterfrom
airborne12:improve-perf

Conversation

@airborne12

@airborne12airborne12 commented Oct 18, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Proposed changes

This PR improves the segment iterator by reducing the reserved rows for a column when the row bitmap is smaller than the block row. This optimization aims to enhance memory efficiency and improve performance by reducing unnecessary row allocations.

@doris-robot

doris-robot commented Oct 18, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@airborne12airborne12 changed the title [Improve](segment iterator) reserve less rows for column when row bit…[Improvement](segment iterator) Optimize column row reservation to reduce overheadOct 18, 2024
@github-actions

github-actionsBot commented Oct 18, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@airborne12

airborne12 commented Oct 18, 2024

Copy link
Copy Markdown
MemberAuthor

run buildall

@doris-robot

doris-robot commented Oct 18, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.47% (9711/25919)
Line Coverage: 28.74% (80642/280617)
Region Coverage: 28.17% (41713/148088)
Branch Coverage: 24.75% (21209/85700)
Coverage Report: http://coverage.selectdb-in.cc/coverage/820e3984e22499a150acfa8a322cfd59cfc815cc_820e3984e22499a150acfa8a322cfd59cfc815cc/report/index.html

eldenmoon
eldenmoon reviewed Oct 18, 2024
View reviewed changes
Comment threadbe/src/olap/rowset/segment_v2/segment_iterator.cpp Outdated
@airborne12

airborne12 commented Oct 21, 2024

Copy link
Copy Markdown
MemberAuthor

run buildall

@github-actions

github-actionsBot commented Oct 21, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Oct 21, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.47% (9708/25912)
Line Coverage: 28.75% (80651/280523)
Region Coverage: 28.18% (41718/148045)
Branch Coverage: 24.77% (21213/85656)
Coverage Report: http://coverage.selectdb-in.cc/coverage/48b8cd544d27ec4cc4715d0b7631a9a4f7fe57df_48b8cd544d27ec4cc4715d0b7631a9a4f7fe57df/report/index.html

@github-actions

github-actionsBot commented Oct 22, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@airborne12

airborne12 commented Oct 22, 2024

Copy link
Copy Markdown
MemberAuthor

run buildall

@doris-robot

doris-robot commented Oct 22, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.45% (9703/25907)
Line Coverage: 28.71% (80547/280527)
Region Coverage: 28.14% (41656/148014)
Branch Coverage: 24.72% (21168/85644)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fb2fc0ee44681e1b5ec219f5009c1c7431dd82c8_fb2fc0ee44681e1b5ec219f5009c1c7431dd82c8/report/index.html

zzzxl1993
zzzxl1993 approved these changes Oct 22, 2024
View reviewed changes

@zzzxl1993zzzxl1993 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actionsBot commented Oct 22, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

qidaye
qidaye approved these changes Oct 22, 2024
View reviewed changes

@qidayeqidaye left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actionsBot commented Oct 22, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 22, 2024
@airborne12

airborne12 commented Oct 22, 2024

Copy link
Copy Markdown
MemberAuthor

run p1

@airborne12

airborne12 commented Oct 22, 2024

Copy link
Copy Markdown
MemberAuthor

run cloud_p1

@airborne12
airborne12 merged commit 8653616 into apache:masterOct 22, 2024
@airborne12
airborne12 deleted the improve-perf branch October 22, 2024 10:53
airborne12 added a commit to airborne12/apache-doris that referenced this pull request Oct 24, 2024
@airborne12
[Improvement](segment iterator) Optimize column row reservation to re…
b4b1b2c
…duce overhead (apache#42060)
## Proposed changes
This PR improves the segment iterator by reducing the reserved rows for
a column when the row bitmap is smaller than the block row. This
optimization aims to enhance memory efficiency and improve performance
by reducing unnecessary row allocations.
@airborne12airborne12 mentioned this pull request Oct 24, 2024
Merged
airborne12 added a commit that referenced this pull request Oct 24, 2024
@airborne12
[Improvement](segment iterator) Optimize column row reservation to re…
f96ac18
…duce overhead #42060 (#42372)
cherry pick from #42060
@gavinchougavinchou mentioned this pull request Nov 26, 2024
Closed
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@airborne12@HappenLee
[Improvement](segment iterator) Optimize column row reservation to re…
ff0713b
…duce overhead (apache#42060)
## Proposed changes
This PR improves the segment iterator by reducing the reserved rows for
a column when the row bitmap is smaller than the block row. This
optimization aims to enhance memory efficiency and improve performance
by reducing unnecessary row allocations.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eldenmooneldenmooneldenmoon left review comments
@qidayeqidayeqidaye approved these changes
@zzzxl1993zzzxl1993zzzxl1993 approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/3.0.3-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@airborne12@doris-robot@qidaye@zzzxl1993@eldenmoon