Skip to content

[profile](rpc) add RpcInstanceDetails profile to debug rpc consumption problem - #43284

Merged
Gabriel39 merged 3 commits into
apache:masterfrom
Mryange:rpc-time-profile
Nov 12, 2024
Merged

[profile](rpc) add RpcInstanceDetails profile to debug rpc consumption problem#43284
Gabriel39 merged 3 commits into
apache:masterfrom
Mryange:rpc-time-profile

Conversation

@Mryange

@MryangeMryange commented Nov 5, 2024•
edited by Gabriel39
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Picked from the 2.1 branch, only the RPC profile-related code was selected.
#39852
#40117

 DATA_STREAM_SINK_OPERATOR (id=2,dst_id=2):
- RpcCount: sum 16, avg 4, max 4, min 4
- RpcMaxTime: avg 1.15ms, max 1.163ms, min 818.493us
- RpcAvgTime: 11.850ms
- RpcCount: 10
- RpcMaxTime: 86.891ms
- RpcMinTime: 15.200ms
- RpcSumTime: 118.503ms
- SerializeBatchTime: 13.517ms
- SplitBlockDistributeByChannelTime: 38.923ms
- SplitBlockHashComputeTime: 2.659ms
- UncompressedRowBatchSize: 135.19 KB
- WaitForDependencyTime: 0ns
- WaitForRpcBufferQueue: 0ns
RpcInstanceDetails:
- Instance 85d4f75b72a9ea61: Count: 4, MaxTime: 36.238ms, MinTime: 12.107ms, AvgTime: 21.722ms, SumTime: 86.891ms
- Instance 85d4f75b72a9ea91: Count: 3, MaxTime: 11.107ms, MinTime: 2.431ms, AvgTime: 5.470ms, SumTime: 16.412ms
- Instance 85d4f75b72a9eac1: Count: 3, MaxTime: 7.554ms, MinTime: 3.160ms, AvgTime: 5.066ms, SumTime: 15.200m

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Check List (For Committer)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No colde files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

  • Release note

    None

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot

doris-robot commented Nov 5, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Mryange

Mryange commented Nov 5, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Nov 5, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@Mryange

Mryange commented Nov 6, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Nov 6, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.85% (9836/25986)
Line Coverage: 29.02% (81811/281909)
Region Coverage: 28.25% (42148/149181)
Branch Coverage: 24.84% (21391/86116)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8797d4171878d3b97522d304a5b898672b5dbdc9_8797d4171878d3b97522d304a5b898672b5dbdc9/report/index.html

@Mryange

Mryange commented Nov 6, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Nov 6, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Nov 6, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.85% (9835/25986)
Line Coverage: 29.02% (81809/281909)
Region Coverage: 28.26% (42154/149181)
Branch Coverage: 24.83% (21384/86116)
Coverage Report: http://coverage.selectdb-in.cc/coverage/db64af31e200cb8a3cd816f76a4bfc0f8ed93113_db64af31e200cb8a3cd816f76a4bfc0f8ed93113/report/index.html

HappenLee
HappenLee reviewed Nov 6, 2024
View reviewed changes
Comment threadbe/src/pipeline/exec/exchange_sink_buffer.cpp Outdated
// attach task for memory tracker and query id when core
SCOPED_ATTACH_TASK(_state);
set_rpc_time(id, start_rpc_time, result.receive_time());
{

@HappenLeeHappenLeeNov 6, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need the {}

HappenLee
HappenLee previously approved these changes Nov 6, 2024
View reviewed changes

@HappenLeeHappenLee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 6, 2024
@github-actions

github-actionsBot commented Nov 6, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Nov 6, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@Mryange

Mryange commented Nov 6, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actionsgithub-actionsBot removed the approved Indicates a PR has been approved by one committer. label Nov 6, 2024
@github-actions

github-actionsBot commented Nov 6, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Nov 6, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.87% (9848/26002)
Line Coverage: 29.03% (81883/282062)
Region Coverage: 28.25% (42167/149253)
Branch Coverage: 24.83% (21394/86150)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e595f7ace16526356f753cae5cbbf2ebb451dd54_e595f7ace16526356f753cae5cbbf2ebb451dd54/report/index.html

@github-actions

github-actionsBot commented Nov 12, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 12, 2024
@Gabriel39
Gabriel39 merged commit 736f1ad into apache:masterNov 12, 2024
zzzxl1993 pushed a commit to zzzxl1993/doris that referenced this pull request Nov 12, 2024
@Mryange@zzzxl1993
[profile](rpc) add RpcInstanceDetails profile to debug rpc consumptio…
79e3297
…n problem (apache#43284)
Picked from the 2.1 branch, only the RPC profile-related code was
selected.
apache#39852apache#40117
```
DATA_STREAM_SINK_OPERATOR (id=2,dst_id=2):
- RpcCount: sum 16, avg 4, max 4, min 4
- RpcMaxTime: avg 1.15ms, max 1.163ms, min 818.493us
- RpcAvgTime: 11.850ms
- RpcCount: 10
- RpcMaxTime: 86.891ms
- RpcMinTime: 15.200ms
- RpcSumTime: 118.503ms
- SerializeBatchTime: 13.517ms
- SplitBlockDistributeByChannelTime: 38.923ms
- SplitBlockHashComputeTime: 2.659ms
- UncompressedRowBatchSize: 135.19 KB
- WaitForDependencyTime: 0ns
- WaitForRpcBufferQueue: 0ns
RpcInstanceDetails:
- Instance 85d4f75b72a9ea61: Count: 4, MaxTime: 36.238ms, MinTime: 12.107ms, AvgTime: 21.722ms, SumTime: 86.891ms
- Instance 85d4f75b72a9ea91: Count: 3, MaxTime: 11.107ms, MinTime: 2.431ms, AvgTime: 5.470ms, SumTime: 16.412ms
- Instance 85d4f75b72a9eac1: Count: 3, MaxTime: 7.554ms, MinTime: 3.160ms, AvgTime: 5.066ms, SumTime: 15.200m
```
py023 pushed a commit to py023/doris that referenced this pull request Nov 13, 2024
@Mryange@py023
[profile](rpc) add RpcInstanceDetails profile to debug rpc consumptio…
a31a941
…n problem (apache#43284)
Picked from the 2.1 branch, only the RPC profile-related code was
selected.
apache#39852apache#40117
```
DATA_STREAM_SINK_OPERATOR (id=2,dst_id=2):
- RpcCount: sum 16, avg 4, max 4, min 4
- RpcMaxTime: avg 1.15ms, max 1.163ms, min 818.493us
- RpcAvgTime: 11.850ms
- RpcCount: 10
- RpcMaxTime: 86.891ms
- RpcMinTime: 15.200ms
- RpcSumTime: 118.503ms
- SerializeBatchTime: 13.517ms
- SplitBlockDistributeByChannelTime: 38.923ms
- SplitBlockHashComputeTime: 2.659ms
- UncompressedRowBatchSize: 135.19 KB
- WaitForDependencyTime: 0ns
- WaitForRpcBufferQueue: 0ns
RpcInstanceDetails:
- Instance 85d4f75b72a9ea61: Count: 4, MaxTime: 36.238ms, MinTime: 12.107ms, AvgTime: 21.722ms, SumTime: 86.891ms
- Instance 85d4f75b72a9ea91: Count: 3, MaxTime: 11.107ms, MinTime: 2.431ms, AvgTime: 5.470ms, SumTime: 16.412ms
- Instance 85d4f75b72a9eac1: Count: 3, MaxTime: 7.554ms, MinTime: 3.160ms, AvgTime: 5.066ms, SumTime: 15.200m
```
Mryange added a commit to Mryange/doris that referenced this pull request Dec 4, 2024
@Mryange
[profile](rpc) add RpcInstanceDetails profile to debug rpc consumptio…
f12669d
…n problem (apache#43284)
Picked from the 2.1 branch, only the RPC profile-related code was
selected.
apache#39852apache#40117
```
DATA_STREAM_SINK_OPERATOR (id=2,dst_id=2):
- RpcCount: sum 16, avg 4, max 4, min 4
- RpcMaxTime: avg 1.15ms, max 1.163ms, min 818.493us
- RpcAvgTime: 11.850ms
- RpcCount: 10
- RpcMaxTime: 86.891ms
- RpcMinTime: 15.200ms
- RpcSumTime: 118.503ms
- SerializeBatchTime: 13.517ms
- SplitBlockDistributeByChannelTime: 38.923ms
- SplitBlockHashComputeTime: 2.659ms
- UncompressedRowBatchSize: 135.19 KB
- WaitForDependencyTime: 0ns
- WaitForRpcBufferQueue: 0ns
RpcInstanceDetails:
- Instance 85d4f75b72a9ea61: Count: 4, MaxTime: 36.238ms, MinTime: 12.107ms, AvgTime: 21.722ms, SumTime: 86.891ms
- Instance 85d4f75b72a9ea91: Count: 3, MaxTime: 11.107ms, MinTime: 2.431ms, AvgTime: 5.470ms, SumTime: 16.412ms
- Instance 85d4f75b72a9eac1: Count: 3, MaxTime: 7.554ms, MinTime: 3.160ms, AvgTime: 5.066ms, SumTime: 15.200m
```
@MryangeMryange mentioned this pull request Dec 4, 2024
Closed
16 tasks
Mryange added a commit to Mryange/doris that referenced this pull request Jan 10, 2025
@Mryange
[profile](rpc) add RpcInstanceDetails profile to debug rpc consumptio…
f770be1
…n problem (apache#43284)
Picked from the 2.1 branch, only the RPC profile-related code was
selected.
apache#39852apache#40117
```
DATA_STREAM_SINK_OPERATOR (id=2,dst_id=2):
- RpcCount: sum 16, avg 4, max 4, min 4
- RpcMaxTime: avg 1.15ms, max 1.163ms, min 818.493us
- RpcAvgTime: 11.850ms
- RpcCount: 10
- RpcMaxTime: 86.891ms
- RpcMinTime: 15.200ms
- RpcSumTime: 118.503ms
- SerializeBatchTime: 13.517ms
- SplitBlockDistributeByChannelTime: 38.923ms
- SplitBlockHashComputeTime: 2.659ms
- UncompressedRowBatchSize: 135.19 KB
- WaitForDependencyTime: 0ns
- WaitForRpcBufferQueue: 0ns
RpcInstanceDetails:
- Instance 85d4f75b72a9ea61: Count: 4, MaxTime: 36.238ms, MinTime: 12.107ms, AvgTime: 21.722ms, SumTime: 86.891ms
- Instance 85d4f75b72a9ea91: Count: 3, MaxTime: 11.107ms, MinTime: 2.431ms, AvgTime: 5.470ms, SumTime: 16.412ms
- Instance 85d4f75b72a9eac1: Count: 3, MaxTime: 7.554ms, MinTime: 3.160ms, AvgTime: 5.066ms, SumTime: 15.200m
```
@MryangeMryange mentioned this pull request Jan 10, 2025
Closed
16 tasks
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@Mryange@HappenLee
[profile](rpc) add RpcInstanceDetails profile to debug rpc consumptio…
b783ebf
…n problem (apache#43284)
Picked from the 2.1 branch, only the RPC profile-related code was
selected.
apache#39852apache#40117
```
DATA_STREAM_SINK_OPERATOR (id=2,dst_id=2):
- RpcCount: sum 16, avg 4, max 4, min 4
- RpcMaxTime: avg 1.15ms, max 1.163ms, min 818.493us
- RpcAvgTime: 11.850ms
- RpcCount: 10
- RpcMaxTime: 86.891ms
- RpcMinTime: 15.200ms
- RpcSumTime: 118.503ms
- SerializeBatchTime: 13.517ms
- SplitBlockDistributeByChannelTime: 38.923ms
- SplitBlockHashComputeTime: 2.659ms
- UncompressedRowBatchSize: 135.19 KB
- WaitForDependencyTime: 0ns
- WaitForRpcBufferQueue: 0ns
RpcInstanceDetails:
- Instance 85d4f75b72a9ea61: Count: 4, MaxTime: 36.238ms, MinTime: 12.107ms, AvgTime: 21.722ms, SumTime: 86.891ms
- Instance 85d4f75b72a9ea91: Count: 3, MaxTime: 11.107ms, MinTime: 2.431ms, AvgTime: 5.470ms, SumTime: 16.412ms
- Instance 85d4f75b72a9eac1: Count: 3, MaxTime: 7.554ms, MinTime: 3.160ms, AvgTime: 5.066ms, SumTime: 15.200m
```
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jacktenggjacktenggjacktengg approved these changes
@Gabriel39Gabriel39Gabriel39 approved these changes
@HappenLeeHappenLeeHappenLee left review comments

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.6-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Mryange@doris-robot@HappenLee@jacktengg@Gabriel39@yiguolei