This repository was archived by the owner on Mar 31, 2023. It is now read-only.

[Performance Analysis] DPM/ACA gRPC Performance Report - #384

Open
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc
Open

[Performance Analysis] DPM/ACA gRPC Performance Report#384
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc

Conversation

@haboy52581

Copy link
Copy Markdown
Contributor

No description provided.

@xieusxieus changed the title add adoc for dpm aca grpc analysis[Performance Analysis] DPM/ACA gRPC Performance ReportSep 24, 2020
@xieusxieus added the perf testing Performance Testing label Sep 24, 2020
@xieusxieus added this to the Version 0.9.2020.09.30 milestone Sep 24, 2020

@xieusxieus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haboy52581 Some initial comments. Thanks for setting up tests and collecting the data points.

@@ -0,0 +1,215 @@
= ALCOR CONTROL AGENT-ALCOR DATAPLANE MANAGER Test Report

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested to change to "Alcor gRPC Performance Test Report"

|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection
|*Storage* |LSI raid (no ssd) |AVAGO (no ssd)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the DPM machine (.188) has 6X1600GB SSD. Could you confirm?

|*Model Name* |Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz |Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the network bandwidth. As the results shows DPM client is network bounded, so we would need to revisit this configuration.

[arabic, start=2]
. *Test step:*

F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you upload the test scripts or codes that generate the payload to https://github.com/futurewei-cloud/alcor-int/tree/master/tools? This can be done in a sperate PR.


F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

On A-E there are 2600 ACA running on each box, ACA code has been revised to cut off the ovsdb and mq operations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2,600 or 2,000? I thought 2,000 is the stable setup. Need to update the image accordingly.

image::128-2.png["128 thread 2nd time",width=262,height=156]
____

for 256 threads and below, the success rate is 100%

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add one more data point of 256 threads? People will be interested in seeing the limit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can we put some resource utilization diagram including CPU, RAM, Disk IO and Network IO in this extreme case? This would help.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the x-axis, what do those numbers represent? for example, first number is number of threads and the second number is number of successful run out of a total of 10K runs.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as discussed, we need to verify the extreme large value (5,594,098) and rerun the test.



____
* when neighbor number changed, every connection time cost and overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image is important. Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other.

____

____
* when neighbor number changed, overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as to image other-ov-jc.png.

"Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other."

we can take out the data point for "1t-1w" and explain in the texts.


different payload sizes vary from 1 neighbor to 10000 neighbor(2MB) each

*1WR+other OV-MAX+average*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate what this means?

@@ -65,6 +65,28 @@ image::p1.png["Test Deployment",width=488,height=302]
|*90% TILE* |12 |11 |32 |28 |78 |84 |292 |262
|===

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The column and row of this table is opposite of the next one. Could we make them consistent?

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

perf testingPerformance Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@haboy52581@xieus
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
This repository was archived by the owner on Mar 31, 2023. It is now read-only.

[Performance Analysis] DPM/ACA gRPC Performance Report - #384

Open
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc
Open

[Performance Analysis] DPM/ACA gRPC Performance Report#384
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc

Conversation

@haboy52581

Copy link
Copy Markdown
Contributor

No description provided.

@xieusxieus changed the title add adoc for dpm aca grpc analysis[Performance Analysis] DPM/ACA gRPC Performance ReportSep 24, 2020
@xieusxieus added the perf testing Performance Testing label Sep 24, 2020
@xieusxieus added this to the Version 0.9.2020.09.30 milestone Sep 24, 2020

@xieusxieus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haboy52581 Some initial comments. Thanks for setting up tests and collecting the data points.

@@ -0,0 +1,215 @@
= ALCOR CONTROL AGENT-ALCOR DATAPLANE MANAGER Test Report

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested to change to "Alcor gRPC Performance Test Report"

|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection
|*Storage* |LSI raid (no ssd) |AVAGO (no ssd)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the DPM machine (.188) has 6X1600GB SSD. Could you confirm?

|*Model Name* |Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz |Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the network bandwidth. As the results shows DPM client is network bounded, so we would need to revisit this configuration.

[arabic, start=2]
. *Test step:*

F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you upload the test scripts or codes that generate the payload to https://github.com/futurewei-cloud/alcor-int/tree/master/tools? This can be done in a sperate PR.


F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

On A-E there are 2600 ACA running on each box, ACA code has been revised to cut off the ovsdb and mq operations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2,600 or 2,000? I thought 2,000 is the stable setup. Need to update the image accordingly.

image::128-2.png["128 thread 2nd time",width=262,height=156]
____

for 256 threads and below, the success rate is 100%

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add one more data point of 256 threads? People will be interested in seeing the limit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can we put some resource utilization diagram including CPU, RAM, Disk IO and Network IO in this extreme case? This would help.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the x-axis, what do those numbers represent? for example, first number is number of threads and the second number is number of successful run out of a total of 10K runs.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as discussed, we need to verify the extreme large value (5,594,098) and rerun the test.



____
* when neighbor number changed, every connection time cost and overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image is important. Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other.

____

____
* when neighbor number changed, overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as to image other-ov-jc.png.

"Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other."

we can take out the data point for "1t-1w" and explain in the texts.


different payload sizes vary from 1 neighbor to 10000 neighbor(2MB) each

*1WR+other OV-MAX+average*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate what this means?

@@ -65,6 +65,28 @@ image::p1.png["Test Deployment",width=488,height=302]
|*90% TILE* |12 |11 |32 |28 |78 |84 |292 |262
|===

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The column and row of this table is opposite of the next one. Could we make them consistent?

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

perf testingPerformance Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@haboy52581@xieus
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Mar 31, 2023. It is now read-only.

[Performance Analysis] DPM/ACA gRPC Performance Report - #384

Open
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc
Open

[Performance Analysis] DPM/ACA gRPC Performance Report#384
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc

Conversation

@haboy52581

Copy link
Copy Markdown
Contributor

No description provided.

@xieusxieus changed the title add adoc for dpm aca grpc analysis[Performance Analysis] DPM/ACA gRPC Performance ReportSep 24, 2020
@xieusxieus added the perf testing Performance Testing label Sep 24, 2020
@xieusxieus added this to the Version 0.9.2020.09.30 milestone Sep 24, 2020

@xieusxieus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haboy52581 Some initial comments. Thanks for setting up tests and collecting the data points.

@@ -0,0 +1,215 @@
= ALCOR CONTROL AGENT-ALCOR DATAPLANE MANAGER Test Report

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested to change to "Alcor gRPC Performance Test Report"

|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection
|*Storage* |LSI raid (no ssd) |AVAGO (no ssd)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the DPM machine (.188) has 6X1600GB SSD. Could you confirm?

|*Model Name* |Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz |Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the network bandwidth. As the results shows DPM client is network bounded, so we would need to revisit this configuration.

[arabic, start=2]
. *Test step:*

F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you upload the test scripts or codes that generate the payload to https://github.com/futurewei-cloud/alcor-int/tree/master/tools? This can be done in a sperate PR.


F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

On A-E there are 2600 ACA running on each box, ACA code has been revised to cut off the ovsdb and mq operations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2,600 or 2,000? I thought 2,000 is the stable setup. Need to update the image accordingly.

image::128-2.png["128 thread 2nd time",width=262,height=156]
____

for 256 threads and below, the success rate is 100%

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add one more data point of 256 threads? People will be interested in seeing the limit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can we put some resource utilization diagram including CPU, RAM, Disk IO and Network IO in this extreme case? This would help.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the x-axis, what do those numbers represent? for example, first number is number of threads and the second number is number of successful run out of a total of 10K runs.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as discussed, we need to verify the extreme large value (5,594,098) and rerun the test.



____
* when neighbor number changed, every connection time cost and overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image is important. Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other.

____

____
* when neighbor number changed, overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as to image other-ov-jc.png.

"Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other."

we can take out the data point for "1t-1w" and explain in the texts.


different payload sizes vary from 1 neighbor to 10000 neighbor(2MB) each

*1WR+other OV-MAX+average*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate what this means?

@@ -65,6 +65,28 @@ image::p1.png["Test Deployment",width=488,height=302]
|*90% TILE* |12 |11 |32 |28 |78 |84 |292 |262
|===

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The column and row of this table is opposite of the next one. Could we make them consistent?

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

perf testingPerformance Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@haboy52581@xieus
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Mar 31, 2023. It is now read-only.

[Performance Analysis] DPM/ACA gRPC Performance Report - #384

Open
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc
Open

[Performance Analysis] DPM/ACA gRPC Performance Report#384
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc

Conversation

@haboy52581

Copy link
Copy Markdown
Contributor

No description provided.

@xieusxieus changed the title add adoc for dpm aca grpc analysis[Performance Analysis] DPM/ACA gRPC Performance ReportSep 24, 2020
@xieusxieus added the perf testing Performance Testing label Sep 24, 2020
@xieusxieus added this to the Version 0.9.2020.09.30 milestone Sep 24, 2020

@xieusxieus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haboy52581 Some initial comments. Thanks for setting up tests and collecting the data points.

@@ -0,0 +1,215 @@
= ALCOR CONTROL AGENT-ALCOR DATAPLANE MANAGER Test Report

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested to change to "Alcor gRPC Performance Test Report"

|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection
|*Storage* |LSI raid (no ssd) |AVAGO (no ssd)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the DPM machine (.188) has 6X1600GB SSD. Could you confirm?

|*Model Name* |Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz |Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the network bandwidth. As the results shows DPM client is network bounded, so we would need to revisit this configuration.

[arabic, start=2]
. *Test step:*

F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you upload the test scripts or codes that generate the payload to https://github.com/futurewei-cloud/alcor-int/tree/master/tools? This can be done in a sperate PR.


F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

On A-E there are 2600 ACA running on each box, ACA code has been revised to cut off the ovsdb and mq operations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2,600 or 2,000? I thought 2,000 is the stable setup. Need to update the image accordingly.

image::128-2.png["128 thread 2nd time",width=262,height=156]
____

for 256 threads and below, the success rate is 100%

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add one more data point of 256 threads? People will be interested in seeing the limit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can we put some resource utilization diagram including CPU, RAM, Disk IO and Network IO in this extreme case? This would help.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the x-axis, what do those numbers represent? for example, first number is number of threads and the second number is number of successful run out of a total of 10K runs.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as discussed, we need to verify the extreme large value (5,594,098) and rerun the test.



____
* when neighbor number changed, every connection time cost and overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image is important. Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other.

____

____
* when neighbor number changed, overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as to image other-ov-jc.png.

"Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other."

we can take out the data point for "1t-1w" and explain in the texts.


different payload sizes vary from 1 neighbor to 10000 neighbor(2MB) each

*1WR+other OV-MAX+average*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate what this means?

@@ -65,6 +65,28 @@ image::p1.png["Test Deployment",width=488,height=302]
|*90% TILE* |12 |11 |32 |28 |78 |84 |292 |262
|===

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The column and row of this table is opposite of the next one. Could we make them consistent?

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

perf testingPerformance Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@haboy52581@xieus
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
This repository was archived by the owner on Mar 31, 2023. It is now read-only.

[Performance Analysis] DPM/ACA gRPC Performance Report - #384

Open
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc
Open

[Performance Analysis] DPM/ACA gRPC Performance Report#384
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc

Conversation

@haboy52581

Copy link
Copy Markdown
Contributor

No description provided.

@xieusxieus changed the title add adoc for dpm aca grpc analysis[Performance Analysis] DPM/ACA gRPC Performance ReportSep 24, 2020
@xieusxieus added the perf testing Performance Testing label Sep 24, 2020
@xieusxieus added this to the Version 0.9.2020.09.30 milestone Sep 24, 2020

@xieusxieus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haboy52581 Some initial comments. Thanks for setting up tests and collecting the data points.

@@ -0,0 +1,215 @@
= ALCOR CONTROL AGENT-ALCOR DATAPLANE MANAGER Test Report

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested to change to "Alcor gRPC Performance Test Report"

|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection
|*Storage* |LSI raid (no ssd) |AVAGO (no ssd)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the DPM machine (.188) has 6X1600GB SSD. Could you confirm?

|*Model Name* |Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz |Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the network bandwidth. As the results shows DPM client is network bounded, so we would need to revisit this configuration.

[arabic, start=2]
. *Test step:*

F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you upload the test scripts or codes that generate the payload to https://github.com/futurewei-cloud/alcor-int/tree/master/tools? This can be done in a sperate PR.


F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

On A-E there are 2600 ACA running on each box, ACA code has been revised to cut off the ovsdb and mq operations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2,600 or 2,000? I thought 2,000 is the stable setup. Need to update the image accordingly.

image::128-2.png["128 thread 2nd time",width=262,height=156]
____

for 256 threads and below, the success rate is 100%

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add one more data point of 256 threads? People will be interested in seeing the limit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can we put some resource utilization diagram including CPU, RAM, Disk IO and Network IO in this extreme case? This would help.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the x-axis, what do those numbers represent? for example, first number is number of threads and the second number is number of successful run out of a total of 10K runs.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as discussed, we need to verify the extreme large value (5,594,098) and rerun the test.



____
* when neighbor number changed, every connection time cost and overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image is important. Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other.

____

____
* when neighbor number changed, overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as to image other-ov-jc.png.

"Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other."

we can take out the data point for "1t-1w" and explain in the texts.


different payload sizes vary from 1 neighbor to 10000 neighbor(2MB) each

*1WR+other OV-MAX+average*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate what this means?

@@ -65,6 +65,28 @@ image::p1.png["Test Deployment",width=488,height=302]
|*90% TILE* |12 |11 |32 |28 |78 |84 |292 |262
|===

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The column and row of this table is opposite of the next one. Could we make them consistent?

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

perf testingPerformance Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@haboy52581@xieus
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Mar 31, 2023. It is now read-only.

[Performance Analysis] DPM/ACA gRPC Performance Report - #384

Open
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc
Open

[Performance Analysis] DPM/ACA gRPC Performance Report#384
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc

Conversation

@haboy52581

Copy link
Copy Markdown
Contributor

No description provided.

@xieusxieus changed the title add adoc for dpm aca grpc analysis[Performance Analysis] DPM/ACA gRPC Performance ReportSep 24, 2020
@xieusxieus added the perf testing Performance Testing label Sep 24, 2020
@xieusxieus added this to the Version 0.9.2020.09.30 milestone Sep 24, 2020

@xieusxieus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haboy52581 Some initial comments. Thanks for setting up tests and collecting the data points.

@@ -0,0 +1,215 @@
= ALCOR CONTROL AGENT-ALCOR DATAPLANE MANAGER Test Report

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested to change to "Alcor gRPC Performance Test Report"

|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection
|*Storage* |LSI raid (no ssd) |AVAGO (no ssd)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the DPM machine (.188) has 6X1600GB SSD. Could you confirm?

|*Model Name* |Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz |Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the network bandwidth. As the results shows DPM client is network bounded, so we would need to revisit this configuration.

[arabic, start=2]
. *Test step:*

F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you upload the test scripts or codes that generate the payload to https://github.com/futurewei-cloud/alcor-int/tree/master/tools? This can be done in a sperate PR.


F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

On A-E there are 2600 ACA running on each box, ACA code has been revised to cut off the ovsdb and mq operations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2,600 or 2,000? I thought 2,000 is the stable setup. Need to update the image accordingly.

image::128-2.png["128 thread 2nd time",width=262,height=156]
____

for 256 threads and below, the success rate is 100%

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add one more data point of 256 threads? People will be interested in seeing the limit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can we put some resource utilization diagram including CPU, RAM, Disk IO and Network IO in this extreme case? This would help.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the x-axis, what do those numbers represent? for example, first number is number of threads and the second number is number of successful run out of a total of 10K runs.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as discussed, we need to verify the extreme large value (5,594,098) and rerun the test.



____
* when neighbor number changed, every connection time cost and overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image is important. Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other.

____

____
* when neighbor number changed, overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as to image other-ov-jc.png.

"Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other."

we can take out the data point for "1t-1w" and explain in the texts.


different payload sizes vary from 1 neighbor to 10000 neighbor(2MB) each

*1WR+other OV-MAX+average*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate what this means?

@@ -65,6 +65,28 @@ image::p1.png["Test Deployment",width=488,height=302]
|*90% TILE* |12 |11 |32 |28 |78 |84 |292 |262
|===

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The column and row of this table is opposite of the next one. Could we make them consistent?

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

perf testingPerformance Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@haboy52581@xieus
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Mar 31, 2023. It is now read-only.

[Performance Analysis] DPM/ACA gRPC Performance Report - #384

Open
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc
Open

[Performance Analysis] DPM/ACA gRPC Performance Report#384
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc

Conversation

@haboy52581

Copy link
Copy Markdown
Contributor

No description provided.

@xieusxieus changed the title add adoc for dpm aca grpc analysis[Performance Analysis] DPM/ACA gRPC Performance ReportSep 24, 2020
@xieusxieus added the perf testing Performance Testing label Sep 24, 2020
@xieusxieus added this to the Version 0.9.2020.09.30 milestone Sep 24, 2020

@xieusxieus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haboy52581 Some initial comments. Thanks for setting up tests and collecting the data points.

@@ -0,0 +1,215 @@
= ALCOR CONTROL AGENT-ALCOR DATAPLANE MANAGER Test Report

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested to change to "Alcor gRPC Performance Test Report"

|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection
|*Storage* |LSI raid (no ssd) |AVAGO (no ssd)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the DPM machine (.188) has 6X1600GB SSD. Could you confirm?

|*Model Name* |Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz |Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the network bandwidth. As the results shows DPM client is network bounded, so we would need to revisit this configuration.

[arabic, start=2]
. *Test step:*

F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you upload the test scripts or codes that generate the payload to https://github.com/futurewei-cloud/alcor-int/tree/master/tools? This can be done in a sperate PR.


F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

On A-E there are 2600 ACA running on each box, ACA code has been revised to cut off the ovsdb and mq operations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2,600 or 2,000? I thought 2,000 is the stable setup. Need to update the image accordingly.

image::128-2.png["128 thread 2nd time",width=262,height=156]
____

for 256 threads and below, the success rate is 100%

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add one more data point of 256 threads? People will be interested in seeing the limit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can we put some resource utilization diagram including CPU, RAM, Disk IO and Network IO in this extreme case? This would help.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the x-axis, what do those numbers represent? for example, first number is number of threads and the second number is number of successful run out of a total of 10K runs.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as discussed, we need to verify the extreme large value (5,594,098) and rerun the test.



____
* when neighbor number changed, every connection time cost and overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image is important. Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other.

____

____
* when neighbor number changed, overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as to image other-ov-jc.png.

"Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other."

we can take out the data point for "1t-1w" and explain in the texts.


different payload sizes vary from 1 neighbor to 10000 neighbor(2MB) each

*1WR+other OV-MAX+average*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate what this means?

@@ -65,6 +65,28 @@ image::p1.png["Test Deployment",width=488,height=302]
|*90% TILE* |12 |11 |32 |28 |78 |84 |292 |262
|===

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The column and row of this table is opposite of the next one. Could we make them consistent?

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

perf testingPerformance Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@haboy52581@xieus
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
This repository was archived by the owner on Mar 31, 2023. It is now read-only.

[Performance Analysis] DPM/ACA gRPC Performance Report - #384

Open
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc
Open

[Performance Analysis] DPM/ACA gRPC Performance Report#384
haboy52581 wants to merge 11 commits into
futurewei-cloud:masterfrom
haboy52581:feature/add-adoc

Conversation

@haboy52581

Copy link
Copy Markdown
Contributor

No description provided.

@xieusxieus changed the title add adoc for dpm aca grpc analysis[Performance Analysis] DPM/ACA gRPC Performance ReportSep 24, 2020
@xieusxieus added the perf testing Performance Testing label Sep 24, 2020
@xieusxieus added this to the Version 0.9.2020.09.30 milestone Sep 24, 2020

@xieusxieus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haboy52581 Some initial comments. Thanks for setting up tests and collecting the data points.

@@ -0,0 +1,215 @@
= ALCOR CONTROL AGENT-ALCOR DATAPLANE MANAGER Test Report

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested to change to "Alcor gRPC Performance Test Report"

|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection
|*Storage* |LSI raid (no ssd) |AVAGO (no ssd)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the DPM machine (.188) has 6X1600GB SSD. Could you confirm?

|*Model Name* |Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz |Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
|*cpu MHz* |2231.772 |2599.079
|*Memory* |192GB |386GB
|*Network* |NetXtreme BCM5719 Gigabit Ethernet PCIe (GB network) |82599ES 10-Gigabit SFI/SFP+ Network Connection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the network bandwidth. As the results shows DPM client is network bounded, so we would need to revisit this configuration.

[arabic, start=2]
. *Test step:*

F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you upload the test scripts or codes that generate the payload to https://github.com/futurewei-cloud/alcor-int/tree/master/tools? This can be done in a sperate PR.


F send goal state message to A-E at the same time concurrently after first warming up then wait for the response, goal state message is different in each payload

On A-E there are 2600 ACA running on each box, ACA code has been revised to cut off the ovsdb and mq operations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2,600 or 2,000? I thought 2,000 is the stable setup. Need to update the image accordingly.

image::128-2.png["128 thread 2nd time",width=262,height=156]
____

for 256 threads and below, the success rate is 100%

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add one more data point of 256 threads? People will be interested in seeing the limit.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also can we put some resource utilization diagram including CPU, RAM, Disk IO and Network IO in this extreme case? This would help.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain the x-axis, what do those numbers represent? for example, first number is number of threads and the second number is number of successful run out of a total of 10K runs.

____

____
* 10k neighbor, every connection time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, as discussed, we need to verify the extreme large value (5,594,098) and rerun the test.



____
* when neighbor number changed, every connection time cost and overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image is important. Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other.

____

____
* when neighbor number changed, overall time cost for different concurrent thread number*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as to image other-ov-jc.png.

"Let us work to collect more data based on two dimensions (concurrent thread # and neighbor numbers), fix one and adjust the other."

we can take out the data point for "1t-1w" and explain in the texts.


different payload sizes vary from 1 neighbor to 10000 neighbor(2MB) each

*1WR+other OV-MAX+average*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate what this means?

@@ -65,6 +65,28 @@ image::p1.png["Test Deployment",width=488,height=302]
|*90% TILE* |12 |11 |32 |28 |78 |84 |292 |262
|===

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The column and row of this table is opposite of the next one. Could we make them consistent?

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

perf testingPerformance Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@haboy52581@xieus