[MetaSchedule] Restore num_threads parameter in tuning API - #13561

Merged
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage
Dec 9, 2022
Merged

[MetaSchedule] Restore num_threads parameter in tuning API #13561
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage

Conversation

@masahi

@masahimasahi commented Dec 6, 2022

Copy link
Copy Markdown
Member

num_threads parameter in the Relay tuning API was (accidentally?) removed in #12895. This PR restores this parameter and also uses it consistently for XGB model training and builder / runner as well.

@zxybazh

@tvm-bot

tvm-bot commented Dec 6, 2022

Copy link
Copy Markdown
Collaborator

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@junrushao

Copy link
Copy Markdown
Member

I am not sure it should be a top-level parameter as it’s less frequently used by introductory-level users, but don’t have much strong opinions.

In the meantime, I’m not sure if using the same num_threads for xgb and for evo search is a good idea because I assume xgb prefers physical cores? I don’t have numbers handy so I’m not 100% sure.

Removed this because @spectrometerHBH@Hzfengsy@jinhongyii@MasterJH5574 suggested to, so I’d love to hear more from you guys.

@spectrometerHBH

spectrometerHBH commented Dec 6, 2022

Copy link
Copy Markdown
Contributor

I would suggest using a better name because num_threads can be confusing, especially when the TIR programs to be tuned are using parallelization.

@masahi

masahi commented Dec 6, 2022

Copy link
Copy Markdown
MemberAuthor

The main use case of this param is for tuning on a high-core system shared by many users. Currently if one user starts tuning, it occupies all CPU resources, which disrupts other users. So the goal is to limit the number of cores used by MS throughout the tuning process (evo search, post order apply, XGB training, builder / runner).

Also, tune_tir API has num_threads param as well.

num_threads: Union[Literal["physical", "logical"], int] ="physical",

I would suggest using a better name because num_threads can be confusing

Agreed, but that's what TuneContext calls... I can replace num_threads in the high-level API with max_workers or something, and initialize TuneContext by num_threads=max_workers. If people think this is better I can do that, otherwise I'd keep the existing convention.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kinda in favor of this change because it allows more customization. We may want to find a good naming and default value for num_threads though.

Comment threadpython/tvm/meta_schedule/relay_integration.py Outdated
Comment threadtests/python/contrib/test_hexagon/metaschedule_e2e/test_resnet50_int8.py Outdated
@junrushao

Copy link
Copy Markdown
Member

I don't have a clear idea (very bad at naming). Perhaps @tqchen you could suggest?

@masahi

masahi commented Dec 7, 2022

Copy link
Copy Markdown
MemberAuthor

I'm going to go with num_tuning_cores per discussion with @zxybazh, if there is no objection. For now I'll keep num_threads in TuneContext.

@junrushao

Copy link
Copy Markdown
Member

The reason that I don’t like “cores” is that they are threads, which are not necessarily related to physical cpu cores. Perhaps num_threads makes more sense at the moment.

@zxybazh

Copy link
Copy Markdown
Member

Just want to add that Runner & Builder are using new processes, that's why we may want to consider using other terminology. On the other hand, if we don't have very good idea on new naming, I don't mind just sticking to the current naming nun_threads before that.

@masahi

Copy link
Copy Markdown
MemberAuthor

I think "cores" is better because we are also using this parameter to set the number of workers used by builder / runner. Also, when a user wants to limit the amount of CPU resources used by MS, they would think in terms of the number of "cores", not "threads". So as an API, "core" sounds more intuitive to me.

@junrushao

Copy link
Copy Markdown
Member

The argument that builder/runner workers are using processes rather than threads makes sense to me. Thanks for the explanation! Then I'm happy with the "core" terminology

@masahi

Copy link
Copy Markdown
MemberAuthor

Replaced num_threads with num_tuning_cores in the API.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@junrushaojunrushao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masahi
masahi merged commit 3b001ef into apache:mainDec 9, 2022
fzi-peccia pushed a commit to fzi-peccia/tvm that referenced this pull request Mar 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@masahi@tvm-bot@junrushao@spectrometerHBH@zxybazh@shingjan
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

[MetaSchedule] Restore num_threads parameter in tuning API - #13561

Merged
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage
Dec 9, 2022
Merged

[MetaSchedule] Restore num_threads parameter in tuning API #13561
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage

Conversation

@masahi

@masahimasahi commented Dec 6, 2022

Copy link
Copy Markdown
Member

num_threads parameter in the Relay tuning API was (accidentally?) removed in #12895. This PR restores this parameter and also uses it consistently for XGB model training and builder / runner as well.

@zxybazh

@tvm-bot

tvm-bot commented Dec 6, 2022

Copy link
Copy Markdown
Collaborator

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@junrushao

Copy link
Copy Markdown
Member

I am not sure it should be a top-level parameter as it’s less frequently used by introductory-level users, but don’t have much strong opinions.

In the meantime, I’m not sure if using the same num_threads for xgb and for evo search is a good idea because I assume xgb prefers physical cores? I don’t have numbers handy so I’m not 100% sure.

Removed this because @spectrometerHBH@Hzfengsy@jinhongyii@MasterJH5574 suggested to, so I’d love to hear more from you guys.

@spectrometerHBH

spectrometerHBH commented Dec 6, 2022

Copy link
Copy Markdown
Contributor

I would suggest using a better name because num_threads can be confusing, especially when the TIR programs to be tuned are using parallelization.

@masahi

masahi commented Dec 6, 2022

Copy link
Copy Markdown
MemberAuthor

The main use case of this param is for tuning on a high-core system shared by many users. Currently if one user starts tuning, it occupies all CPU resources, which disrupts other users. So the goal is to limit the number of cores used by MS throughout the tuning process (evo search, post order apply, XGB training, builder / runner).

Also, tune_tir API has num_threads param as well.

num_threads: Union[Literal["physical", "logical"], int] ="physical",

I would suggest using a better name because num_threads can be confusing

Agreed, but that's what TuneContext calls... I can replace num_threads in the high-level API with max_workers or something, and initialize TuneContext by num_threads=max_workers. If people think this is better I can do that, otherwise I'd keep the existing convention.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kinda in favor of this change because it allows more customization. We may want to find a good naming and default value for num_threads though.

Comment threadpython/tvm/meta_schedule/relay_integration.py Outdated
Comment threadtests/python/contrib/test_hexagon/metaschedule_e2e/test_resnet50_int8.py Outdated
@junrushao

Copy link
Copy Markdown
Member

I don't have a clear idea (very bad at naming). Perhaps @tqchen you could suggest?

@masahi

masahi commented Dec 7, 2022

Copy link
Copy Markdown
MemberAuthor

I'm going to go with num_tuning_cores per discussion with @zxybazh, if there is no objection. For now I'll keep num_threads in TuneContext.

@junrushao

Copy link
Copy Markdown
Member

The reason that I don’t like “cores” is that they are threads, which are not necessarily related to physical cpu cores. Perhaps num_threads makes more sense at the moment.

@zxybazh

Copy link
Copy Markdown
Member

Just want to add that Runner & Builder are using new processes, that's why we may want to consider using other terminology. On the other hand, if we don't have very good idea on new naming, I don't mind just sticking to the current naming nun_threads before that.

@masahi

Copy link
Copy Markdown
MemberAuthor

I think "cores" is better because we are also using this parameter to set the number of workers used by builder / runner. Also, when a user wants to limit the amount of CPU resources used by MS, they would think in terms of the number of "cores", not "threads". So as an API, "core" sounds more intuitive to me.

@junrushao

Copy link
Copy Markdown
Member

The argument that builder/runner workers are using processes rather than threads makes sense to me. Thanks for the explanation! Then I'm happy with the "core" terminology

@masahi

Copy link
Copy Markdown
MemberAuthor

Replaced num_threads with num_tuning_cores in the API.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@junrushaojunrushao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masahi
masahi merged commit 3b001ef into apache:mainDec 9, 2022
fzi-peccia pushed a commit to fzi-peccia/tvm that referenced this pull request Mar 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@masahi@tvm-bot@junrushao@spectrometerHBH@zxybazh@shingjan
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[MetaSchedule] Restore num_threads parameter in tuning API - #13561

Merged
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage
Dec 9, 2022
Merged

[MetaSchedule] Restore num_threads parameter in tuning API #13561
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage

Conversation

@masahi

@masahimasahi commented Dec 6, 2022

Copy link
Copy Markdown
Member

num_threads parameter in the Relay tuning API was (accidentally?) removed in #12895. This PR restores this parameter and also uses it consistently for XGB model training and builder / runner as well.

@zxybazh

@tvm-bot

tvm-bot commented Dec 6, 2022

Copy link
Copy Markdown
Collaborator

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@junrushao

Copy link
Copy Markdown
Member

I am not sure it should be a top-level parameter as it’s less frequently used by introductory-level users, but don’t have much strong opinions.

In the meantime, I’m not sure if using the same num_threads for xgb and for evo search is a good idea because I assume xgb prefers physical cores? I don’t have numbers handy so I’m not 100% sure.

Removed this because @spectrometerHBH@Hzfengsy@jinhongyii@MasterJH5574 suggested to, so I’d love to hear more from you guys.

@spectrometerHBH

spectrometerHBH commented Dec 6, 2022

Copy link
Copy Markdown
Contributor

I would suggest using a better name because num_threads can be confusing, especially when the TIR programs to be tuned are using parallelization.

@masahi

masahi commented Dec 6, 2022

Copy link
Copy Markdown
MemberAuthor

The main use case of this param is for tuning on a high-core system shared by many users. Currently if one user starts tuning, it occupies all CPU resources, which disrupts other users. So the goal is to limit the number of cores used by MS throughout the tuning process (evo search, post order apply, XGB training, builder / runner).

Also, tune_tir API has num_threads param as well.

num_threads: Union[Literal["physical", "logical"], int] ="physical",

I would suggest using a better name because num_threads can be confusing

Agreed, but that's what TuneContext calls... I can replace num_threads in the high-level API with max_workers or something, and initialize TuneContext by num_threads=max_workers. If people think this is better I can do that, otherwise I'd keep the existing convention.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kinda in favor of this change because it allows more customization. We may want to find a good naming and default value for num_threads though.

Comment threadpython/tvm/meta_schedule/relay_integration.py Outdated
Comment threadtests/python/contrib/test_hexagon/metaschedule_e2e/test_resnet50_int8.py Outdated
@junrushao

Copy link
Copy Markdown
Member

I don't have a clear idea (very bad at naming). Perhaps @tqchen you could suggest?

@masahi

masahi commented Dec 7, 2022

Copy link
Copy Markdown
MemberAuthor

I'm going to go with num_tuning_cores per discussion with @zxybazh, if there is no objection. For now I'll keep num_threads in TuneContext.

@junrushao

Copy link
Copy Markdown
Member

The reason that I don’t like “cores” is that they are threads, which are not necessarily related to physical cpu cores. Perhaps num_threads makes more sense at the moment.

@zxybazh

Copy link
Copy Markdown
Member

Just want to add that Runner & Builder are using new processes, that's why we may want to consider using other terminology. On the other hand, if we don't have very good idea on new naming, I don't mind just sticking to the current naming nun_threads before that.

@masahi

Copy link
Copy Markdown
MemberAuthor

I think "cores" is better because we are also using this parameter to set the number of workers used by builder / runner. Also, when a user wants to limit the amount of CPU resources used by MS, they would think in terms of the number of "cores", not "threads". So as an API, "core" sounds more intuitive to me.

@junrushao

Copy link
Copy Markdown
Member

The argument that builder/runner workers are using processes rather than threads makes sense to me. Thanks for the explanation! Then I'm happy with the "core" terminology

@masahi

Copy link
Copy Markdown
MemberAuthor

Replaced num_threads with num_tuning_cores in the API.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@junrushaojunrushao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masahi
masahi merged commit 3b001ef into apache:mainDec 9, 2022
fzi-peccia pushed a commit to fzi-peccia/tvm that referenced this pull request Mar 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@masahi@tvm-bot@junrushao@spectrometerHBH@zxybazh@shingjan
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[MetaSchedule] Restore num_threads parameter in tuning API - #13561

Merged
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage
Dec 9, 2022
Merged

[MetaSchedule] Restore num_threads parameter in tuning API #13561
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage

Conversation

@masahi

@masahimasahi commented Dec 6, 2022

Copy link
Copy Markdown
Member

num_threads parameter in the Relay tuning API was (accidentally?) removed in #12895. This PR restores this parameter and also uses it consistently for XGB model training and builder / runner as well.

@zxybazh

@tvm-bot

tvm-bot commented Dec 6, 2022

Copy link
Copy Markdown
Collaborator

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@junrushao

Copy link
Copy Markdown
Member

I am not sure it should be a top-level parameter as it’s less frequently used by introductory-level users, but don’t have much strong opinions.

In the meantime, I’m not sure if using the same num_threads for xgb and for evo search is a good idea because I assume xgb prefers physical cores? I don’t have numbers handy so I’m not 100% sure.

Removed this because @spectrometerHBH@Hzfengsy@jinhongyii@MasterJH5574 suggested to, so I’d love to hear more from you guys.

@spectrometerHBH

spectrometerHBH commented Dec 6, 2022

Copy link
Copy Markdown
Contributor

I would suggest using a better name because num_threads can be confusing, especially when the TIR programs to be tuned are using parallelization.

@masahi

masahi commented Dec 6, 2022

Copy link
Copy Markdown
MemberAuthor

The main use case of this param is for tuning on a high-core system shared by many users. Currently if one user starts tuning, it occupies all CPU resources, which disrupts other users. So the goal is to limit the number of cores used by MS throughout the tuning process (evo search, post order apply, XGB training, builder / runner).

Also, tune_tir API has num_threads param as well.

num_threads: Union[Literal["physical", "logical"], int] ="physical",

I would suggest using a better name because num_threads can be confusing

Agreed, but that's what TuneContext calls... I can replace num_threads in the high-level API with max_workers or something, and initialize TuneContext by num_threads=max_workers. If people think this is better I can do that, otherwise I'd keep the existing convention.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kinda in favor of this change because it allows more customization. We may want to find a good naming and default value for num_threads though.

Comment threadpython/tvm/meta_schedule/relay_integration.py Outdated
Comment threadtests/python/contrib/test_hexagon/metaschedule_e2e/test_resnet50_int8.py Outdated
@junrushao

Copy link
Copy Markdown
Member

I don't have a clear idea (very bad at naming). Perhaps @tqchen you could suggest?

@masahi

masahi commented Dec 7, 2022

Copy link
Copy Markdown
MemberAuthor

I'm going to go with num_tuning_cores per discussion with @zxybazh, if there is no objection. For now I'll keep num_threads in TuneContext.

@junrushao

Copy link
Copy Markdown
Member

The reason that I don’t like “cores” is that they are threads, which are not necessarily related to physical cpu cores. Perhaps num_threads makes more sense at the moment.

@zxybazh

Copy link
Copy Markdown
Member

Just want to add that Runner & Builder are using new processes, that's why we may want to consider using other terminology. On the other hand, if we don't have very good idea on new naming, I don't mind just sticking to the current naming nun_threads before that.

@masahi

Copy link
Copy Markdown
MemberAuthor

I think "cores" is better because we are also using this parameter to set the number of workers used by builder / runner. Also, when a user wants to limit the amount of CPU resources used by MS, they would think in terms of the number of "cores", not "threads". So as an API, "core" sounds more intuitive to me.

@junrushao

Copy link
Copy Markdown
Member

The argument that builder/runner workers are using processes rather than threads makes sense to me. Thanks for the explanation! Then I'm happy with the "core" terminology

@masahi

Copy link
Copy Markdown
MemberAuthor

Replaced num_threads with num_tuning_cores in the API.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@junrushaojunrushao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masahi
masahi merged commit 3b001ef into apache:mainDec 9, 2022
fzi-peccia pushed a commit to fzi-peccia/tvm that referenced this pull request Mar 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@masahi@tvm-bot@junrushao@spectrometerHBH@zxybazh@shingjan
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

[MetaSchedule] Restore num_threads parameter in tuning API - #13561

Merged
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage
Dec 9, 2022
Merged

[MetaSchedule] Restore num_threads parameter in tuning API #13561
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage

Conversation

@masahi

@masahimasahi commented Dec 6, 2022

Copy link
Copy Markdown
Member

num_threads parameter in the Relay tuning API was (accidentally?) removed in #12895. This PR restores this parameter and also uses it consistently for XGB model training and builder / runner as well.

@zxybazh

@tvm-bot

tvm-bot commented Dec 6, 2022

Copy link
Copy Markdown
Collaborator

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@junrushao

Copy link
Copy Markdown
Member

I am not sure it should be a top-level parameter as it’s less frequently used by introductory-level users, but don’t have much strong opinions.

In the meantime, I’m not sure if using the same num_threads for xgb and for evo search is a good idea because I assume xgb prefers physical cores? I don’t have numbers handy so I’m not 100% sure.

Removed this because @spectrometerHBH@Hzfengsy@jinhongyii@MasterJH5574 suggested to, so I’d love to hear more from you guys.

@spectrometerHBH

spectrometerHBH commented Dec 6, 2022

Copy link
Copy Markdown
Contributor

I would suggest using a better name because num_threads can be confusing, especially when the TIR programs to be tuned are using parallelization.

@masahi

masahi commented Dec 6, 2022

Copy link
Copy Markdown
MemberAuthor

The main use case of this param is for tuning on a high-core system shared by many users. Currently if one user starts tuning, it occupies all CPU resources, which disrupts other users. So the goal is to limit the number of cores used by MS throughout the tuning process (evo search, post order apply, XGB training, builder / runner).

Also, tune_tir API has num_threads param as well.

num_threads: Union[Literal["physical", "logical"], int] ="physical",

I would suggest using a better name because num_threads can be confusing

Agreed, but that's what TuneContext calls... I can replace num_threads in the high-level API with max_workers or something, and initialize TuneContext by num_threads=max_workers. If people think this is better I can do that, otherwise I'd keep the existing convention.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kinda in favor of this change because it allows more customization. We may want to find a good naming and default value for num_threads though.

Comment threadpython/tvm/meta_schedule/relay_integration.py Outdated
Comment threadtests/python/contrib/test_hexagon/metaschedule_e2e/test_resnet50_int8.py Outdated
@junrushao

Copy link
Copy Markdown
Member

I don't have a clear idea (very bad at naming). Perhaps @tqchen you could suggest?

@masahi

masahi commented Dec 7, 2022

Copy link
Copy Markdown
MemberAuthor

I'm going to go with num_tuning_cores per discussion with @zxybazh, if there is no objection. For now I'll keep num_threads in TuneContext.

@junrushao

Copy link
Copy Markdown
Member

The reason that I don’t like “cores” is that they are threads, which are not necessarily related to physical cpu cores. Perhaps num_threads makes more sense at the moment.

@zxybazh

Copy link
Copy Markdown
Member

Just want to add that Runner & Builder are using new processes, that's why we may want to consider using other terminology. On the other hand, if we don't have very good idea on new naming, I don't mind just sticking to the current naming nun_threads before that.

@masahi

Copy link
Copy Markdown
MemberAuthor

I think "cores" is better because we are also using this parameter to set the number of workers used by builder / runner. Also, when a user wants to limit the amount of CPU resources used by MS, they would think in terms of the number of "cores", not "threads". So as an API, "core" sounds more intuitive to me.

@junrushao

Copy link
Copy Markdown
Member

The argument that builder/runner workers are using processes rather than threads makes sense to me. Thanks for the explanation! Then I'm happy with the "core" terminology

@masahi

Copy link
Copy Markdown
MemberAuthor

Replaced num_threads with num_tuning_cores in the API.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@junrushaojunrushao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masahi
masahi merged commit 3b001ef into apache:mainDec 9, 2022
fzi-peccia pushed a commit to fzi-peccia/tvm that referenced this pull request Mar 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@masahi@tvm-bot@junrushao@spectrometerHBH@zxybazh@shingjan
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[MetaSchedule] Restore num_threads parameter in tuning API - #13561

Merged
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage
Dec 9, 2022
Merged

[MetaSchedule] Restore num_threads parameter in tuning API #13561
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage

Conversation

@masahi

@masahimasahi commented Dec 6, 2022

Copy link
Copy Markdown
Member

num_threads parameter in the Relay tuning API was (accidentally?) removed in #12895. This PR restores this parameter and also uses it consistently for XGB model training and builder / runner as well.

@zxybazh

@tvm-bot

tvm-bot commented Dec 6, 2022

Copy link
Copy Markdown
Collaborator

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@junrushao

Copy link
Copy Markdown
Member

I am not sure it should be a top-level parameter as it’s less frequently used by introductory-level users, but don’t have much strong opinions.

In the meantime, I’m not sure if using the same num_threads for xgb and for evo search is a good idea because I assume xgb prefers physical cores? I don’t have numbers handy so I’m not 100% sure.

Removed this because @spectrometerHBH@Hzfengsy@jinhongyii@MasterJH5574 suggested to, so I’d love to hear more from you guys.

@spectrometerHBH

spectrometerHBH commented Dec 6, 2022

Copy link
Copy Markdown
Contributor

I would suggest using a better name because num_threads can be confusing, especially when the TIR programs to be tuned are using parallelization.

@masahi

masahi commented Dec 6, 2022

Copy link
Copy Markdown
MemberAuthor

The main use case of this param is for tuning on a high-core system shared by many users. Currently if one user starts tuning, it occupies all CPU resources, which disrupts other users. So the goal is to limit the number of cores used by MS throughout the tuning process (evo search, post order apply, XGB training, builder / runner).

Also, tune_tir API has num_threads param as well.

num_threads: Union[Literal["physical", "logical"], int] ="physical",

I would suggest using a better name because num_threads can be confusing

Agreed, but that's what TuneContext calls... I can replace num_threads in the high-level API with max_workers or something, and initialize TuneContext by num_threads=max_workers. If people think this is better I can do that, otherwise I'd keep the existing convention.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kinda in favor of this change because it allows more customization. We may want to find a good naming and default value for num_threads though.

Comment threadpython/tvm/meta_schedule/relay_integration.py Outdated
Comment threadtests/python/contrib/test_hexagon/metaschedule_e2e/test_resnet50_int8.py Outdated
@junrushao

Copy link
Copy Markdown
Member

I don't have a clear idea (very bad at naming). Perhaps @tqchen you could suggest?

@masahi

masahi commented Dec 7, 2022

Copy link
Copy Markdown
MemberAuthor

I'm going to go with num_tuning_cores per discussion with @zxybazh, if there is no objection. For now I'll keep num_threads in TuneContext.

@junrushao

Copy link
Copy Markdown
Member

The reason that I don’t like “cores” is that they are threads, which are not necessarily related to physical cpu cores. Perhaps num_threads makes more sense at the moment.

@zxybazh

Copy link
Copy Markdown
Member

Just want to add that Runner & Builder are using new processes, that's why we may want to consider using other terminology. On the other hand, if we don't have very good idea on new naming, I don't mind just sticking to the current naming nun_threads before that.

@masahi

Copy link
Copy Markdown
MemberAuthor

I think "cores" is better because we are also using this parameter to set the number of workers used by builder / runner. Also, when a user wants to limit the amount of CPU resources used by MS, they would think in terms of the number of "cores", not "threads". So as an API, "core" sounds more intuitive to me.

@junrushao

Copy link
Copy Markdown
Member

The argument that builder/runner workers are using processes rather than threads makes sense to me. Thanks for the explanation! Then I'm happy with the "core" terminology

@masahi

Copy link
Copy Markdown
MemberAuthor

Replaced num_threads with num_tuning_cores in the API.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@junrushaojunrushao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masahi
masahi merged commit 3b001ef into apache:mainDec 9, 2022
fzi-peccia pushed a commit to fzi-peccia/tvm that referenced this pull request Mar 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@masahi@tvm-bot@junrushao@spectrometerHBH@zxybazh@shingjan
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[MetaSchedule] Restore num_threads parameter in tuning API - #13561

Merged
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage
Dec 9, 2022
Merged

[MetaSchedule] Restore num_threads parameter in tuning API #13561
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage

Conversation

@masahi

@masahimasahi commented Dec 6, 2022

Copy link
Copy Markdown
Member

num_threads parameter in the Relay tuning API was (accidentally?) removed in #12895. This PR restores this parameter and also uses it consistently for XGB model training and builder / runner as well.

@zxybazh

@tvm-bot

tvm-bot commented Dec 6, 2022

Copy link
Copy Markdown
Collaborator

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@junrushao

Copy link
Copy Markdown
Member

I am not sure it should be a top-level parameter as it’s less frequently used by introductory-level users, but don’t have much strong opinions.

In the meantime, I’m not sure if using the same num_threads for xgb and for evo search is a good idea because I assume xgb prefers physical cores? I don’t have numbers handy so I’m not 100% sure.

Removed this because @spectrometerHBH@Hzfengsy@jinhongyii@MasterJH5574 suggested to, so I’d love to hear more from you guys.

@spectrometerHBH

spectrometerHBH commented Dec 6, 2022

Copy link
Copy Markdown
Contributor

I would suggest using a better name because num_threads can be confusing, especially when the TIR programs to be tuned are using parallelization.

@masahi

masahi commented Dec 6, 2022

Copy link
Copy Markdown
MemberAuthor

The main use case of this param is for tuning on a high-core system shared by many users. Currently if one user starts tuning, it occupies all CPU resources, which disrupts other users. So the goal is to limit the number of cores used by MS throughout the tuning process (evo search, post order apply, XGB training, builder / runner).

Also, tune_tir API has num_threads param as well.

num_threads: Union[Literal["physical", "logical"], int] ="physical",

I would suggest using a better name because num_threads can be confusing

Agreed, but that's what TuneContext calls... I can replace num_threads in the high-level API with max_workers or something, and initialize TuneContext by num_threads=max_workers. If people think this is better I can do that, otherwise I'd keep the existing convention.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kinda in favor of this change because it allows more customization. We may want to find a good naming and default value for num_threads though.

Comment threadpython/tvm/meta_schedule/relay_integration.py Outdated
Comment threadtests/python/contrib/test_hexagon/metaschedule_e2e/test_resnet50_int8.py Outdated
@junrushao

Copy link
Copy Markdown
Member

I don't have a clear idea (very bad at naming). Perhaps @tqchen you could suggest?

@masahi

masahi commented Dec 7, 2022

Copy link
Copy Markdown
MemberAuthor

I'm going to go with num_tuning_cores per discussion with @zxybazh, if there is no objection. For now I'll keep num_threads in TuneContext.

@junrushao

Copy link
Copy Markdown
Member

The reason that I don’t like “cores” is that they are threads, which are not necessarily related to physical cpu cores. Perhaps num_threads makes more sense at the moment.

@zxybazh

Copy link
Copy Markdown
Member

Just want to add that Runner & Builder are using new processes, that's why we may want to consider using other terminology. On the other hand, if we don't have very good idea on new naming, I don't mind just sticking to the current naming nun_threads before that.

@masahi

Copy link
Copy Markdown
MemberAuthor

I think "cores" is better because we are also using this parameter to set the number of workers used by builder / runner. Also, when a user wants to limit the amount of CPU resources used by MS, they would think in terms of the number of "cores", not "threads". So as an API, "core" sounds more intuitive to me.

@junrushao

Copy link
Copy Markdown
Member

The argument that builder/runner workers are using processes rather than threads makes sense to me. Thanks for the explanation! Then I'm happy with the "core" terminology

@masahi

Copy link
Copy Markdown
MemberAuthor

Replaced num_threads with num_tuning_cores in the API.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@junrushaojunrushao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masahi
masahi merged commit 3b001ef into apache:mainDec 9, 2022
fzi-peccia pushed a commit to fzi-peccia/tvm that referenced this pull request Mar 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@masahi@tvm-bot@junrushao@spectrometerHBH@zxybazh@shingjan
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

[MetaSchedule] Restore num_threads parameter in tuning API - #13561

Merged
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage
Dec 9, 2022
Merged

[MetaSchedule] Restore num_threads parameter in tuning API #13561
masahi merged 11 commits into
apache:mainfrom
masahi:ms-thread-usage

Conversation

@masahi

@masahimasahi commented Dec 6, 2022

Copy link
Copy Markdown
Member

num_threads parameter in the Relay tuning API was (accidentally?) removed in #12895. This PR restores this parameter and also uses it consistently for XGB model training and builder / runner as well.

@zxybazh

@tvm-bot

tvm-bot commented Dec 6, 2022

Copy link
Copy Markdown
Collaborator

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@junrushao

Copy link
Copy Markdown
Member

I am not sure it should be a top-level parameter as it’s less frequently used by introductory-level users, but don’t have much strong opinions.

In the meantime, I’m not sure if using the same num_threads for xgb and for evo search is a good idea because I assume xgb prefers physical cores? I don’t have numbers handy so I’m not 100% sure.

Removed this because @spectrometerHBH@Hzfengsy@jinhongyii@MasterJH5574 suggested to, so I’d love to hear more from you guys.

@spectrometerHBH

spectrometerHBH commented Dec 6, 2022

Copy link
Copy Markdown
Contributor

I would suggest using a better name because num_threads can be confusing, especially when the TIR programs to be tuned are using parallelization.

@masahi

masahi commented Dec 6, 2022

Copy link
Copy Markdown
MemberAuthor

The main use case of this param is for tuning on a high-core system shared by many users. Currently if one user starts tuning, it occupies all CPU resources, which disrupts other users. So the goal is to limit the number of cores used by MS throughout the tuning process (evo search, post order apply, XGB training, builder / runner).

Also, tune_tir API has num_threads param as well.

num_threads: Union[Literal["physical", "logical"], int] ="physical",

I would suggest using a better name because num_threads can be confusing

Agreed, but that's what TuneContext calls... I can replace num_threads in the high-level API with max_workers or something, and initialize TuneContext by num_threads=max_workers. If people think this is better I can do that, otherwise I'd keep the existing convention.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kinda in favor of this change because it allows more customization. We may want to find a good naming and default value for num_threads though.

Comment threadpython/tvm/meta_schedule/relay_integration.py Outdated
Comment threadtests/python/contrib/test_hexagon/metaschedule_e2e/test_resnet50_int8.py Outdated
@junrushao

Copy link
Copy Markdown
Member

I don't have a clear idea (very bad at naming). Perhaps @tqchen you could suggest?

@masahi

masahi commented Dec 7, 2022

Copy link
Copy Markdown
MemberAuthor

I'm going to go with num_tuning_cores per discussion with @zxybazh, if there is no objection. For now I'll keep num_threads in TuneContext.

@junrushao

Copy link
Copy Markdown
Member

The reason that I don’t like “cores” is that they are threads, which are not necessarily related to physical cpu cores. Perhaps num_threads makes more sense at the moment.

@zxybazh

Copy link
Copy Markdown
Member

Just want to add that Runner & Builder are using new processes, that's why we may want to consider using other terminology. On the other hand, if we don't have very good idea on new naming, I don't mind just sticking to the current naming nun_threads before that.

@masahi

Copy link
Copy Markdown
MemberAuthor

I think "cores" is better because we are also using this parameter to set the number of workers used by builder / runner. Also, when a user wants to limit the amount of CPU resources used by MS, they would think in terms of the number of "cores", not "threads". So as an API, "core" sounds more intuitive to me.

@junrushao

Copy link
Copy Markdown
Member

The argument that builder/runner workers are using processes rather than threads makes sense to me. Thanks for the explanation! Then I'm happy with the "core" terminology

@masahi

Copy link
Copy Markdown
MemberAuthor

Replaced num_threads with num_tuning_cores in the API.

@zxybazhzxybazh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@junrushaojunrushao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masahi
masahi merged commit 3b001ef into apache:mainDec 9, 2022
fzi-peccia pushed a commit to fzi-peccia/tvm that referenced this pull request Mar 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023
…13561)
* [MetaSchedule] Restore num_threads argument in tune_relay
* pass num_threads to XGBModel
* fix default
* pass num_threads as max_workers to Builder and Runner
* add test
* clean up
* fix kwarg
* num_threads -> num_tuning_cores
* typo
* num_threads -> num_tuning_cores in contrib/torch
* typo in document
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@masahi@tvm-bot@junrushao@spectrometerHBH@zxybazh@shingjan