[Adreno] Add markup pass of relay tensors for static texture planning#11878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
csullivan merged 15 commits into from
Aug 2, 2022
Merged

[Adreno] Add markup pass of relay tensors for static texture planning #11878

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
a3cf22d
[Adreno] Add static texture markup relay pass
elvin-n Jun 24, 2022
c58549d
lint check
elvin-n Jun 24, 2022
eb41312
Remove hardcoded texture limit, check through target options
elvin-n Jun 24, 2022
6b28ec6
fix cpplint
elvin-n Jul 14, 2022
7b07585
Add winograd into annotation pass
elvin-n Jul 14, 2022
94ad477
fix clang
elvin-n Jul 18, 2022
43ef0db
Remove extra call of PlanDevice in OptimizeImpl
elvin-n Jul 19, 2022
d51c1e4
Remove one more extra call of PlanDevice in OptimizeImpl
elvin-n Jul 19, 2022
dcc39f6
Fix/add scopes for static texture planning tests
elvin-n Jul 22, 2022
183a270
Remove test_2conv2d as duplication of test_plan_device_issue
elvin-n Jul 26, 2022
f105c65
remove comments in test_residual_block
elvin-n Jul 26, 2022
d4cc2b4
address review comments
elvin-n Jul 26, 2022
9d37fee
fix black hits
elvin-n Jul 26, 2022
8a0fce2
Add textures test descriptions
elvin-n Jul 28, 2022
1c8abb5
Address PR comments
elvin-n Jul 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions include/tvm/relay/transform.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -580,6 +580,11 @@ TVM_DLL Pass AnnotateUsedMemory();
*/
TVM_DLL Pass CapturePostDfsIndexInSpans();

/*!
* \brief Calls device dependent memory scope analysis pass, collects mapping of desirable
* expr->memory_scope and annotates expressions by VirtualDevice with required memory_scope
*/
TVM_DLL Pass AnnotateMemoryScope(CompilationConfig config);
} // namespace transform

/*!
Expand Down
19 changes: 14 additions & 5 deletions python/tvm/topi/adreno/conv2d_nchw.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -29,6 +29,7 @@
add_pad,
bind_data_copy,
get_default_conv2d_config,
get_texture_storage,
)


Expand DownExpand Up@@ -214,8 +215,11 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
5d tensors
4. pad should be scheduled separately to create independent opencl kernel. If pad is
inlined into convolution, this gives 1.5x performance drop
5. We are using cache_read to produce texture and guarantee the best performance
on the next stage.
5. We are using cache_read for intermediate tensors to produce texture and guarantee
the best performance on the next stage.
The weights are managed through static texture planning mechanism and guarantied come
in texture memory scope.
Thus way we are calling cache_read only for data tensor
6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
for textures
For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -288,10 +292,15 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
s[output].compute_inline()

# create cache stage
AT = s.cache_read(pad_data, "global.texture", [conv])
AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
bind_data_copy(s[AT])
WT = s.cache_read(kernel, "global.texture-weight", [conv])
bind_data_copy(s[WT])
if (
autotvm.GLOBAL_SCOPE.in_tuning
or isinstance(kernel.op, tvm.te.ComputeOp)
and "filter_pack" in kernel.op.tag
):
WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
bind_data_copy(s[WT])

# tile and bind spatial axes
n, fc, y, x, fb = s[latest_blocked].op.axis
Expand Down
16 changes: 12 additions & 4 deletions python/tvm/topi/adreno/conv2d_nhwc.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -210,8 +210,11 @@ def schedule_conv2d_NHWC(cfg, s, output):
5d tensors
4. pad should be scheduled separately to create independent opencl kernel. If pad is
inlined into convolution, this gives 1.5x performance drop
5. We are using cache_read to produce texture and guarantee the best performance
on the next stage.
5. We are using cache_read for intermediate tensors to produce texture and guarantee
the best performance on the next stage.
The weights are managed through static texture planning mechanism and guarantied come
in texture memory scope.
Thus way we are calling cache_read only for data tensor
6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
for textures
For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -287,8 +290,13 @@ def schedule_conv2d_NHWC(cfg, s, output):
# create cache stage
AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
bind_data_copy(s[AT])
WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
bind_data_copy(s[WT])
if (
autotvm.GLOBAL_SCOPE.in_tuning
or isinstance(kernel.op, tvm.te.ComputeOp)
and "filter_pack" in kernel.op.tag
):
WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
bind_data_copy(s[WT])

# tile and bind spatial axes
n, y, x, fc, fb = s[latest_blocked].op.axis
Expand Down
1 change: 1 addition & 0 deletions src/relay/backend/build_module.cc
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -396,6 +396,7 @@ class RelayBuildModule : public runtime::ModuleNode {
relay_module = transform::Inline()(relay_module);
relay_module = transform::InferType()(relay_module);
relay_module = transform::LabelOps()(relay_module);
relay_module = transform::AnnotateMemoryScope(config_)(relay_module);

ICHECK(relay_module.defined());

Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
[Adreno] Add markup pass of relay tensors for static texture planning#11878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

csullivan
merged 15 commits into

from

Aug 2, 2022
 Merged


[Adreno] Add markup pass of relay tensors for static texture planning
#11878
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


15 commits


Select commit
Hold shift + click to select a range
a3cf22d

[Adreno] Add static texture markup relay pass


elvin-n Jun 24, 2022
c58549d

lint check


elvin-n Jun 24, 2022
eb41312

Remove hardcoded texture limit, check through target options


elvin-n Jun 24, 2022
6b28ec6

fix cpplint


elvin-n Jul 14, 2022
7b07585

Add winograd into annotation pass


elvin-n Jul 14, 2022
94ad477

fix clang


elvin-n Jul 18, 2022
43ef0db

Remove extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
d51c1e4

Remove one more extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
dcc39f6

Fix/add scopes for static texture planning tests


elvin-n Jul 22, 2022
183a270

Remove test_2conv2d as duplication of test_plan_device_issue


elvin-n Jul 26, 2022
f105c65

remove comments in test_residual_block


elvin-n Jul 26, 2022
d4cc2b4

address review comments


elvin-n Jul 26, 2022
9d37fee

fix black hits


elvin-n Jul 26, 2022
8a0fce2

Add textures test descriptions


elvin-n Jul 28, 2022
1c8abb5

Address PR comments


elvin-n Jul 28, 2022

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


5 changes: 5 additions & 0 deletions
include/tvm/relay/transform.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -580,6 +580,11 @@ TVM_DLL Pass AnnotateUsedMemory();
 */
TVM_DLL Pass CapturePostDfsIndexInSpans();

/*!
 * \brief Calls device dependent memory scope analysis pass, collects mapping of desirable
 * expr->memory_scope and annotates expressions by VirtualDevice with required memory_scope
 */
TVM_DLL Pass AnnotateMemoryScope(CompilationConfig config);
} // namespace transform

/*!
Expand Down

19 changes: 14 additions & 5 deletions
python/tvm/topi/adreno/conv2d_nchw.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -29,6 +29,7 @@
 add_pad,
 bind_data_copy,
 get_default_conv2d_config,
 get_texture_storage,
)


Expand DownExpand Up@@ -214,8 +215,11 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -288,10 +292,15 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 s[output].compute_inline()

 # create cache stage
 AT = s.cache_read(pad_data, "global.texture", [conv])
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, "global.texture-weight", [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, fc, y, x, fb = s[latest_blocked].op.axis
Expand Down

16 changes: 12 additions & 4 deletions
python/tvm/topi/adreno/conv2d_nhwc.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -210,8 +210,11 @@ def schedule_conv2d_NHWC(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -287,8 +290,13 @@ def schedule_conv2d_NHWC(cfg, s, output):
 # create cache stage
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, y, x, fc, fb = s[latest_blocked].op.axis
Expand Down

1 change: 1 addition & 0 deletions
src/relay/backend/build_module.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -396,6 +396,7 @@ class RelayBuildModule : public runtime::ModuleNode {
 relay_module = transform::Inline()(relay_module);
 relay_module = transform::InferType()(relay_module);
 relay_module = transform::LabelOps()(relay_module);
 relay_module = transform::AnnotateMemoryScope(config_)(relay_module);

 ICHECK(relay_module.defined());

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[Adreno] Add markup pass of relay tensors for static texture planning#11878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

csullivan
merged 15 commits into

from

Aug 2, 2022
 Merged


[Adreno] Add markup pass of relay tensors for static texture planning
#11878
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


15 commits


Select commit
Hold shift + click to select a range
a3cf22d

[Adreno] Add static texture markup relay pass


elvin-n Jun 24, 2022
c58549d

lint check


elvin-n Jun 24, 2022
eb41312

Remove hardcoded texture limit, check through target options


elvin-n Jun 24, 2022
6b28ec6

fix cpplint


elvin-n Jul 14, 2022
7b07585

Add winograd into annotation pass


elvin-n Jul 14, 2022
94ad477

fix clang


elvin-n Jul 18, 2022
43ef0db

Remove extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
d51c1e4

Remove one more extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
dcc39f6

Fix/add scopes for static texture planning tests


elvin-n Jul 22, 2022
183a270

Remove test_2conv2d as duplication of test_plan_device_issue


elvin-n Jul 26, 2022
f105c65

remove comments in test_residual_block


elvin-n Jul 26, 2022
d4cc2b4

address review comments


elvin-n Jul 26, 2022
9d37fee

fix black hits


elvin-n Jul 26, 2022
8a0fce2

Add textures test descriptions


elvin-n Jul 28, 2022
1c8abb5

Address PR comments


elvin-n Jul 28, 2022

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


5 changes: 5 additions & 0 deletions
include/tvm/relay/transform.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -580,6 +580,11 @@ TVM_DLL Pass AnnotateUsedMemory();
 */
TVM_DLL Pass CapturePostDfsIndexInSpans();

/*!
 * \brief Calls device dependent memory scope analysis pass, collects mapping of desirable
 * expr->memory_scope and annotates expressions by VirtualDevice with required memory_scope
 */
TVM_DLL Pass AnnotateMemoryScope(CompilationConfig config);
} // namespace transform

/*!
Expand Down

19 changes: 14 additions & 5 deletions
python/tvm/topi/adreno/conv2d_nchw.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -29,6 +29,7 @@
 add_pad,
 bind_data_copy,
 get_default_conv2d_config,
 get_texture_storage,
)


Expand DownExpand Up@@ -214,8 +215,11 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -288,10 +292,15 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 s[output].compute_inline()

 # create cache stage
 AT = s.cache_read(pad_data, "global.texture", [conv])
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, "global.texture-weight", [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, fc, y, x, fb = s[latest_blocked].op.axis
Expand Down

16 changes: 12 additions & 4 deletions
python/tvm/topi/adreno/conv2d_nhwc.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -210,8 +210,11 @@ def schedule_conv2d_NHWC(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -287,8 +290,13 @@ def schedule_conv2d_NHWC(cfg, s, output):
 # create cache stage
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, y, x, fc, fb = s[latest_blocked].op.axis
Expand Down

1 change: 1 addition & 0 deletions
src/relay/backend/build_module.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -396,6 +396,7 @@ class RelayBuildModule : public runtime::ModuleNode {
 relay_module = transform::Inline()(relay_module);
 relay_module = transform::InferType()(relay_module);
 relay_module = transform::LabelOps()(relay_module);
 relay_module = transform::AnnotateMemoryScope(config_)(relay_module);

 ICHECK(relay_module.defined());

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[Adreno] Add markup pass of relay tensors for static texture planning#11878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

csullivan
merged 15 commits into

from

Aug 2, 2022
 Merged


[Adreno] Add markup pass of relay tensors for static texture planning
#11878
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


15 commits


Select commit
Hold shift + click to select a range
a3cf22d

[Adreno] Add static texture markup relay pass


elvin-n Jun 24, 2022
c58549d

lint check


elvin-n Jun 24, 2022
eb41312

Remove hardcoded texture limit, check through target options


elvin-n Jun 24, 2022
6b28ec6

fix cpplint


elvin-n Jul 14, 2022
7b07585

Add winograd into annotation pass


elvin-n Jul 14, 2022
94ad477

fix clang


elvin-n Jul 18, 2022
43ef0db

Remove extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
d51c1e4

Remove one more extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
dcc39f6

Fix/add scopes for static texture planning tests


elvin-n Jul 22, 2022
183a270

Remove test_2conv2d as duplication of test_plan_device_issue


elvin-n Jul 26, 2022
f105c65

remove comments in test_residual_block


elvin-n Jul 26, 2022
d4cc2b4

address review comments


elvin-n Jul 26, 2022
9d37fee

fix black hits


elvin-n Jul 26, 2022
8a0fce2

Add textures test descriptions


elvin-n Jul 28, 2022
1c8abb5

Address PR comments


elvin-n Jul 28, 2022

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


5 changes: 5 additions & 0 deletions
include/tvm/relay/transform.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -580,6 +580,11 @@ TVM_DLL Pass AnnotateUsedMemory();
 */
TVM_DLL Pass CapturePostDfsIndexInSpans();

/*!
 * \brief Calls device dependent memory scope analysis pass, collects mapping of desirable
 * expr->memory_scope and annotates expressions by VirtualDevice with required memory_scope
 */
TVM_DLL Pass AnnotateMemoryScope(CompilationConfig config);
} // namespace transform

/*!
Expand Down

19 changes: 14 additions & 5 deletions
python/tvm/topi/adreno/conv2d_nchw.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -29,6 +29,7 @@
 add_pad,
 bind_data_copy,
 get_default_conv2d_config,
 get_texture_storage,
)


Expand DownExpand Up@@ -214,8 +215,11 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -288,10 +292,15 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 s[output].compute_inline()

 # create cache stage
 AT = s.cache_read(pad_data, "global.texture", [conv])
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, "global.texture-weight", [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, fc, y, x, fb = s[latest_blocked].op.axis
Expand Down

16 changes: 12 additions & 4 deletions
python/tvm/topi/adreno/conv2d_nhwc.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -210,8 +210,11 @@ def schedule_conv2d_NHWC(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -287,8 +290,13 @@ def schedule_conv2d_NHWC(cfg, s, output):
 # create cache stage
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, y, x, fc, fb = s[latest_blocked].op.axis
Expand Down

1 change: 1 addition & 0 deletions
src/relay/backend/build_module.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -396,6 +396,7 @@ class RelayBuildModule : public runtime::ModuleNode {
 relay_module = transform::Inline()(relay_module);
 relay_module = transform::InferType()(relay_module);
 relay_module = transform::LabelOps()(relay_module);
 relay_module = transform::AnnotateMemoryScope(config_)(relay_module);

 ICHECK(relay_module.defined());

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
[Adreno] Add markup pass of relay tensors for static texture planning#11878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

csullivan
merged 15 commits into

from

Aug 2, 2022
 Merged


[Adreno] Add markup pass of relay tensors for static texture planning
#11878
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


15 commits


Select commit
Hold shift + click to select a range
a3cf22d

[Adreno] Add static texture markup relay pass


elvin-n Jun 24, 2022
c58549d

lint check


elvin-n Jun 24, 2022
eb41312

Remove hardcoded texture limit, check through target options


elvin-n Jun 24, 2022
6b28ec6

fix cpplint


elvin-n Jul 14, 2022
7b07585

Add winograd into annotation pass


elvin-n Jul 14, 2022
94ad477

fix clang


elvin-n Jul 18, 2022
43ef0db

Remove extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
d51c1e4

Remove one more extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
dcc39f6

Fix/add scopes for static texture planning tests


elvin-n Jul 22, 2022
183a270

Remove test_2conv2d as duplication of test_plan_device_issue


elvin-n Jul 26, 2022
f105c65

remove comments in test_residual_block


elvin-n Jul 26, 2022
d4cc2b4

address review comments


elvin-n Jul 26, 2022
9d37fee

fix black hits


elvin-n Jul 26, 2022
8a0fce2

Add textures test descriptions


elvin-n Jul 28, 2022
1c8abb5

Address PR comments


elvin-n Jul 28, 2022

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


5 changes: 5 additions & 0 deletions
include/tvm/relay/transform.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -580,6 +580,11 @@ TVM_DLL Pass AnnotateUsedMemory();
 */
TVM_DLL Pass CapturePostDfsIndexInSpans();

/*!
 * \brief Calls device dependent memory scope analysis pass, collects mapping of desirable
 * expr->memory_scope and annotates expressions by VirtualDevice with required memory_scope
 */
TVM_DLL Pass AnnotateMemoryScope(CompilationConfig config);
} // namespace transform

/*!
Expand Down

19 changes: 14 additions & 5 deletions
python/tvm/topi/adreno/conv2d_nchw.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -29,6 +29,7 @@
 add_pad,
 bind_data_copy,
 get_default_conv2d_config,
 get_texture_storage,
)


Expand DownExpand Up@@ -214,8 +215,11 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -288,10 +292,15 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 s[output].compute_inline()

 # create cache stage
 AT = s.cache_read(pad_data, "global.texture", [conv])
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, "global.texture-weight", [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, fc, y, x, fb = s[latest_blocked].op.axis
Expand Down

16 changes: 12 additions & 4 deletions
python/tvm/topi/adreno/conv2d_nhwc.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -210,8 +210,11 @@ def schedule_conv2d_NHWC(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -287,8 +290,13 @@ def schedule_conv2d_NHWC(cfg, s, output):
 # create cache stage
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, y, x, fc, fb = s[latest_blocked].op.axis
Expand Down

1 change: 1 addition & 0 deletions
src/relay/backend/build_module.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -396,6 +396,7 @@ class RelayBuildModule : public runtime::ModuleNode {
 relay_module = transform::Inline()(relay_module);
 relay_module = transform::InferType()(relay_module);
 relay_module = transform::LabelOps()(relay_module);
 relay_module = transform::AnnotateMemoryScope(config_)(relay_module);

 ICHECK(relay_module.defined());

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[Adreno] Add markup pass of relay tensors for static texture planning#11878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

csullivan
merged 15 commits into

from

Aug 2, 2022
 Merged


[Adreno] Add markup pass of relay tensors for static texture planning
#11878
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


15 commits


Select commit
Hold shift + click to select a range
a3cf22d

[Adreno] Add static texture markup relay pass


elvin-n Jun 24, 2022
c58549d

lint check


elvin-n Jun 24, 2022
eb41312

Remove hardcoded texture limit, check through target options


elvin-n Jun 24, 2022
6b28ec6

fix cpplint


elvin-n Jul 14, 2022
7b07585

Add winograd into annotation pass


elvin-n Jul 14, 2022
94ad477

fix clang


elvin-n Jul 18, 2022
43ef0db

Remove extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
d51c1e4

Remove one more extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
dcc39f6

Fix/add scopes for static texture planning tests


elvin-n Jul 22, 2022
183a270

Remove test_2conv2d as duplication of test_plan_device_issue


elvin-n Jul 26, 2022
f105c65

remove comments in test_residual_block


elvin-n Jul 26, 2022
d4cc2b4

address review comments


elvin-n Jul 26, 2022
9d37fee

fix black hits


elvin-n Jul 26, 2022
8a0fce2

Add textures test descriptions


elvin-n Jul 28, 2022
1c8abb5

Address PR comments


elvin-n Jul 28, 2022

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


5 changes: 5 additions & 0 deletions
include/tvm/relay/transform.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -580,6 +580,11 @@ TVM_DLL Pass AnnotateUsedMemory();
 */
TVM_DLL Pass CapturePostDfsIndexInSpans();

/*!
 * \brief Calls device dependent memory scope analysis pass, collects mapping of desirable
 * expr->memory_scope and annotates expressions by VirtualDevice with required memory_scope
 */
TVM_DLL Pass AnnotateMemoryScope(CompilationConfig config);
} // namespace transform

/*!
Expand Down

19 changes: 14 additions & 5 deletions
python/tvm/topi/adreno/conv2d_nchw.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -29,6 +29,7 @@
 add_pad,
 bind_data_copy,
 get_default_conv2d_config,
 get_texture_storage,
)


Expand DownExpand Up@@ -214,8 +215,11 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -288,10 +292,15 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 s[output].compute_inline()

 # create cache stage
 AT = s.cache_read(pad_data, "global.texture", [conv])
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, "global.texture-weight", [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, fc, y, x, fb = s[latest_blocked].op.axis
Expand Down

16 changes: 12 additions & 4 deletions
python/tvm/topi/adreno/conv2d_nhwc.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -210,8 +210,11 @@ def schedule_conv2d_NHWC(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -287,8 +290,13 @@ def schedule_conv2d_NHWC(cfg, s, output):
 # create cache stage
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, y, x, fc, fb = s[latest_blocked].op.axis
Expand Down

1 change: 1 addition & 0 deletions
src/relay/backend/build_module.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -396,6 +396,7 @@ class RelayBuildModule : public runtime::ModuleNode {
 relay_module = transform::Inline()(relay_module);
 relay_module = transform::InferType()(relay_module);
 relay_module = transform::LabelOps()(relay_module);
 relay_module = transform::AnnotateMemoryScope(config_)(relay_module);

 ICHECK(relay_module.defined());

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[Adreno] Add markup pass of relay tensors for static texture planning#11878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

csullivan
merged 15 commits into

from

Aug 2, 2022
 Merged


[Adreno] Add markup pass of relay tensors for static texture planning
#11878
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


15 commits


Select commit
Hold shift + click to select a range
a3cf22d

[Adreno] Add static texture markup relay pass


elvin-n Jun 24, 2022
c58549d

lint check


elvin-n Jun 24, 2022
eb41312

Remove hardcoded texture limit, check through target options


elvin-n Jun 24, 2022
6b28ec6

fix cpplint


elvin-n Jul 14, 2022
7b07585

Add winograd into annotation pass


elvin-n Jul 14, 2022
94ad477

fix clang


elvin-n Jul 18, 2022
43ef0db

Remove extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
d51c1e4

Remove one more extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
dcc39f6

Fix/add scopes for static texture planning tests


elvin-n Jul 22, 2022
183a270

Remove test_2conv2d as duplication of test_plan_device_issue


elvin-n Jul 26, 2022
f105c65

remove comments in test_residual_block


elvin-n Jul 26, 2022
d4cc2b4

address review comments


elvin-n Jul 26, 2022
9d37fee

fix black hits


elvin-n Jul 26, 2022
8a0fce2

Add textures test descriptions


elvin-n Jul 28, 2022
1c8abb5

Address PR comments


elvin-n Jul 28, 2022

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


5 changes: 5 additions & 0 deletions
include/tvm/relay/transform.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -580,6 +580,11 @@ TVM_DLL Pass AnnotateUsedMemory();
 */
TVM_DLL Pass CapturePostDfsIndexInSpans();

/*!
 * \brief Calls device dependent memory scope analysis pass, collects mapping of desirable
 * expr->memory_scope and annotates expressions by VirtualDevice with required memory_scope
 */
TVM_DLL Pass AnnotateMemoryScope(CompilationConfig config);
} // namespace transform

/*!
Expand Down

19 changes: 14 additions & 5 deletions
python/tvm/topi/adreno/conv2d_nchw.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -29,6 +29,7 @@
 add_pad,
 bind_data_copy,
 get_default_conv2d_config,
 get_texture_storage,
)


Expand DownExpand Up@@ -214,8 +215,11 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -288,10 +292,15 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 s[output].compute_inline()

 # create cache stage
 AT = s.cache_read(pad_data, "global.texture", [conv])
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, "global.texture-weight", [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, fc, y, x, fb = s[latest_blocked].op.axis
Expand Down

16 changes: 12 additions & 4 deletions
python/tvm/topi/adreno/conv2d_nhwc.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -210,8 +210,11 @@ def schedule_conv2d_NHWC(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -287,8 +290,13 @@ def schedule_conv2d_NHWC(cfg, s, output):
 # create cache stage
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, y, x, fc, fb = s[latest_blocked].op.axis
Expand Down

1 change: 1 addition & 0 deletions
src/relay/backend/build_module.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -396,6 +396,7 @@ class RelayBuildModule : public runtime::ModuleNode {
 relay_module = transform::Inline()(relay_module);
 relay_module = transform::InferType()(relay_module);
 relay_module = transform::LabelOps()(relay_module);
 relay_module = transform::AnnotateMemoryScope(config_)(relay_module);

 ICHECK(relay_module.defined());

Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
[Adreno] Add markup pass of relay tensors for static texture planning#11878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

csullivan
merged 15 commits into

from

Aug 2, 2022
 Merged


[Adreno] Add markup pass of relay tensors for static texture planning
#11878
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


15 commits


Select commit
Hold shift + click to select a range
a3cf22d

[Adreno] Add static texture markup relay pass


elvin-n Jun 24, 2022
c58549d

lint check


elvin-n Jun 24, 2022
eb41312

Remove hardcoded texture limit, check through target options


elvin-n Jun 24, 2022
6b28ec6

fix cpplint


elvin-n Jul 14, 2022
7b07585

Add winograd into annotation pass


elvin-n Jul 14, 2022
94ad477

fix clang


elvin-n Jul 18, 2022
43ef0db

Remove extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
d51c1e4

Remove one more extra call of PlanDevice in OptimizeImpl


elvin-n Jul 19, 2022
dcc39f6

Fix/add scopes for static texture planning tests


elvin-n Jul 22, 2022
183a270

Remove test_2conv2d as duplication of test_plan_device_issue


elvin-n Jul 26, 2022
f105c65

remove comments in test_residual_block


elvin-n Jul 26, 2022
d4cc2b4

address review comments


elvin-n Jul 26, 2022
9d37fee

fix black hits


elvin-n Jul 26, 2022
8a0fce2

Add textures test descriptions


elvin-n Jul 28, 2022
1c8abb5

Address PR comments


elvin-n Jul 28, 2022

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


5 changes: 5 additions & 0 deletions
include/tvm/relay/transform.h

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -580,6 +580,11 @@ TVM_DLL Pass AnnotateUsedMemory();
 */
TVM_DLL Pass CapturePostDfsIndexInSpans();

/*!
 * \brief Calls device dependent memory scope analysis pass, collects mapping of desirable
 * expr->memory_scope and annotates expressions by VirtualDevice with required memory_scope
 */
TVM_DLL Pass AnnotateMemoryScope(CompilationConfig config);
} // namespace transform

/*!
Expand Down

19 changes: 14 additions & 5 deletions
python/tvm/topi/adreno/conv2d_nchw.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -29,6 +29,7 @@
 add_pad,
 bind_data_copy,
 get_default_conv2d_config,
 get_texture_storage,
)


Expand DownExpand Up@@ -214,8 +215,11 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -288,10 +292,15 @@ def schedule_conv2d_NCHWc_KCRSk(cfg, s, output):
 s[output].compute_inline()

 # create cache stage
 AT = s.cache_read(pad_data, "global.texture", [conv])
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, "global.texture-weight", [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, fc, y, x, fb = s[latest_blocked].op.axis
Expand Down

16 changes: 12 additions & 4 deletions
python/tvm/topi/adreno/conv2d_nhwc.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -210,8 +210,11 @@ def schedule_conv2d_NHWC(cfg, s, output):
 5d tensors
 4. pad should be scheduled separately to create independent opencl kernel. If pad is
 inlined into convolution, this gives 1.5x performance drop
 5. We are using cache_read to produce texture and guarantee the best performance
 on the next stage.
 5. We are using cache_read for intermediate tensors to produce texture and guarantee
 the best performance on the next stage.
 The weights are managed through static texture planning mechanism and guarantied come
 in texture memory scope.
 Thus way we are calling cache_read only for data tensor
 6. For 5d convolution we schedule the latest op with binding 5d axis and vectorize
 for textures
 For 4d tensor we are doing the same for the latest blocked stage, i.e. conversion
Expand DownExpand Up@@ -287,8 +290,13 @@ def schedule_conv2d_NHWC(cfg, s, output):
 # create cache stage
 AT = s.cache_read(pad_data, get_texture_storage(pad_data.shape), [conv])
 bind_data_copy(s[AT])
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])
 if (
 autotvm.GLOBAL_SCOPE.in_tuning
 or isinstance(kernel.op, tvm.te.ComputeOp)
 and "filter_pack" in kernel.op.tag
 ):
 WT = s.cache_read(kernel, get_texture_storage(kernel.shape), [conv])
 bind_data_copy(s[WT])

 # tile and bind spatial axes
 n, y, x, fc, fb = s[latest_blocked].op.axis
Expand Down

1 change: 1 addition & 0 deletions
src/relay/backend/build_module.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -396,6 +396,7 @@ class RelayBuildModule : public runtime::ModuleNode {
 relay_module = transform::Inline()(relay_module);
 relay_module = transform::InferType()(relay_module);
 relay_module = transform::LabelOps()(relay_module);
 relay_module = transform::AnnotateMemoryScope(config_)(relay_module);

 ICHECK(relay_module.defined());

Expand Down
Loading