Skip to content

Fix BF16 ONNX export for successful ONNX Runtime Verification - #290

Merged
ksivaman merged 1 commit into
NVIDIA:mainfrom
asfiyab-nvidia:dev-bf16-onnx-export-fix
Jun 21, 2023
Merged

Fix BF16 ONNX export for successful ONNX Runtime Verification#290
ksivaman merged 1 commit into
NVIDIA:mainfrom
asfiyab-nvidia:dev-bf16-onnx-export-fix

Conversation

@asfiyab-nvidia

@asfiyab-nvidiaasfiyab-nvidia commented Jun 20, 2023

Copy link
Copy Markdown
Contributor

Models with BF16 precision export to ONNX successfully but fail to run through ONNX Runtime. Issues observed:

  1. Output of GEMM is FP32 - The output of gemm is not type-compliant as an input to a subsequent node that expects a BF16 input.
  2. The torch.empty call in the forward definition of UnfusedDotProductAttention sets the type of the value attribute of the ConstantOfShape node to BF16. This is not supported by ONNX.

The changes in the MR address the issues above

@asfiyab-nvidia
Fix BF16 ONNX export for successful ONNX Runtime Verification
0b2db29
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@asfiyab-nvidia

asfiyab-nvidia commented Jun 20, 2023

Copy link
Copy Markdown
ContributorAuthor

cc @galagam@nzmora-nvidia

@asfiyab-nvidia

asfiyab-nvidia commented Jun 20, 2023

Copy link
Copy Markdown
ContributorAuthor

cc @ksivaman

@ksivaman
ksivaman self-requested a review June 20, 2023 21:57
@ksivaman

ksivaman commented Jun 20, 2023

Copy link
Copy Markdown
Member

/te-ci

@ksivaman

ksivaman commented Jun 21, 2023

Copy link
Copy Markdown
Member

Pipeline 8694390

ksivaman
ksivaman approved these changes Jun 21, 2023
View reviewed changes

@ksivamanksivaman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ksivaman
ksivaman merged commit 804f120 into NVIDIA:mainJun 21, 2023
ksivaman pushed a commit that referenced this pull request Jun 26, 2023
@asfiyab-nvidia@ksivaman
Fix BF16 ONNX export for successful ONNX Runtime Verification (#290)
ac919e4
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@hungryGeek16hungryGeek16 mentioned this pull request May 31, 2026
Open
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ksivamanksivamanksivaman approved these changes

Assignees

No one assigned

Labels

0.10.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@asfiyab-nvidia@ksivaman
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Fix BF16 ONNX export for successful ONNX Runtime Verification by asfiyab-nvidia · Pull Request #290 · NVIDIA/TransformerEngine · GitHub
Skip to content
Fix BF16 ONNX export for successful ONNX Runtime Verification - #290
Merged
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Jun 21, 2023
Merged
Fix BF16 ONNX export for successful ONNX Runtime Verification#290
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Conversation
@asfiyab-nvidia
@asfiyab-nvidiaasfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

Contributor
Models with BF16 precision export to ONNX successfully but fail to run through ONNX Runtime. Issues observed:
  1. Output of GEMM is FP32 - The output of gemm is not type-compliant as an input to a subsequent node that expects a BF16 input.
  2. The torch.empty call in the forward definition of UnfusedDotProductAttention sets the type of the value attribute of the ConstantOfShape node to BF16. This is not supported by ONNX.
The changes in the MR address the issues above
@asfiyab-nvidia
Fix BF16 ONNX export for successful ONNX Runtime Verification
0b2db29
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @galagam@nzmora-nvidia
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @ksivaman
@ksivaman
ksivaman
self-requested a review
June 20, 2023 21:57
@ksivaman
ksivaman
commented
Jun 20, 2023

Copy link


Copy Markdown

Member
/te-ci
@ksivaman
ksivaman
commented
Jun 21, 2023

Copy link


Copy Markdown

Member
Pipeline 8694390
ksivaman
ksivaman
approved these changes
Jun 21, 2023
View reviewed changes
@ksivamanksivaman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@ksivaman
ksivaman
merged commit 804f120
into
NVIDIA:mainJun 21, 2023
ksivaman pushed a commit
that referenced
this pull request
Jun 26, 2023
@asfiyab-nvidia@ksivaman
Fix BF16 ONNX export for successful ONNX Runtime Verification (#290)
ac919e4
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@hungryGeek16hungryGeek16
mentioned this pull request
May 31, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@ksivamanksivamanksivaman approved these changes

Assignees


No one assigned


Labels

0.10.0

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@asfiyab-nvidia@ksivaman

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Fix BF16 ONNX export for successful ONNX Runtime Verification by asfiyab-nvidia · Pull Request #290 · NVIDIA/TransformerEngine · GitHub
Skip to content
Fix BF16 ONNX export for successful ONNX Runtime Verification - #290
Merged
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Jun 21, 2023
Merged
Fix BF16 ONNX export for successful ONNX Runtime Verification#290
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Conversation
@asfiyab-nvidia
@asfiyab-nvidiaasfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

Contributor
Models with BF16 precision export to ONNX successfully but fail to run through ONNX Runtime. Issues observed:
  1. Output of GEMM is FP32 - The output of gemm is not type-compliant as an input to a subsequent node that expects a BF16 input.
  2. The torch.empty call in the forward definition of UnfusedDotProductAttention sets the type of the value attribute of the ConstantOfShape node to BF16. This is not supported by ONNX.
The changes in the MR address the issues above
@asfiyab-nvidia
Fix BF16 ONNX export for successful ONNX Runtime Verification
0b2db29
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @galagam@nzmora-nvidia
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @ksivaman
@ksivaman
ksivaman
self-requested a review
June 20, 2023 21:57
@ksivaman
ksivaman
commented
Jun 20, 2023

Copy link


Copy Markdown

Member
/te-ci
@ksivaman
ksivaman
commented
Jun 21, 2023

Copy link


Copy Markdown

Member
Pipeline 8694390
ksivaman
ksivaman
approved these changes
Jun 21, 2023
View reviewed changes
@ksivamanksivaman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@ksivaman
ksivaman
merged commit 804f120
into
NVIDIA:mainJun 21, 2023
ksivaman pushed a commit
that referenced
this pull request
Jun 26, 2023
@asfiyab-nvidia@ksivaman
Fix BF16 ONNX export for successful ONNX Runtime Verification (#290)
ac919e4
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@hungryGeek16hungryGeek16
mentioned this pull request
May 31, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@ksivamanksivamanksivaman approved these changes

Assignees


No one assigned


Labels

0.10.0

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@asfiyab-nvidia@ksivaman

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Fix BF16 ONNX export for successful ONNX Runtime Verification by asfiyab-nvidia · Pull Request #290 · NVIDIA/TransformerEngine · GitHub
Skip to content
Fix BF16 ONNX export for successful ONNX Runtime Verification - #290
Merged
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Jun 21, 2023
Merged
Fix BF16 ONNX export for successful ONNX Runtime Verification#290
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Conversation
@asfiyab-nvidia
@asfiyab-nvidiaasfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

Contributor
Models with BF16 precision export to ONNX successfully but fail to run through ONNX Runtime. Issues observed:
  1. Output of GEMM is FP32 - The output of gemm is not type-compliant as an input to a subsequent node that expects a BF16 input.
  2. The torch.empty call in the forward definition of UnfusedDotProductAttention sets the type of the value attribute of the ConstantOfShape node to BF16. This is not supported by ONNX.
The changes in the MR address the issues above
@asfiyab-nvidia
Fix BF16 ONNX export for successful ONNX Runtime Verification
0b2db29
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @galagam@nzmora-nvidia
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @ksivaman
@ksivaman
ksivaman
self-requested a review
June 20, 2023 21:57
@ksivaman
ksivaman
commented
Jun 20, 2023

Copy link


Copy Markdown

Member
/te-ci
@ksivaman
ksivaman
commented
Jun 21, 2023

Copy link


Copy Markdown

Member
Pipeline 8694390
ksivaman
ksivaman
approved these changes
Jun 21, 2023
View reviewed changes
@ksivamanksivaman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@ksivaman
ksivaman
merged commit 804f120
into
NVIDIA:mainJun 21, 2023
ksivaman pushed a commit
that referenced
this pull request
Jun 26, 2023
@asfiyab-nvidia@ksivaman
Fix BF16 ONNX export for successful ONNX Runtime Verification (#290)
ac919e4
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@hungryGeek16hungryGeek16
mentioned this pull request
May 31, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@ksivamanksivamanksivaman approved these changes

Assignees


No one assigned


Labels

0.10.0

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@asfiyab-nvidia@ksivaman

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Fix BF16 ONNX export for successful ONNX Runtime Verification by asfiyab-nvidia · Pull Request #290 · NVIDIA/TransformerEngine · GitHub
Skip to content
Fix BF16 ONNX export for successful ONNX Runtime Verification - #290
Merged
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Jun 21, 2023
Merged
Fix BF16 ONNX export for successful ONNX Runtime Verification#290
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Conversation
@asfiyab-nvidia
@asfiyab-nvidiaasfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

Contributor
Models with BF16 precision export to ONNX successfully but fail to run through ONNX Runtime. Issues observed:
  1. Output of GEMM is FP32 - The output of gemm is not type-compliant as an input to a subsequent node that expects a BF16 input.
  2. The torch.empty call in the forward definition of UnfusedDotProductAttention sets the type of the value attribute of the ConstantOfShape node to BF16. This is not supported by ONNX.
The changes in the MR address the issues above
@asfiyab-nvidia
Fix BF16 ONNX export for successful ONNX Runtime Verification
0b2db29
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @galagam@nzmora-nvidia
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @ksivaman
@ksivaman
ksivaman
self-requested a review
June 20, 2023 21:57
@ksivaman
ksivaman
commented
Jun 20, 2023

Copy link


Copy Markdown

Member
/te-ci
@ksivaman
ksivaman
commented
Jun 21, 2023

Copy link


Copy Markdown

Member
Pipeline 8694390
ksivaman
ksivaman
approved these changes
Jun 21, 2023
View reviewed changes
@ksivamanksivaman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@ksivaman
ksivaman
merged commit 804f120
into
NVIDIA:mainJun 21, 2023
ksivaman pushed a commit
that referenced
this pull request
Jun 26, 2023
@asfiyab-nvidia@ksivaman
Fix BF16 ONNX export for successful ONNX Runtime Verification (#290)
ac919e4
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@hungryGeek16hungryGeek16
mentioned this pull request
May 31, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@ksivamanksivamanksivaman approved these changes

Assignees


No one assigned


Labels

0.10.0

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@asfiyab-nvidia@ksivaman

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Fix BF16 ONNX export for successful ONNX Runtime Verification by asfiyab-nvidia · Pull Request #290 · NVIDIA/TransformerEngine · GitHub
Skip to content
Fix BF16 ONNX export for successful ONNX Runtime Verification - #290
Merged
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Jun 21, 2023
Merged
Fix BF16 ONNX export for successful ONNX Runtime Verification#290
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Conversation
@asfiyab-nvidia
@asfiyab-nvidiaasfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

Contributor
Models with BF16 precision export to ONNX successfully but fail to run through ONNX Runtime. Issues observed:
  1. Output of GEMM is FP32 - The output of gemm is not type-compliant as an input to a subsequent node that expects a BF16 input.
  2. The torch.empty call in the forward definition of UnfusedDotProductAttention sets the type of the value attribute of the ConstantOfShape node to BF16. This is not supported by ONNX.
The changes in the MR address the issues above
@asfiyab-nvidia
Fix BF16 ONNX export for successful ONNX Runtime Verification
0b2db29
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @galagam@nzmora-nvidia
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @ksivaman
@ksivaman
ksivaman
self-requested a review
June 20, 2023 21:57
@ksivaman
ksivaman
commented
Jun 20, 2023

Copy link


Copy Markdown

Member
/te-ci
@ksivaman
ksivaman
commented
Jun 21, 2023

Copy link


Copy Markdown

Member
Pipeline 8694390
ksivaman
ksivaman
approved these changes
Jun 21, 2023
View reviewed changes
@ksivamanksivaman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@ksivaman
ksivaman
merged commit 804f120
into
NVIDIA:mainJun 21, 2023
ksivaman pushed a commit
that referenced
this pull request
Jun 26, 2023
@asfiyab-nvidia@ksivaman
Fix BF16 ONNX export for successful ONNX Runtime Verification (#290)
ac919e4
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@hungryGeek16hungryGeek16
mentioned this pull request
May 31, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@ksivamanksivamanksivaman approved these changes

Assignees


No one assigned


Labels

0.10.0

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@asfiyab-nvidia@ksivaman

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
})();
Fix BF16 ONNX export for successful ONNX Runtime Verification by asfiyab-nvidia · Pull Request #290 · NVIDIA/TransformerEngine · GitHub
Skip to content
Fix BF16 ONNX export for successful ONNX Runtime Verification - #290
Merged
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Jun 21, 2023
Merged
Fix BF16 ONNX export for successful ONNX Runtime Verification#290
ksivaman merged 1 commit into
NVIDIA:mainfrom 
asfiyab-nvidia:dev-bf16-onnx-export-fix
Conversation
@asfiyab-nvidia
@asfiyab-nvidiaasfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

Contributor
Models with BF16 precision export to ONNX successfully but fail to run through ONNX Runtime. Issues observed:
  1. Output of GEMM is FP32 - The output of gemm is not type-compliant as an input to a subsequent node that expects a BF16 input.
  2. The torch.empty call in the forward definition of UnfusedDotProductAttention sets the type of the value attribute of the ConstantOfShape node to BF16. This is not supported by ONNX.
The changes in the MR address the issues above
@asfiyab-nvidia
Fix BF16 ONNX export for successful ONNX Runtime Verification
0b2db29
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @galagam@nzmora-nvidia
@asfiyab-nvidia
asfiyab-nvidia
commented
Jun 20, 2023

Copy link


Copy Markdown

ContributorAuthor
cc @ksivaman
@ksivaman
ksivaman
self-requested a review
June 20, 2023 21:57
@ksivaman
ksivaman
commented
Jun 20, 2023

Copy link


Copy Markdown

Member
/te-ci
@ksivaman
ksivaman
commented
Jun 21, 2023

Copy link


Copy Markdown

Member
Pipeline 8694390
ksivaman
ksivaman
approved these changes
Jun 21, 2023
View reviewed changes
@ksivamanksivaman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@ksivaman
ksivaman
merged commit 804f120
into
NVIDIA:mainJun 21, 2023
ksivaman pushed a commit
that referenced
this pull request
Jun 26, 2023
@asfiyab-nvidia@ksivaman
Fix BF16 ONNX export for successful ONNX Runtime Verification (#290)
ac919e4
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@hungryGeek16hungryGeek16
mentioned this pull request
May 31, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@ksivamanksivamanksivaman approved these changes

Assignees


No one assigned


Labels

0.10.0

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@asfiyab-nvidia@ksivaman