Skip to content

Port stackoverflow fix from Roslyn to SourceGenerator PolyFill - #76955

Merged
joperezr merged 1 commit into
dotnet:mainfrom
CyrusNajmabadi:generatorStackOverflowMain
Oct 20, 2022
Merged

Port stackoverflow fix from Roslyn to SourceGenerator PolyFill#76955
joperezr merged 1 commit into
dotnet:mainfrom
CyrusNajmabadi:generatorStackOverflowMain

Conversation

@CyrusNajmabadi

@CyrusNajmabadiCyrusNajmabadi commented Oct 12, 2022•
edited
Loading

Copy link
Copy Markdown
Contributor

Fixes#76953 for 8.0 and onwards
Ports dotnet/roslyn#64322 back to the runtime polyfill.

Corresponding 6.0 servicing fix is here: #76946
Corresponding 7.0 servicing fix is here: #76954

Customer Impact

Customers using 7.0.0 sdk can run into stack-overflows in the Source-Generator helper code code, in both VS and compiler, when compiling code with certain problematic code-constructs. This happens for tehse customers even if they are not using Source-Generators themselves as the overflow occurs in the code that is trying to determine if the generator should run.

This commonly happens when working with, or trying to compile, generated code produced by tools like T4 or Antlr (which commonly generate deeply recursive trees).

Examples of code that causes this are:

stringdata="..."+"..."+"..."+/* thousands more concatenations */+"..."+"..."+"...";

In a case like this, the concatenation code is not a balanced tree but instead, effectively, a linear tree like so:

image

The existing code works by recursing the user's parse tree, which ends up blowing the stack in cases like these.

As this is a stack overflow, it is fairly catastrophic for the user. Absent them changing this code outside of VS, which may not be possible for tool-generated code, the only workaround is to disable these generators (like System.Text.Json). Disabling the generator may be suitable for customers that are not using that generator, but will completely break users who are using it and who do have these code constructs.

Testing

Risk

Low. This is a simple port of a change made in roslyn to switch from implicit recursion, to using an explicit stack. This is a refactoring that roslyn is familiar with as stack-overflows are not uncommon for us as users do have these sorts of files, and recursive solutions have often had to be refactored in a similar fashion.

@ghost

ghost commented Oct 12, 2022

Copy link
Copy Markdown

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

@build-analysisbuild-analysisBot mentioned this pull request Oct 13, 2022
Closed
2 tasks
@CyrusNajmabadiCyrusNajmabadi mentioned this pull request Oct 13, 2022
Closed
@CyrusNajmabadi

CyrusNajmabadi commented Oct 14, 2022

Copy link
Copy Markdown
ContributorAuthor

@joperezr Can we take this? Or will the 7.0 fix naturally flow into main?

@joperezr

joperezr commented Oct 14, 2022

Copy link
Copy Markdown
Member

Yes, we can take it. 7.0 doesn't flow to main in dotnet/runtime, so we actually have to take it, I just missed the email for this PR. Will manually validate in a bit and merge once it is ready

@CyrusNajmabadi

CyrusNajmabadi commented Oct 14, 2022

Copy link
Copy Markdown
ContributorAuthor

Great, thanks!

@CyrusNajmabadi

CyrusNajmabadi commented Oct 20, 2022

Copy link
Copy Markdown
ContributorAuthor

@joperezr Just pinging to make sure this is not lost :)

@joperezr

joperezr commented Oct 20, 2022

Copy link
Copy Markdown
Member

Failures in OSX are unrelated, so I'll go ahead and merge. Thanks @CyrusNajmabadi!

@joperezr
joperezr merged commit 6e6032c into dotnet:mainOct 20, 2022
@CyrusNajmabadi

CyrusNajmabadi commented Oct 20, 2022

Copy link
Copy Markdown
ContributorAuthor

Thanks!

@CyrusNajmabadi
CyrusNajmabadi deleted the generatorStackOverflowMain branch October 20, 2022 19:06
@ghostghost locked as resolved and limited conversation to collaborators Nov 20, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@joperezrjoperezrjoperezr approved these changes

Assignees

@CyrusNajmabadiCyrusNajmabadi

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

StackOverflowException on web project code analysis

2 participants

@CyrusNajmabadi@joperezr
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill by CyrusNajmabadi · Pull Request #76955 · dotnet/runtime · GitHub
Skip to content
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill - #76955
Merged
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Oct 20, 2022
Merged
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill#76955
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Conversation
@CyrusNajmabadi
@CyrusNajmabadiCyrusNajmabadi
commented
Oct 12, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Fixes#76953 for 8.0 and onwards

Ports dotnet/roslyn#64322 back to the runtime polyfill.
Corresponding 6.0 servicing fix is here: #76946

Corresponding 7.0 servicing fix is here: #76954
Customer Impact
Customers using 7.0.0 sdk can run into stack-overflows in the Source-Generator helper code code, in both VS and compiler, when compiling code with certain problematic code-constructs. This happens for tehse customers even if they are not using Source-Generators themselves as the overflow occurs in the code that is trying to determine if the generator should run.
This commonly happens when working with, or trying to compile, generated code produced by tools like T4 or Antlr (which commonly generate deeply recursive trees).
Examples of code that causes this are:
stringdata="..."+"..."+"..."+/* thousands more concatenations */+"..."+"..."+"...";
In a case like this, the concatenation code is not a balanced tree but instead, effectively, a linear tree like so:
image
The existing code works by recursing the user's parse tree, which ends up blowing the stack in cases like these.
As this is a stack overflow, it is fairly catastrophic for the user. Absent them changing this code outside of VS, which may not be possible for tool-generated code, the only workaround is to disable these generators (like System.Text.Json). Disabling the generator may be suitable for customers that are not using that generator, but will completely break users who are using it and who do have these code constructs.
Testing
Risk
Low. This is a simple port of a change made in roslyn to switch from implicit recursion, to using an explicit stack. This is a refactoring that roslyn is familiar with as stack-overflows are not uncommon for us as users do have these sorts of files, and recursive solutions have often had to be refactored in a similar fashion.
@ghost
ghost
commented
Oct 12, 2022

Copy link


Copy Markdown

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.
@build-analysisbuild-analysisBot
mentioned this pull request
Oct 13, 2022

Closed

2 tasks
@CyrusNajmabadiCyrusNajmabadi
mentioned this pull request
Oct 13, 2022

Closed

@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Can we take this? Or will the 7.0 fix naturally flow into main?
@joperezr
joperezr
commented
Oct 14, 2022

Copy link


Copy Markdown

Member
Yes, we can take it. 7.0 doesn't flow to main in dotnet/runtime, so we actually have to take it, I just missed the email for this PR. Will manually validate in a bit and merge once it is ready
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
Great, thanks!
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Just pinging to make sure this is not lost :)
@joperezr
joperezr
commented
Oct 20, 2022

Copy link


Copy Markdown

Member
Failures in OSX are unrelated, so I'll go ahead and merge. Thanks @CyrusNajmabadi!
@joperezr
joperezr
merged commit 6e6032c
into
dotnet:mainOct 20, 2022
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
Thanks!
@CyrusNajmabadi
CyrusNajmabadi
deleted the

generatorStackOverflowMain

branch
October 20, 2022 19:06
@ghostghost
locked as resolved and limited conversation to collaborators
Nov 20, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@joperezrjoperezrjoperezr approved these changes

Assignees

@CyrusNajmabadiCyrusNajmabadi

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

StackOverflowException on web project code analysis


2 participants

@CyrusNajmabadi@joperezr

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill by CyrusNajmabadi · Pull Request #76955 · dotnet/runtime · GitHub
Skip to content
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill - #76955
Merged
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Oct 20, 2022
Merged
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill#76955
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Conversation
@CyrusNajmabadi
@CyrusNajmabadiCyrusNajmabadi
commented
Oct 12, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Fixes#76953 for 8.0 and onwards

Ports dotnet/roslyn#64322 back to the runtime polyfill.
Corresponding 6.0 servicing fix is here: #76946

Corresponding 7.0 servicing fix is here: #76954
Customer Impact
Customers using 7.0.0 sdk can run into stack-overflows in the Source-Generator helper code code, in both VS and compiler, when compiling code with certain problematic code-constructs. This happens for tehse customers even if they are not using Source-Generators themselves as the overflow occurs in the code that is trying to determine if the generator should run.
This commonly happens when working with, or trying to compile, generated code produced by tools like T4 or Antlr (which commonly generate deeply recursive trees).
Examples of code that causes this are:
stringdata="..."+"..."+"..."+/* thousands more concatenations */+"..."+"..."+"...";
In a case like this, the concatenation code is not a balanced tree but instead, effectively, a linear tree like so:
image
The existing code works by recursing the user's parse tree, which ends up blowing the stack in cases like these.
As this is a stack overflow, it is fairly catastrophic for the user. Absent them changing this code outside of VS, which may not be possible for tool-generated code, the only workaround is to disable these generators (like System.Text.Json). Disabling the generator may be suitable for customers that are not using that generator, but will completely break users who are using it and who do have these code constructs.
Testing
Risk
Low. This is a simple port of a change made in roslyn to switch from implicit recursion, to using an explicit stack. This is a refactoring that roslyn is familiar with as stack-overflows are not uncommon for us as users do have these sorts of files, and recursive solutions have often had to be refactored in a similar fashion.
@ghost
ghost
commented
Oct 12, 2022

Copy link


Copy Markdown

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.
@build-analysisbuild-analysisBot
mentioned this pull request
Oct 13, 2022

Closed

2 tasks
@CyrusNajmabadiCyrusNajmabadi
mentioned this pull request
Oct 13, 2022

Closed

@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Can we take this? Or will the 7.0 fix naturally flow into main?
@joperezr
joperezr
commented
Oct 14, 2022

Copy link


Copy Markdown

Member
Yes, we can take it. 7.0 doesn't flow to main in dotnet/runtime, so we actually have to take it, I just missed the email for this PR. Will manually validate in a bit and merge once it is ready
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
Great, thanks!
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Just pinging to make sure this is not lost :)
@joperezr
joperezr
commented
Oct 20, 2022

Copy link


Copy Markdown

Member
Failures in OSX are unrelated, so I'll go ahead and merge. Thanks @CyrusNajmabadi!
@joperezr
joperezr
merged commit 6e6032c
into
dotnet:mainOct 20, 2022
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
Thanks!
@CyrusNajmabadi
CyrusNajmabadi
deleted the

generatorStackOverflowMain

branch
October 20, 2022 19:06
@ghostghost
locked as resolved and limited conversation to collaborators
Nov 20, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@joperezrjoperezrjoperezr approved these changes

Assignees

@CyrusNajmabadiCyrusNajmabadi

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

StackOverflowException on web project code analysis


2 participants

@CyrusNajmabadi@joperezr

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill by CyrusNajmabadi · Pull Request #76955 · dotnet/runtime · GitHub
Skip to content
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill - #76955
Merged
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Oct 20, 2022
Merged
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill#76955
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Conversation
@CyrusNajmabadi
@CyrusNajmabadiCyrusNajmabadi
commented
Oct 12, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Fixes#76953 for 8.0 and onwards

Ports dotnet/roslyn#64322 back to the runtime polyfill.
Corresponding 6.0 servicing fix is here: #76946

Corresponding 7.0 servicing fix is here: #76954
Customer Impact
Customers using 7.0.0 sdk can run into stack-overflows in the Source-Generator helper code code, in both VS and compiler, when compiling code with certain problematic code-constructs. This happens for tehse customers even if they are not using Source-Generators themselves as the overflow occurs in the code that is trying to determine if the generator should run.
This commonly happens when working with, or trying to compile, generated code produced by tools like T4 or Antlr (which commonly generate deeply recursive trees).
Examples of code that causes this are:
stringdata="..."+"..."+"..."+/* thousands more concatenations */+"..."+"..."+"...";
In a case like this, the concatenation code is not a balanced tree but instead, effectively, a linear tree like so:
image
The existing code works by recursing the user's parse tree, which ends up blowing the stack in cases like these.
As this is a stack overflow, it is fairly catastrophic for the user. Absent them changing this code outside of VS, which may not be possible for tool-generated code, the only workaround is to disable these generators (like System.Text.Json). Disabling the generator may be suitable for customers that are not using that generator, but will completely break users who are using it and who do have these code constructs.
Testing
Risk
Low. This is a simple port of a change made in roslyn to switch from implicit recursion, to using an explicit stack. This is a refactoring that roslyn is familiar with as stack-overflows are not uncommon for us as users do have these sorts of files, and recursive solutions have often had to be refactored in a similar fashion.
@ghost
ghost
commented
Oct 12, 2022

Copy link


Copy Markdown

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.
@build-analysisbuild-analysisBot
mentioned this pull request
Oct 13, 2022

Closed

2 tasks
@CyrusNajmabadiCyrusNajmabadi
mentioned this pull request
Oct 13, 2022

Closed

@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Can we take this? Or will the 7.0 fix naturally flow into main?
@joperezr
joperezr
commented
Oct 14, 2022

Copy link


Copy Markdown

Member
Yes, we can take it. 7.0 doesn't flow to main in dotnet/runtime, so we actually have to take it, I just missed the email for this PR. Will manually validate in a bit and merge once it is ready
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
Great, thanks!
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Just pinging to make sure this is not lost :)
@joperezr
joperezr
commented
Oct 20, 2022

Copy link


Copy Markdown

Member
Failures in OSX are unrelated, so I'll go ahead and merge. Thanks @CyrusNajmabadi!
@joperezr
joperezr
merged commit 6e6032c
into
dotnet:mainOct 20, 2022
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
Thanks!
@CyrusNajmabadi
CyrusNajmabadi
deleted the

generatorStackOverflowMain

branch
October 20, 2022 19:06
@ghostghost
locked as resolved and limited conversation to collaborators
Nov 20, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@joperezrjoperezrjoperezr approved these changes

Assignees

@CyrusNajmabadiCyrusNajmabadi

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

StackOverflowException on web project code analysis


2 participants

@CyrusNajmabadi@joperezr

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill by CyrusNajmabadi · Pull Request #76955 · dotnet/runtime · GitHub
Skip to content
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill - #76955
Merged
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Oct 20, 2022
Merged
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill#76955
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Conversation
@CyrusNajmabadi
@CyrusNajmabadiCyrusNajmabadi
commented
Oct 12, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Fixes#76953 for 8.0 and onwards

Ports dotnet/roslyn#64322 back to the runtime polyfill.
Corresponding 6.0 servicing fix is here: #76946

Corresponding 7.0 servicing fix is here: #76954
Customer Impact
Customers using 7.0.0 sdk can run into stack-overflows in the Source-Generator helper code code, in both VS and compiler, when compiling code with certain problematic code-constructs. This happens for tehse customers even if they are not using Source-Generators themselves as the overflow occurs in the code that is trying to determine if the generator should run.
This commonly happens when working with, or trying to compile, generated code produced by tools like T4 or Antlr (which commonly generate deeply recursive trees).
Examples of code that causes this are:
stringdata="..."+"..."+"..."+/* thousands more concatenations */+"..."+"..."+"...";
In a case like this, the concatenation code is not a balanced tree but instead, effectively, a linear tree like so:
image
The existing code works by recursing the user's parse tree, which ends up blowing the stack in cases like these.
As this is a stack overflow, it is fairly catastrophic for the user. Absent them changing this code outside of VS, which may not be possible for tool-generated code, the only workaround is to disable these generators (like System.Text.Json). Disabling the generator may be suitable for customers that are not using that generator, but will completely break users who are using it and who do have these code constructs.
Testing
Risk
Low. This is a simple port of a change made in roslyn to switch from implicit recursion, to using an explicit stack. This is a refactoring that roslyn is familiar with as stack-overflows are not uncommon for us as users do have these sorts of files, and recursive solutions have often had to be refactored in a similar fashion.
@ghost
ghost
commented
Oct 12, 2022

Copy link


Copy Markdown

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.
@build-analysisbuild-analysisBot
mentioned this pull request
Oct 13, 2022

Closed

2 tasks
@CyrusNajmabadiCyrusNajmabadi
mentioned this pull request
Oct 13, 2022

Closed

@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Can we take this? Or will the 7.0 fix naturally flow into main?
@joperezr
joperezr
commented
Oct 14, 2022

Copy link


Copy Markdown

Member
Yes, we can take it. 7.0 doesn't flow to main in dotnet/runtime, so we actually have to take it, I just missed the email for this PR. Will manually validate in a bit and merge once it is ready
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
Great, thanks!
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Just pinging to make sure this is not lost :)
@joperezr
joperezr
commented
Oct 20, 2022

Copy link


Copy Markdown

Member
Failures in OSX are unrelated, so I'll go ahead and merge. Thanks @CyrusNajmabadi!
@joperezr
joperezr
merged commit 6e6032c
into
dotnet:mainOct 20, 2022
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
Thanks!
@CyrusNajmabadi
CyrusNajmabadi
deleted the

generatorStackOverflowMain

branch
October 20, 2022 19:06
@ghostghost
locked as resolved and limited conversation to collaborators
Nov 20, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@joperezrjoperezrjoperezr approved these changes

Assignees

@CyrusNajmabadiCyrusNajmabadi

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

StackOverflowException on web project code analysis


2 participants

@CyrusNajmabadi@joperezr

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill by CyrusNajmabadi · Pull Request #76955 · dotnet/runtime · GitHub
Skip to content
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill - #76955
Merged
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Oct 20, 2022
Merged
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill#76955
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Conversation
@CyrusNajmabadi
@CyrusNajmabadiCyrusNajmabadi
commented
Oct 12, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Fixes#76953 for 8.0 and onwards

Ports dotnet/roslyn#64322 back to the runtime polyfill.
Corresponding 6.0 servicing fix is here: #76946

Corresponding 7.0 servicing fix is here: #76954
Customer Impact
Customers using 7.0.0 sdk can run into stack-overflows in the Source-Generator helper code code, in both VS and compiler, when compiling code with certain problematic code-constructs. This happens for tehse customers even if they are not using Source-Generators themselves as the overflow occurs in the code that is trying to determine if the generator should run.
This commonly happens when working with, or trying to compile, generated code produced by tools like T4 or Antlr (which commonly generate deeply recursive trees).
Examples of code that causes this are:
stringdata="..."+"..."+"..."+/* thousands more concatenations */+"..."+"..."+"...";
In a case like this, the concatenation code is not a balanced tree but instead, effectively, a linear tree like so:
image
The existing code works by recursing the user's parse tree, which ends up blowing the stack in cases like these.
As this is a stack overflow, it is fairly catastrophic for the user. Absent them changing this code outside of VS, which may not be possible for tool-generated code, the only workaround is to disable these generators (like System.Text.Json). Disabling the generator may be suitable for customers that are not using that generator, but will completely break users who are using it and who do have these code constructs.
Testing
Risk
Low. This is a simple port of a change made in roslyn to switch from implicit recursion, to using an explicit stack. This is a refactoring that roslyn is familiar with as stack-overflows are not uncommon for us as users do have these sorts of files, and recursive solutions have often had to be refactored in a similar fashion.
@ghost
ghost
commented
Oct 12, 2022

Copy link


Copy Markdown

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.
@build-analysisbuild-analysisBot
mentioned this pull request
Oct 13, 2022

Closed

2 tasks
@CyrusNajmabadiCyrusNajmabadi
mentioned this pull request
Oct 13, 2022

Closed

@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Can we take this? Or will the 7.0 fix naturally flow into main?
@joperezr
joperezr
commented
Oct 14, 2022

Copy link


Copy Markdown

Member
Yes, we can take it. 7.0 doesn't flow to main in dotnet/runtime, so we actually have to take it, I just missed the email for this PR. Will manually validate in a bit and merge once it is ready
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
Great, thanks!
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Just pinging to make sure this is not lost :)
@joperezr
joperezr
commented
Oct 20, 2022

Copy link


Copy Markdown

Member
Failures in OSX are unrelated, so I'll go ahead and merge. Thanks @CyrusNajmabadi!
@joperezr
joperezr
merged commit 6e6032c
into
dotnet:mainOct 20, 2022
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
Thanks!
@CyrusNajmabadi
CyrusNajmabadi
deleted the

generatorStackOverflowMain

branch
October 20, 2022 19:06
@ghostghost
locked as resolved and limited conversation to collaborators
Nov 20, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@joperezrjoperezrjoperezr approved these changes

Assignees

@CyrusNajmabadiCyrusNajmabadi

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

StackOverflowException on web project code analysis


2 participants

@CyrusNajmabadi@joperezr

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill by CyrusNajmabadi · Pull Request #76955 · dotnet/runtime · GitHub
Skip to content
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill - #76955
Merged
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Oct 20, 2022
Merged
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill#76955
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Conversation
@CyrusNajmabadi
@CyrusNajmabadiCyrusNajmabadi
commented
Oct 12, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Fixes#76953 for 8.0 and onwards

Ports dotnet/roslyn#64322 back to the runtime polyfill.
Corresponding 6.0 servicing fix is here: #76946

Corresponding 7.0 servicing fix is here: #76954
Customer Impact
Customers using 7.0.0 sdk can run into stack-overflows in the Source-Generator helper code code, in both VS and compiler, when compiling code with certain problematic code-constructs. This happens for tehse customers even if they are not using Source-Generators themselves as the overflow occurs in the code that is trying to determine if the generator should run.
This commonly happens when working with, or trying to compile, generated code produced by tools like T4 or Antlr (which commonly generate deeply recursive trees).
Examples of code that causes this are:
stringdata="..."+"..."+"..."+/* thousands more concatenations */+"..."+"..."+"...";
In a case like this, the concatenation code is not a balanced tree but instead, effectively, a linear tree like so:
image
The existing code works by recursing the user's parse tree, which ends up blowing the stack in cases like these.
As this is a stack overflow, it is fairly catastrophic for the user. Absent them changing this code outside of VS, which may not be possible for tool-generated code, the only workaround is to disable these generators (like System.Text.Json). Disabling the generator may be suitable for customers that are not using that generator, but will completely break users who are using it and who do have these code constructs.
Testing
Risk
Low. This is a simple port of a change made in roslyn to switch from implicit recursion, to using an explicit stack. This is a refactoring that roslyn is familiar with as stack-overflows are not uncommon for us as users do have these sorts of files, and recursive solutions have often had to be refactored in a similar fashion.
@ghost
ghost
commented
Oct 12, 2022

Copy link


Copy Markdown

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.
@build-analysisbuild-analysisBot
mentioned this pull request
Oct 13, 2022

Closed

2 tasks
@CyrusNajmabadiCyrusNajmabadi
mentioned this pull request
Oct 13, 2022

Closed

@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Can we take this? Or will the 7.0 fix naturally flow into main?
@joperezr
joperezr
commented
Oct 14, 2022

Copy link


Copy Markdown

Member
Yes, we can take it. 7.0 doesn't flow to main in dotnet/runtime, so we actually have to take it, I just missed the email for this PR. Will manually validate in a bit and merge once it is ready
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
Great, thanks!
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Just pinging to make sure this is not lost :)
@joperezr
joperezr
commented
Oct 20, 2022

Copy link


Copy Markdown

Member
Failures in OSX are unrelated, so I'll go ahead and merge. Thanks @CyrusNajmabadi!
@joperezr
joperezr
merged commit 6e6032c
into
dotnet:mainOct 20, 2022
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
Thanks!
@CyrusNajmabadi
CyrusNajmabadi
deleted the

generatorStackOverflowMain

branch
October 20, 2022 19:06
@ghostghost
locked as resolved and limited conversation to collaborators
Nov 20, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@joperezrjoperezrjoperezr approved these changes

Assignees

@CyrusNajmabadiCyrusNajmabadi

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

StackOverflowException on web project code analysis


2 participants

@CyrusNajmabadi@joperezr

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill by CyrusNajmabadi · Pull Request #76955 · dotnet/runtime · GitHub
Skip to content
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill - #76955
Merged
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Oct 20, 2022
Merged
Port stackoverflow fix from Roslyn to SourceGenerator PolyFill#76955
joperezr merged 1 commit into
dotnet:mainfrom 
CyrusNajmabadi:generatorStackOverflowMain
Conversation
@CyrusNajmabadi
@CyrusNajmabadiCyrusNajmabadi
commented
Oct 12, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Fixes#76953 for 8.0 and onwards

Ports dotnet/roslyn#64322 back to the runtime polyfill.
Corresponding 6.0 servicing fix is here: #76946

Corresponding 7.0 servicing fix is here: #76954
Customer Impact
Customers using 7.0.0 sdk can run into stack-overflows in the Source-Generator helper code code, in both VS and compiler, when compiling code with certain problematic code-constructs. This happens for tehse customers even if they are not using Source-Generators themselves as the overflow occurs in the code that is trying to determine if the generator should run.
This commonly happens when working with, or trying to compile, generated code produced by tools like T4 or Antlr (which commonly generate deeply recursive trees).
Examples of code that causes this are:
stringdata="..."+"..."+"..."+/* thousands more concatenations */+"..."+"..."+"...";
In a case like this, the concatenation code is not a balanced tree but instead, effectively, a linear tree like so:
image
The existing code works by recursing the user's parse tree, which ends up blowing the stack in cases like these.
As this is a stack overflow, it is fairly catastrophic for the user. Absent them changing this code outside of VS, which may not be possible for tool-generated code, the only workaround is to disable these generators (like System.Text.Json). Disabling the generator may be suitable for customers that are not using that generator, but will completely break users who are using it and who do have these code constructs.
Testing
Risk
Low. This is a simple port of a change made in roslyn to switch from implicit recursion, to using an explicit stack. This is a refactoring that roslyn is familiar with as stack-overflows are not uncommon for us as users do have these sorts of files, and recursive solutions have often had to be refactored in a similar fashion.
@ghost
ghost
commented
Oct 12, 2022

Copy link


Copy Markdown

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.
@build-analysisbuild-analysisBot
mentioned this pull request
Oct 13, 2022

Closed

2 tasks
@CyrusNajmabadiCyrusNajmabadi
mentioned this pull request
Oct 13, 2022

Closed

@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Can we take this? Or will the 7.0 fix naturally flow into main?
@joperezr
joperezr
commented
Oct 14, 2022

Copy link


Copy Markdown

Member
Yes, we can take it. 7.0 doesn't flow to main in dotnet/runtime, so we actually have to take it, I just missed the email for this PR. Will manually validate in a bit and merge once it is ready
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 14, 2022

Copy link


Copy Markdown

ContributorAuthor
Great, thanks!
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
@joperezr Just pinging to make sure this is not lost :)
@joperezr
joperezr
commented
Oct 20, 2022

Copy link


Copy Markdown

Member
Failures in OSX are unrelated, so I'll go ahead and merge. Thanks @CyrusNajmabadi!
@joperezr
joperezr
merged commit 6e6032c
into
dotnet:mainOct 20, 2022
@CyrusNajmabadi
CyrusNajmabadi
commented
Oct 20, 2022

Copy link


Copy Markdown

ContributorAuthor
Thanks!
@CyrusNajmabadi
CyrusNajmabadi
deleted the

generatorStackOverflowMain

branch
October 20, 2022 19:06
@ghostghost
locked as resolved and limited conversation to collaborators
Nov 20, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@joperezrjoperezrjoperezr approved these changes

Assignees

@CyrusNajmabadiCyrusNajmabadi

Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

StackOverflowException on web project code analysis


2 participants

@CyrusNajmabadi@joperezr