Error out when struct size is bigger than int.MaxValue - #104393

Merged
fanyang-mono merged 20 commits into
dotnet:mainfrom
fanyang-mono:fix_overflow
Aug 21, 2024
Merged

Error out when struct size is bigger than int.MaxValue#104393
fanyang-mono merged 20 commits into
dotnet:mainfrom
fanyang-mono:fix_overflow

Conversation

@fanyang-mono

@fanyang-monofanyang-mono commented Jul 3, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Fixes: #97412

sizeof() in C# returns a signed integer. This PR is to adjust the runtime to match with that.

Added a check for struct size and a test.

@ghostghost added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jul 3, 2024
@jkotas

jkotas commented Jul 3, 2024

Copy link
Copy Markdown
Member

Do we need a test?

jkotas
jkotas reviewed Jul 3, 2024
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/Resources/Strings.resx Outdated
jkotas
jkotas reviewed Jul 3, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/classlayoutinfo.cpp Outdated
This was referenced Jul 3, 2024
Open
Closed
@fanyang-monofanyang-mono added area-TypeSystem-coreclr and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Jul 8, 2024
@build-analysisbuild-analysisBot mentioned this pull request Jul 9, 2024
Closed
3 tasks
@fanyang-mono

fanyang-mono commented Jul 9, 2024•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

@jkotas This PR is ready to be reviewed again.

@MichalStrehovsky
MichalStrehovsky self-requested a review as a code ownerJuly 11, 2024 15:06
@MichalStrehovsky

MichalStrehovsky commented Jul 11, 2024

Copy link
Copy Markdown
Member

I've pushed out a change that makes further progress in the handling within the managed type system, but the test probably still won't compile because BigArray with size of int.MaxValue gets its size bumped past int.MaxValue through alignment after we're done with layout (the final size needs to be divisible by pointer size and int.MaxValue isn't.

CoreCLR is fine with this overflowing but the managed type system isn't. I'm not sure if the overflow is an actual problem but fixing it doesn't look exactly trivial.

jkotas
jkotas reviewed Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/methodtablebuilder.cpp Outdated
jkotas
jkotas reviewed Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/tools/Common/TypeSystem/Common/LayoutInt.cs
@fanyang-mono

fanyang-mono commented Jul 15, 2024

Copy link
Copy Markdown
MemberAuthor

@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?

@MichalStrehovsky

MichalStrehovsky commented Jul 15, 2024

Copy link
Copy Markdown
Member

@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?

Only by placing <NativeAotIncompatible>true</NativeAotIncomplatible> in the project file. It would be good not to disable all existing ManagedSequential type loader testing just because the new test doesn't work (i.e. put it in a separate project file).

Note that this is the managed type system that is shared with crossgen2. Might want to run crossgen2 outerloop testing to see whether this also needs to be removed from precompilation there.

The managed type system doesn't handle this test because the size of the BigArray struct is actually over 2 GB when boxed (this is also true on JIT-based CoreCLR). If a 2 GB overflow is problematic, is the 2 GB overflow of that one not problematic?

@fanyang-mono

fanyang-mono commented Jul 15, 2024

Copy link
Copy Markdown
MemberAuthor

@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?

@MichalStrehovsky

MichalStrehovsky commented Jul 15, 2024

Copy link
Copy Markdown
Member

@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?

The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:

BigArray.Instance=newBigArray();longbefore=GC.GetAllocatedBytesForCurrentThread();BigArray.Instance=newBigArray();Console.WriteLine(GC.GetAllocatedBytesForCurrentThread()-before);[StructLayout(LayoutKind.Sequential,Size=int.MaxValue)]structBigArray{publicstaticobjectInstance;}

Will print 2,147,483,664, which is larger than int.MaxValue.

@fanyang-mono

fanyang-mono commented Jul 15, 2024

Copy link
Copy Markdown
MemberAuthor

/azp list

@azure-pipelines

azure-pipelinesBot commented Jul 15, 2024

Copy link
Copy Markdown
CI/CD Pipelines for this repository:

@MichalStrehovsky

MichalStrehovsky commented Jul 15, 2024

Copy link
Copy Markdown
Member

The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:

We chatted with Fan on Teams about this and apparently the program I posted only works on a Release build (also without debugger attached) and throws InvalidProgram otherwise. On .NET 8 CoreCLR-JIT. Curious.

This was referenced Aug 7, 2024
Closed
Closed
Closed
@fanyang-mono

fanyang-mono commented Aug 7, 2024

Copy link
Copy Markdown
MemberAuthor

@MichalStrehovsky CoreCLR and Mono side of work has been completed and approved. The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.

Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change. @jkotas Please correct me, if I were wrong.

@MichalStrehovsky

MichalStrehovsky commented Aug 7, 2024

Copy link
Copy Markdown
Member

The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.

I'll not have time to look at this any time soon. Might be better to just block the test on an issue and not block this PR.

Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change.

Throwing an exception in a situation that didn't throw before is the definition of a breaking change.

@fanyang-mono

fanyang-mono commented Aug 7, 2024

Copy link
Copy Markdown
MemberAuthor

Discussed with @jkotas offline. He suggested to wait for .NET10 to merge it, together with the NativeAOT change, when it is ready.

There is no rush. @MichalStrehovsky

@steveisoksteveisok mentioned this pull request Aug 8, 2024
Open
@fanyang-mono

fanyang-mono commented Aug 9, 2024

Copy link
Copy Markdown
MemberAuthor

I plan to merge this PR when main is .NET10. The managed type system part used by NativeAOT and crossgen can be worked on later during .NET10 release.

@fanyang-mono

fanyang-mono commented Aug 9, 2024

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-coreclr crossgen2

@azure-pipelines

azure-pipelinesBot commented Aug 9, 2024

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@build-analysisbuild-analysisBot mentioned this pull request Aug 9, 2024
Open
This was referenced Aug 10, 2024
Closed
Closed
@fanyang-mono

fanyang-mono commented Aug 11, 2024

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-coreclr crossgen2

@azure-pipelines

azure-pipelinesBot commented Aug 11, 2024

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@fanyang-mono

fanyang-mono commented Aug 15, 2024

Copy link
Copy Markdown
MemberAuthor

/azp run runtime-coreclr crossgen2

@azure-pipelines

azure-pipelinesBot commented Aug 15, 2024

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@fanyang-mono

fanyang-mono commented Aug 21, 2024

Copy link
Copy Markdown
MemberAuthor

CI failures are unrelated to this PR.

@fanyang-mono
fanyang-mono merged commit 423faed into dotnet:mainAug 21, 2024
@matouskozakmatouskozak mentioned this pull request Aug 22, 2024
Closed
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Sep 21, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@MichalStrehovskyMichalStrehovskyMichalStrehovsky left review comments
@lambdageeklambdageeklambdageek approved these changes
@jkotasjkotasjkotas approved these changes
@steveisoksteveisokAwaiting requested review from steveisoksteveisok is a code owner
@davidwrightondavidwrightonAwaiting requested review from davidwrighton
@VSadovVSadovAwaiting requested review from VSadov
@thaystgthaystgAwaiting requested review from thaystgthaystg is a code owner

Assignees

@fanyang-monofanyang-mono

Labels

area-TypeSystem-coreclr

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

The type loader has silent integer overflows

4 participants

@fanyang-mono@jkotas@MichalStrehovsky@lambdageek
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Error out when struct size is bigger than int.MaxValue - #104393
Merged
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Aug 21, 2024
Merged
Error out when struct size is bigger than int.MaxValue#104393
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Conversation
@fanyang-mono
@fanyang-monofanyang-mono
commented
Jul 3, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Fixes: #97412
sizeof() in C# returns a signed integer. This PR is to adjust the runtime to match with that.
Added a check for struct size and a test.
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Jul 3, 2024
@jkotas
jkotas
commented
Jul 3, 2024

Copy link


Copy Markdown

Member
Do we need a test?
jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/Resources/Strings.resx
Outdated

jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/classlayoutinfo.cpp
Outdated


This was referenced Jul 3, 2024

Open


Closed

@fanyang-monofanyang-mono
added

area-TypeSystem-coreclr

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Jul 8, 2024
@build-analysisbuild-analysisBot
mentioned this pull request
Jul 9, 2024

Closed

3 tasks
@fanyang-mono
fanyang-mono
commented
Jul 9, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
@jkotas This PR is ready to be reviewed again.
@MichalStrehovsky
MichalStrehovsky
self-requested a review
as a code ownerJuly 11, 2024 15:06
@MichalStrehovsky
MichalStrehovsky
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
I've pushed out a change that makes further progress in the handling within the managed type system, but the test probably still won't compile because BigArray with size of int.MaxValue gets its size bumped past int.MaxValue through alignment after we're done with layout (the final size needs to be divisible by pointer size and int.MaxValue isn't.
CoreCLR is fine with this overflowing but the managed type system isn't. I'm not sure if the overflow is an actual problem but fixing it doesn't look exactly trivial.
jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/methodtablebuilder.cpp
Outdated

jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/tools/Common/TypeSystem/Common/LayoutInt.cs
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
Only by placing <NativeAotIncompatible>true</NativeAotIncomplatible> in the project file. It would be good not to disable all existing ManagedSequential type loader testing just because the new test doesn't work (i.e. put it in a separate project file).
Note that this is the managed type system that is shared with crossgen2. Might want to run crossgen2 outerloop testing to see whether this also needs to be removed from precompilation there.
The managed type system doesn't handle this test because the size of the BigArray struct is actually over 2 GB when boxed (this is also true on JIT-based CoreCLR). If a 2 GB overflow is problematic, is the 2 GB overflow of that one not problematic?
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
BigArray.Instance=newBigArray();longbefore=GC.GetAllocatedBytesForCurrentThread();BigArray.Instance=newBigArray();Console.WriteLine(GC.GetAllocatedBytesForCurrentThread()-before);[StructLayout(LayoutKind.Sequential,Size=int.MaxValue)]structBigArray{publicstaticobjectInstance;}
Will print 2,147,483,664, which is larger than int.MaxValue.
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp list
@azure-pipelines
azure-pipelinesBot
commented
Jul 15, 2024

Copy link


Copy Markdown


CI/CD Pipelines for this repository:
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
We chatted with Fan on Teams about this and apparently the program I posted only works on a Release build (also without debugger attached) and throws InvalidProgram otherwise. On .NET 8 CoreCLR-JIT. Curious.

This was referenced Aug 7, 2024

Closed


Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky CoreCLR and Mono side of work has been completed and approved. The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change. @jkotas Please correct me, if I were wrong.
@MichalStrehovsky
MichalStrehovsky
commented
Aug 7, 2024

Copy link


Copy Markdown

Member
The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
I'll not have time to look at this any time soon. Might be better to just block the test on an issue and not block this PR.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change.
Throwing an exception in a situation that didn't throw before is the definition of a breaking change.
@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
Discussed with @jkotas offline. He suggested to wait for .NET10 to merge it, together with the NativeAOT change, when it is ready.
There is no rush. @MichalStrehovsky
@steveisoksteveisok
mentioned this pull request
Aug 8, 2024

Open

@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
I plan to merge this PR when main is .NET10. The managed type system part used by NativeAOT and crossgen can be worked on later during .NET10 release.
@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 9, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Aug 9, 2024

Open


This was referenced Aug 10, 2024

Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 11, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 11, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 15, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 21, 2024

Copy link


Copy Markdown

MemberAuthor
CI failures are unrelated to this PR.
@fanyang-mono
fanyang-mono
merged commit 423faed
into
dotnet:mainAug 21, 2024
@matouskozakmatouskozak
mentioned this pull request
Aug 22, 2024

Closed

@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Sep 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@MichalStrehovskyMichalStrehovskyMichalStrehovsky left review comments
@lambdageeklambdageeklambdageek approved these changes
@jkotasjkotasjkotas approved these changes
@steveisoksteveisokAwaiting requested review from steveisoksteveisok is a code owner
@davidwrightondavidwrightonAwaiting requested review from davidwrighton
@VSadovVSadovAwaiting requested review from VSadov
@thaystgthaystgAwaiting requested review from thaystgthaystg is a code owner

Assignees

@fanyang-monofanyang-mono

Labels

area-TypeSystem-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

The type loader has silent integer overflows


4 participants

@fanyang-mono@jkotas@MichalStrehovsky@lambdageek

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Error out when struct size is bigger than int.MaxValue - #104393
Merged
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Aug 21, 2024
Merged
Error out when struct size is bigger than int.MaxValue#104393
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Conversation
@fanyang-mono
@fanyang-monofanyang-mono
commented
Jul 3, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Fixes: #97412
sizeof() in C# returns a signed integer. This PR is to adjust the runtime to match with that.
Added a check for struct size and a test.
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Jul 3, 2024
@jkotas
jkotas
commented
Jul 3, 2024

Copy link


Copy Markdown

Member
Do we need a test?
jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/Resources/Strings.resx
Outdated

jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/classlayoutinfo.cpp
Outdated


This was referenced Jul 3, 2024

Open


Closed

@fanyang-monofanyang-mono
added

area-TypeSystem-coreclr

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Jul 8, 2024
@build-analysisbuild-analysisBot
mentioned this pull request
Jul 9, 2024

Closed

3 tasks
@fanyang-mono
fanyang-mono
commented
Jul 9, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
@jkotas This PR is ready to be reviewed again.
@MichalStrehovsky
MichalStrehovsky
self-requested a review
as a code ownerJuly 11, 2024 15:06
@MichalStrehovsky
MichalStrehovsky
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
I've pushed out a change that makes further progress in the handling within the managed type system, but the test probably still won't compile because BigArray with size of int.MaxValue gets its size bumped past int.MaxValue through alignment after we're done with layout (the final size needs to be divisible by pointer size and int.MaxValue isn't.
CoreCLR is fine with this overflowing but the managed type system isn't. I'm not sure if the overflow is an actual problem but fixing it doesn't look exactly trivial.
jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/methodtablebuilder.cpp
Outdated

jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/tools/Common/TypeSystem/Common/LayoutInt.cs
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
Only by placing <NativeAotIncompatible>true</NativeAotIncomplatible> in the project file. It would be good not to disable all existing ManagedSequential type loader testing just because the new test doesn't work (i.e. put it in a separate project file).
Note that this is the managed type system that is shared with crossgen2. Might want to run crossgen2 outerloop testing to see whether this also needs to be removed from precompilation there.
The managed type system doesn't handle this test because the size of the BigArray struct is actually over 2 GB when boxed (this is also true on JIT-based CoreCLR). If a 2 GB overflow is problematic, is the 2 GB overflow of that one not problematic?
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
BigArray.Instance=newBigArray();longbefore=GC.GetAllocatedBytesForCurrentThread();BigArray.Instance=newBigArray();Console.WriteLine(GC.GetAllocatedBytesForCurrentThread()-before);[StructLayout(LayoutKind.Sequential,Size=int.MaxValue)]structBigArray{publicstaticobjectInstance;}
Will print 2,147,483,664, which is larger than int.MaxValue.
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp list
@azure-pipelines
azure-pipelinesBot
commented
Jul 15, 2024

Copy link


Copy Markdown


CI/CD Pipelines for this repository:
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
We chatted with Fan on Teams about this and apparently the program I posted only works on a Release build (also without debugger attached) and throws InvalidProgram otherwise. On .NET 8 CoreCLR-JIT. Curious.

This was referenced Aug 7, 2024

Closed


Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky CoreCLR and Mono side of work has been completed and approved. The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change. @jkotas Please correct me, if I were wrong.
@MichalStrehovsky
MichalStrehovsky
commented
Aug 7, 2024

Copy link


Copy Markdown

Member
The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
I'll not have time to look at this any time soon. Might be better to just block the test on an issue and not block this PR.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change.
Throwing an exception in a situation that didn't throw before is the definition of a breaking change.
@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
Discussed with @jkotas offline. He suggested to wait for .NET10 to merge it, together with the NativeAOT change, when it is ready.
There is no rush. @MichalStrehovsky
@steveisoksteveisok
mentioned this pull request
Aug 8, 2024

Open

@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
I plan to merge this PR when main is .NET10. The managed type system part used by NativeAOT and crossgen can be worked on later during .NET10 release.
@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 9, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Aug 9, 2024

Open


This was referenced Aug 10, 2024

Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 11, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 11, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 15, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 21, 2024

Copy link


Copy Markdown

MemberAuthor
CI failures are unrelated to this PR.
@fanyang-mono
fanyang-mono
merged commit 423faed
into
dotnet:mainAug 21, 2024
@matouskozakmatouskozak
mentioned this pull request
Aug 22, 2024

Closed

@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Sep 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@MichalStrehovskyMichalStrehovskyMichalStrehovsky left review comments
@lambdageeklambdageeklambdageek approved these changes
@jkotasjkotasjkotas approved these changes
@steveisoksteveisokAwaiting requested review from steveisoksteveisok is a code owner
@davidwrightondavidwrightonAwaiting requested review from davidwrighton
@VSadovVSadovAwaiting requested review from VSadov
@thaystgthaystgAwaiting requested review from thaystgthaystg is a code owner

Assignees

@fanyang-monofanyang-mono

Labels

area-TypeSystem-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

The type loader has silent integer overflows


4 participants

@fanyang-mono@jkotas@MichalStrehovsky@lambdageek

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Error out when struct size is bigger than int.MaxValue - #104393
Merged
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Aug 21, 2024
Merged
Error out when struct size is bigger than int.MaxValue#104393
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Conversation
@fanyang-mono
@fanyang-monofanyang-mono
commented
Jul 3, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Fixes: #97412
sizeof() in C# returns a signed integer. This PR is to adjust the runtime to match with that.
Added a check for struct size and a test.
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Jul 3, 2024
@jkotas
jkotas
commented
Jul 3, 2024

Copy link


Copy Markdown

Member
Do we need a test?
jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/Resources/Strings.resx
Outdated

jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/classlayoutinfo.cpp
Outdated


This was referenced Jul 3, 2024

Open


Closed

@fanyang-monofanyang-mono
added

area-TypeSystem-coreclr

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Jul 8, 2024
@build-analysisbuild-analysisBot
mentioned this pull request
Jul 9, 2024

Closed

3 tasks
@fanyang-mono
fanyang-mono
commented
Jul 9, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
@jkotas This PR is ready to be reviewed again.
@MichalStrehovsky
MichalStrehovsky
self-requested a review
as a code ownerJuly 11, 2024 15:06
@MichalStrehovsky
MichalStrehovsky
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
I've pushed out a change that makes further progress in the handling within the managed type system, but the test probably still won't compile because BigArray with size of int.MaxValue gets its size bumped past int.MaxValue through alignment after we're done with layout (the final size needs to be divisible by pointer size and int.MaxValue isn't.
CoreCLR is fine with this overflowing but the managed type system isn't. I'm not sure if the overflow is an actual problem but fixing it doesn't look exactly trivial.
jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/methodtablebuilder.cpp
Outdated

jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/tools/Common/TypeSystem/Common/LayoutInt.cs
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
Only by placing <NativeAotIncompatible>true</NativeAotIncomplatible> in the project file. It would be good not to disable all existing ManagedSequential type loader testing just because the new test doesn't work (i.e. put it in a separate project file).
Note that this is the managed type system that is shared with crossgen2. Might want to run crossgen2 outerloop testing to see whether this also needs to be removed from precompilation there.
The managed type system doesn't handle this test because the size of the BigArray struct is actually over 2 GB when boxed (this is also true on JIT-based CoreCLR). If a 2 GB overflow is problematic, is the 2 GB overflow of that one not problematic?
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
BigArray.Instance=newBigArray();longbefore=GC.GetAllocatedBytesForCurrentThread();BigArray.Instance=newBigArray();Console.WriteLine(GC.GetAllocatedBytesForCurrentThread()-before);[StructLayout(LayoutKind.Sequential,Size=int.MaxValue)]structBigArray{publicstaticobjectInstance;}
Will print 2,147,483,664, which is larger than int.MaxValue.
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp list
@azure-pipelines
azure-pipelinesBot
commented
Jul 15, 2024

Copy link


Copy Markdown


CI/CD Pipelines for this repository:
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
We chatted with Fan on Teams about this and apparently the program I posted only works on a Release build (also without debugger attached) and throws InvalidProgram otherwise. On .NET 8 CoreCLR-JIT. Curious.

This was referenced Aug 7, 2024

Closed


Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky CoreCLR and Mono side of work has been completed and approved. The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change. @jkotas Please correct me, if I were wrong.
@MichalStrehovsky
MichalStrehovsky
commented
Aug 7, 2024

Copy link


Copy Markdown

Member
The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
I'll not have time to look at this any time soon. Might be better to just block the test on an issue and not block this PR.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change.
Throwing an exception in a situation that didn't throw before is the definition of a breaking change.
@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
Discussed with @jkotas offline. He suggested to wait for .NET10 to merge it, together with the NativeAOT change, when it is ready.
There is no rush. @MichalStrehovsky
@steveisoksteveisok
mentioned this pull request
Aug 8, 2024

Open

@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
I plan to merge this PR when main is .NET10. The managed type system part used by NativeAOT and crossgen can be worked on later during .NET10 release.
@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 9, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Aug 9, 2024

Open


This was referenced Aug 10, 2024

Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 11, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 11, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 15, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 21, 2024

Copy link


Copy Markdown

MemberAuthor
CI failures are unrelated to this PR.
@fanyang-mono
fanyang-mono
merged commit 423faed
into
dotnet:mainAug 21, 2024
@matouskozakmatouskozak
mentioned this pull request
Aug 22, 2024

Closed

@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Sep 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@MichalStrehovskyMichalStrehovskyMichalStrehovsky left review comments
@lambdageeklambdageeklambdageek approved these changes
@jkotasjkotasjkotas approved these changes
@steveisoksteveisokAwaiting requested review from steveisoksteveisok is a code owner
@davidwrightondavidwrightonAwaiting requested review from davidwrighton
@VSadovVSadovAwaiting requested review from VSadov
@thaystgthaystgAwaiting requested review from thaystgthaystg is a code owner

Assignees

@fanyang-monofanyang-mono

Labels

area-TypeSystem-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

The type loader has silent integer overflows


4 participants

@fanyang-mono@jkotas@MichalStrehovsky@lambdageek

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Error out when struct size is bigger than int.MaxValue - #104393
Merged
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Aug 21, 2024
Merged
Error out when struct size is bigger than int.MaxValue#104393
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Conversation
@fanyang-mono
@fanyang-monofanyang-mono
commented
Jul 3, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Fixes: #97412
sizeof() in C# returns a signed integer. This PR is to adjust the runtime to match with that.
Added a check for struct size and a test.
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Jul 3, 2024
@jkotas
jkotas
commented
Jul 3, 2024

Copy link


Copy Markdown

Member
Do we need a test?
jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/Resources/Strings.resx
Outdated

jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/classlayoutinfo.cpp
Outdated


This was referenced Jul 3, 2024

Open


Closed

@fanyang-monofanyang-mono
added

area-TypeSystem-coreclr

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Jul 8, 2024
@build-analysisbuild-analysisBot
mentioned this pull request
Jul 9, 2024

Closed

3 tasks
@fanyang-mono
fanyang-mono
commented
Jul 9, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
@jkotas This PR is ready to be reviewed again.
@MichalStrehovsky
MichalStrehovsky
self-requested a review
as a code ownerJuly 11, 2024 15:06
@MichalStrehovsky
MichalStrehovsky
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
I've pushed out a change that makes further progress in the handling within the managed type system, but the test probably still won't compile because BigArray with size of int.MaxValue gets its size bumped past int.MaxValue through alignment after we're done with layout (the final size needs to be divisible by pointer size and int.MaxValue isn't.
CoreCLR is fine with this overflowing but the managed type system isn't. I'm not sure if the overflow is an actual problem but fixing it doesn't look exactly trivial.
jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/methodtablebuilder.cpp
Outdated

jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/tools/Common/TypeSystem/Common/LayoutInt.cs
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
Only by placing <NativeAotIncompatible>true</NativeAotIncomplatible> in the project file. It would be good not to disable all existing ManagedSequential type loader testing just because the new test doesn't work (i.e. put it in a separate project file).
Note that this is the managed type system that is shared with crossgen2. Might want to run crossgen2 outerloop testing to see whether this also needs to be removed from precompilation there.
The managed type system doesn't handle this test because the size of the BigArray struct is actually over 2 GB when boxed (this is also true on JIT-based CoreCLR). If a 2 GB overflow is problematic, is the 2 GB overflow of that one not problematic?
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
BigArray.Instance=newBigArray();longbefore=GC.GetAllocatedBytesForCurrentThread();BigArray.Instance=newBigArray();Console.WriteLine(GC.GetAllocatedBytesForCurrentThread()-before);[StructLayout(LayoutKind.Sequential,Size=int.MaxValue)]structBigArray{publicstaticobjectInstance;}
Will print 2,147,483,664, which is larger than int.MaxValue.
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp list
@azure-pipelines
azure-pipelinesBot
commented
Jul 15, 2024

Copy link


Copy Markdown


CI/CD Pipelines for this repository:
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
We chatted with Fan on Teams about this and apparently the program I posted only works on a Release build (also without debugger attached) and throws InvalidProgram otherwise. On .NET 8 CoreCLR-JIT. Curious.

This was referenced Aug 7, 2024

Closed


Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky CoreCLR and Mono side of work has been completed and approved. The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change. @jkotas Please correct me, if I were wrong.
@MichalStrehovsky
MichalStrehovsky
commented
Aug 7, 2024

Copy link


Copy Markdown

Member
The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
I'll not have time to look at this any time soon. Might be better to just block the test on an issue and not block this PR.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change.
Throwing an exception in a situation that didn't throw before is the definition of a breaking change.
@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
Discussed with @jkotas offline. He suggested to wait for .NET10 to merge it, together with the NativeAOT change, when it is ready.
There is no rush. @MichalStrehovsky
@steveisoksteveisok
mentioned this pull request
Aug 8, 2024

Open

@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
I plan to merge this PR when main is .NET10. The managed type system part used by NativeAOT and crossgen can be worked on later during .NET10 release.
@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 9, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Aug 9, 2024

Open


This was referenced Aug 10, 2024

Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 11, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 11, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 15, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 21, 2024

Copy link


Copy Markdown

MemberAuthor
CI failures are unrelated to this PR.
@fanyang-mono
fanyang-mono
merged commit 423faed
into
dotnet:mainAug 21, 2024
@matouskozakmatouskozak
mentioned this pull request
Aug 22, 2024

Closed

@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Sep 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@MichalStrehovskyMichalStrehovskyMichalStrehovsky left review comments
@lambdageeklambdageeklambdageek approved these changes
@jkotasjkotasjkotas approved these changes
@steveisoksteveisokAwaiting requested review from steveisoksteveisok is a code owner
@davidwrightondavidwrightonAwaiting requested review from davidwrighton
@VSadovVSadovAwaiting requested review from VSadov
@thaystgthaystgAwaiting requested review from thaystgthaystg is a code owner

Assignees

@fanyang-monofanyang-mono

Labels

area-TypeSystem-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

The type loader has silent integer overflows


4 participants

@fanyang-mono@jkotas@MichalStrehovsky@lambdageek

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Error out when struct size is bigger than int.MaxValue - #104393
Merged
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Aug 21, 2024
Merged
Error out when struct size is bigger than int.MaxValue#104393
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Conversation
@fanyang-mono
@fanyang-monofanyang-mono
commented
Jul 3, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Fixes: #97412
sizeof() in C# returns a signed integer. This PR is to adjust the runtime to match with that.
Added a check for struct size and a test.
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Jul 3, 2024
@jkotas
jkotas
commented
Jul 3, 2024

Copy link


Copy Markdown

Member
Do we need a test?
jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/Resources/Strings.resx
Outdated

jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/classlayoutinfo.cpp
Outdated


This was referenced Jul 3, 2024

Open


Closed

@fanyang-monofanyang-mono
added

area-TypeSystem-coreclr

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Jul 8, 2024
@build-analysisbuild-analysisBot
mentioned this pull request
Jul 9, 2024

Closed

3 tasks
@fanyang-mono
fanyang-mono
commented
Jul 9, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
@jkotas This PR is ready to be reviewed again.
@MichalStrehovsky
MichalStrehovsky
self-requested a review
as a code ownerJuly 11, 2024 15:06
@MichalStrehovsky
MichalStrehovsky
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
I've pushed out a change that makes further progress in the handling within the managed type system, but the test probably still won't compile because BigArray with size of int.MaxValue gets its size bumped past int.MaxValue through alignment after we're done with layout (the final size needs to be divisible by pointer size and int.MaxValue isn't.
CoreCLR is fine with this overflowing but the managed type system isn't. I'm not sure if the overflow is an actual problem but fixing it doesn't look exactly trivial.
jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/methodtablebuilder.cpp
Outdated

jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/tools/Common/TypeSystem/Common/LayoutInt.cs
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
Only by placing <NativeAotIncompatible>true</NativeAotIncomplatible> in the project file. It would be good not to disable all existing ManagedSequential type loader testing just because the new test doesn't work (i.e. put it in a separate project file).
Note that this is the managed type system that is shared with crossgen2. Might want to run crossgen2 outerloop testing to see whether this also needs to be removed from precompilation there.
The managed type system doesn't handle this test because the size of the BigArray struct is actually over 2 GB when boxed (this is also true on JIT-based CoreCLR). If a 2 GB overflow is problematic, is the 2 GB overflow of that one not problematic?
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
BigArray.Instance=newBigArray();longbefore=GC.GetAllocatedBytesForCurrentThread();BigArray.Instance=newBigArray();Console.WriteLine(GC.GetAllocatedBytesForCurrentThread()-before);[StructLayout(LayoutKind.Sequential,Size=int.MaxValue)]structBigArray{publicstaticobjectInstance;}
Will print 2,147,483,664, which is larger than int.MaxValue.
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp list
@azure-pipelines
azure-pipelinesBot
commented
Jul 15, 2024

Copy link


Copy Markdown


CI/CD Pipelines for this repository:
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
We chatted with Fan on Teams about this and apparently the program I posted only works on a Release build (also without debugger attached) and throws InvalidProgram otherwise. On .NET 8 CoreCLR-JIT. Curious.

This was referenced Aug 7, 2024

Closed


Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky CoreCLR and Mono side of work has been completed and approved. The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change. @jkotas Please correct me, if I were wrong.
@MichalStrehovsky
MichalStrehovsky
commented
Aug 7, 2024

Copy link


Copy Markdown

Member
The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
I'll not have time to look at this any time soon. Might be better to just block the test on an issue and not block this PR.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change.
Throwing an exception in a situation that didn't throw before is the definition of a breaking change.
@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
Discussed with @jkotas offline. He suggested to wait for .NET10 to merge it, together with the NativeAOT change, when it is ready.
There is no rush. @MichalStrehovsky
@steveisoksteveisok
mentioned this pull request
Aug 8, 2024

Open

@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
I plan to merge this PR when main is .NET10. The managed type system part used by NativeAOT and crossgen can be worked on later during .NET10 release.
@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 9, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Aug 9, 2024

Open


This was referenced Aug 10, 2024

Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 11, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 11, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 15, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 21, 2024

Copy link


Copy Markdown

MemberAuthor
CI failures are unrelated to this PR.
@fanyang-mono
fanyang-mono
merged commit 423faed
into
dotnet:mainAug 21, 2024
@matouskozakmatouskozak
mentioned this pull request
Aug 22, 2024

Closed

@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Sep 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@MichalStrehovskyMichalStrehovskyMichalStrehovsky left review comments
@lambdageeklambdageeklambdageek approved these changes
@jkotasjkotasjkotas approved these changes
@steveisoksteveisokAwaiting requested review from steveisoksteveisok is a code owner
@davidwrightondavidwrightonAwaiting requested review from davidwrighton
@VSadovVSadovAwaiting requested review from VSadov
@thaystgthaystgAwaiting requested review from thaystgthaystg is a code owner

Assignees

@fanyang-monofanyang-mono

Labels

area-TypeSystem-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

The type loader has silent integer overflows


4 participants

@fanyang-mono@jkotas@MichalStrehovsky@lambdageek

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Error out when struct size is bigger than int.MaxValue - #104393
Merged
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Aug 21, 2024
Merged
Error out when struct size is bigger than int.MaxValue#104393
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Conversation
@fanyang-mono
@fanyang-monofanyang-mono
commented
Jul 3, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Fixes: #97412
sizeof() in C# returns a signed integer. This PR is to adjust the runtime to match with that.
Added a check for struct size and a test.
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Jul 3, 2024
@jkotas
jkotas
commented
Jul 3, 2024

Copy link


Copy Markdown

Member
Do we need a test?
jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/Resources/Strings.resx
Outdated

jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/classlayoutinfo.cpp
Outdated


This was referenced Jul 3, 2024

Open


Closed

@fanyang-monofanyang-mono
added

area-TypeSystem-coreclr

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Jul 8, 2024
@build-analysisbuild-analysisBot
mentioned this pull request
Jul 9, 2024

Closed

3 tasks
@fanyang-mono
fanyang-mono
commented
Jul 9, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
@jkotas This PR is ready to be reviewed again.
@MichalStrehovsky
MichalStrehovsky
self-requested a review
as a code ownerJuly 11, 2024 15:06
@MichalStrehovsky
MichalStrehovsky
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
I've pushed out a change that makes further progress in the handling within the managed type system, but the test probably still won't compile because BigArray with size of int.MaxValue gets its size bumped past int.MaxValue through alignment after we're done with layout (the final size needs to be divisible by pointer size and int.MaxValue isn't.
CoreCLR is fine with this overflowing but the managed type system isn't. I'm not sure if the overflow is an actual problem but fixing it doesn't look exactly trivial.
jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/methodtablebuilder.cpp
Outdated

jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/tools/Common/TypeSystem/Common/LayoutInt.cs
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
Only by placing <NativeAotIncompatible>true</NativeAotIncomplatible> in the project file. It would be good not to disable all existing ManagedSequential type loader testing just because the new test doesn't work (i.e. put it in a separate project file).
Note that this is the managed type system that is shared with crossgen2. Might want to run crossgen2 outerloop testing to see whether this also needs to be removed from precompilation there.
The managed type system doesn't handle this test because the size of the BigArray struct is actually over 2 GB when boxed (this is also true on JIT-based CoreCLR). If a 2 GB overflow is problematic, is the 2 GB overflow of that one not problematic?
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
BigArray.Instance=newBigArray();longbefore=GC.GetAllocatedBytesForCurrentThread();BigArray.Instance=newBigArray();Console.WriteLine(GC.GetAllocatedBytesForCurrentThread()-before);[StructLayout(LayoutKind.Sequential,Size=int.MaxValue)]structBigArray{publicstaticobjectInstance;}
Will print 2,147,483,664, which is larger than int.MaxValue.
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp list
@azure-pipelines
azure-pipelinesBot
commented
Jul 15, 2024

Copy link


Copy Markdown


CI/CD Pipelines for this repository:
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
We chatted with Fan on Teams about this and apparently the program I posted only works on a Release build (also without debugger attached) and throws InvalidProgram otherwise. On .NET 8 CoreCLR-JIT. Curious.

This was referenced Aug 7, 2024

Closed


Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky CoreCLR and Mono side of work has been completed and approved. The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change. @jkotas Please correct me, if I were wrong.
@MichalStrehovsky
MichalStrehovsky
commented
Aug 7, 2024

Copy link


Copy Markdown

Member
The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
I'll not have time to look at this any time soon. Might be better to just block the test on an issue and not block this PR.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change.
Throwing an exception in a situation that didn't throw before is the definition of a breaking change.
@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
Discussed with @jkotas offline. He suggested to wait for .NET10 to merge it, together with the NativeAOT change, when it is ready.
There is no rush. @MichalStrehovsky
@steveisoksteveisok
mentioned this pull request
Aug 8, 2024

Open

@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
I plan to merge this PR when main is .NET10. The managed type system part used by NativeAOT and crossgen can be worked on later during .NET10 release.
@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 9, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Aug 9, 2024

Open


This was referenced Aug 10, 2024

Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 11, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 11, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 15, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 21, 2024

Copy link


Copy Markdown

MemberAuthor
CI failures are unrelated to this PR.
@fanyang-mono
fanyang-mono
merged commit 423faed
into
dotnet:mainAug 21, 2024
@matouskozakmatouskozak
mentioned this pull request
Aug 22, 2024

Closed

@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Sep 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@MichalStrehovskyMichalStrehovskyMichalStrehovsky left review comments
@lambdageeklambdageeklambdageek approved these changes
@jkotasjkotasjkotas approved these changes
@steveisoksteveisokAwaiting requested review from steveisoksteveisok is a code owner
@davidwrightondavidwrightonAwaiting requested review from davidwrighton
@VSadovVSadovAwaiting requested review from VSadov
@thaystgthaystgAwaiting requested review from thaystgthaystg is a code owner

Assignees

@fanyang-monofanyang-mono

Labels

area-TypeSystem-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

The type loader has silent integer overflows


4 participants

@fanyang-mono@jkotas@MichalStrehovsky@lambdageek

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Error out when struct size is bigger than int.MaxValue - #104393
Merged
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Aug 21, 2024
Merged
Error out when struct size is bigger than int.MaxValue#104393
fanyang-mono merged 20 commits into
dotnet:mainfrom 
fanyang-mono:fix_overflow
Conversation
@fanyang-mono
@fanyang-monofanyang-mono
commented
Jul 3, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Fixes: #97412
sizeof() in C# returns a signed integer. This PR is to adjust the runtime to match with that.
Added a check for struct size and a test.
@ghostghost
added
the 
needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
label
Jul 3, 2024
@jkotas
jkotas
commented
Jul 3, 2024

Copy link


Copy Markdown

Member
Do we need a test?
jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/libraries/System.Private.CoreLib/src/Resources/Strings.resx
Outdated

jkotas
jkotas
reviewed
Jul 3, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/classlayoutinfo.cpp
Outdated


This was referenced Jul 3, 2024

Open


Closed

@fanyang-monofanyang-mono
added

area-TypeSystem-coreclr

and removed

needs-area-label
An area label is needed to ensure this gets routed to the appropriate area owners
labels
Jul 8, 2024
@build-analysisbuild-analysisBot
mentioned this pull request
Jul 9, 2024

Closed

3 tasks
@fanyang-mono
fanyang-mono
commented
Jul 9, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
@jkotas This PR is ready to be reviewed again.
@MichalStrehovsky
MichalStrehovsky
self-requested a review
as a code ownerJuly 11, 2024 15:06
@MichalStrehovsky
MichalStrehovsky
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
I've pushed out a change that makes further progress in the handling within the managed type system, but the test probably still won't compile because BigArray with size of int.MaxValue gets its size bumped past int.MaxValue through alignment after we're done with layout (the final size needs to be divisible by pointer size and int.MaxValue isn't.
CoreCLR is fine with this overflowing but the managed type system isn't. I'm not sure if the overflow is an actual problem but fixing it doesn't look exactly trivial.
jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/vm/methodtablebuilder.cpp
Outdated

jkotas
jkotas
reviewed
Jul 11, 2024
View reviewed changes
Comment threadsrc/coreclr/tools/Common/TypeSystem/Common/LayoutInt.cs
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Is there a way to stop the new test from being compiled by NativeAOT?
Only by placing <NativeAotIncompatible>true</NativeAotIncomplatible> in the project file. It would be good not to disable all existing ManagedSequential type loader testing just because the new test doesn't work (i.e. put it in a separate project file).
Note that this is the managed type system that is shared with crossgen2. Might want to run crossgen2 outerloop testing to see whether this also needs to be removed from precompilation there.
The managed type system doesn't handle this test because the size of the BigArray struct is actually over 2 GB when boxed (this is also true on JIT-based CoreCLR). If a 2 GB overflow is problematic, is the 2 GB overflow of that one not problematic?
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
@MichalStrehovsky Based on Jan's comment from earlier. Loading BigArray itself should continue to work. Why do we need to align BigArray to pointer size? If that's needed, sizeof(BigArray) would return a bigger value than int.MaxValue. Is that correct?
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
BigArray.Instance=newBigArray();longbefore=GC.GetAllocatedBytesForCurrentThread();BigArray.Instance=newBigArray();Console.WriteLine(GC.GetAllocatedBytesForCurrentThread()-before);[StructLayout(LayoutKind.Sequential,Size=int.MaxValue)]structBigArray{publicstaticobjectInstance;}
Will print 2,147,483,664, which is larger than int.MaxValue.
@fanyang-mono
fanyang-mono
commented
Jul 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp list
@azure-pipelines
azure-pipelinesBot
commented
Jul 15, 2024

Copy link


Copy Markdown


CI/CD Pipelines for this repository:
@MichalStrehovsky
MichalStrehovsky
commented
Jul 15, 2024

Copy link


Copy Markdown

Member
The size of it gets aligned when boxing it, because it's 8 bytes larger (MethodTable pointer) and aligned. On CoreCLR today, doing:
We chatted with Fan on Teams about this and apparently the program I posted only works on a Release build (also without debugger attached) and throws InvalidProgram otherwise. On .NET 8 CoreCLR-JIT. Curious.

This was referenced Aug 7, 2024

Closed


Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
@MichalStrehovsky CoreCLR and Mono side of work has been completed and approved. The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change. @jkotas Please correct me, if I were wrong.
@MichalStrehovsky
MichalStrehovsky
commented
Aug 7, 2024

Copy link


Copy Markdown

Member
The remaining piece of work is related to NativeAOT/Crossgen2. I am going to transfer the ownership of this PR to you, since you will be working on that.
I'll not have time to look at this any time soon. Might be better to just block the test on an issue and not block this PR.
Additionally, this could probably be merged for .NET9, as I don't think that this is a breaking change.
Throwing an exception in a situation that didn't throw before is the definition of a breaking change.
@fanyang-mono
fanyang-mono
commented
Aug 7, 2024

Copy link


Copy Markdown

MemberAuthor
Discussed with @jkotas offline. He suggested to wait for .NET10 to merge it, together with the NativeAOT change, when it is ready.
There is no rush. @MichalStrehovsky
@steveisoksteveisok
mentioned this pull request
Aug 8, 2024

Open

@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
I plan to merge this PR when main is .NET10. The managed type system part used by NativeAOT and crossgen can be worked on later during .NET10 release.
@fanyang-mono
fanyang-mono
commented
Aug 9, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 9, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@build-analysisbuild-analysisBot
mentioned this pull request
Aug 9, 2024

Open


This was referenced Aug 10, 2024

Closed


Closed

@fanyang-mono
fanyang-mono
commented
Aug 11, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 11, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 15, 2024

Copy link


Copy Markdown

MemberAuthor
/azp run runtime-coreclr crossgen2
@azure-pipelines
azure-pipelinesBot
commented
Aug 15, 2024

Copy link


Copy Markdown


Azure Pipelines successfully started running 1 pipeline(s).
@fanyang-mono
fanyang-mono
commented
Aug 21, 2024

Copy link


Copy Markdown

MemberAuthor
CI failures are unrelated to this PR.
@fanyang-mono
fanyang-mono
merged commit 423faed
into
dotnet:mainAug 21, 2024
@matouskozakmatouskozak
mentioned this pull request
Aug 22, 2024

Closed

@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Sep 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@MichalStrehovskyMichalStrehovskyMichalStrehovsky left review comments
@lambdageeklambdageeklambdageek approved these changes
@jkotasjkotasjkotas approved these changes
@steveisoksteveisokAwaiting requested review from steveisoksteveisok is a code owner
@davidwrightondavidwrightonAwaiting requested review from davidwrighton
@VSadovVSadovAwaiting requested review from VSadov
@thaystgthaystgAwaiting requested review from thaystgthaystg is a code owner

Assignees

@fanyang-monofanyang-mono

Labels

area-TypeSystem-coreclr

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

The type loader has silent integer overflows


4 participants

@fanyang-mono@jkotas@MichalStrehovsky@lambdageek