Fix System.Decimal aligment on ARM32 with crossgen2 - #38390

Closed
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master
Closed

Fix System.Decimal aligment on ARM32 with crossgen2#38390
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master

Conversation

@pavel-orekhov

Copy link
Copy Markdown
  • Fix passing Decimal to funcs. Fixes tests/.../decimal.cs and half of tests/.../10w5d.cs

@dnfadmin

dnfadmin commented Jun 25, 2020

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

@MichalStrehovsky

Copy link
Copy Markdown
Member

This should be done similar to how other special types handled:

  • Make a new layout algorithm class deriving from FieldLayoutAlgorithm that does layout for System.Decimal specifically
  • Add it to the if block here:

if(type.IsObject)
return_systemObjectFieldLayoutAlgorithm;
elseif(type==UniversalCanonType)
thrownewNotImplementedException();
elseif(type.IsRuntimeDeterminedType)
thrownewNotImplementedException();
elseif(VectorOfTFieldLayoutAlgorithm.IsVectorOfTType(type))
{
return_vectorOfTFieldLayoutAlgorithm;
}
elseif(VectorFieldLayoutAlgorithm.IsVectorType(type))
{
return_vectorFieldLayoutAlgorithm;
}
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm!=null);
return_r2rFieldLayoutAlgorithm;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The helpers in InteropTypes.cs are specifically for interop marshallers. We don't have a central location with a "is this type X?" functionality. We place the functionality locally where it's needed.

@pavel-orekhovpavel-orekhovJun 25, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks to you!
Ok. Moving modules to another assembly is not very good. So, is it right you suggests to copy-paste InteropTypes::IsSystemDecimal and IsCoreNamedType to R2RCompilerContext.FieldLayoutAlgorithm() / _r2rFieldLayoutAlgorithm and compare strings like

diff --git a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
index bbe1274cae3..d3210f2c5c7 100644
--- a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
+++ b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
@@ -60,6 +60,10 @@ public override FieldLayoutAlgorithm GetLayoutAlgorithmForType(DefType type)
{
return _vectorFieldLayoutAlgorithm;
}
+ else if (type.IsValueType && type.Context.Target.Architecture == TargetArchitecture.ARM && IsSystemDecimal(type.Context, type))
+ {
+ return _decimalOnARM32FieldLayoutAlgorithm;
+ }
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm != null);
return _r2rFieldLayoutAlgorithm;
}

?

Or i can use InteropTypes.IsSystemDecimal() directly here?

May be you suggests to create the flag DefType.isSystemDecimal in order to compare bits, not strings? But I don't know where to set this flag, where the Decimal class loads. :( Do you know it? Can you tell me that?

@MichalStrehovsky

Copy link
Copy Markdown
Member

cc @dotnet/crossgen-contrib

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easiest to check for CoreLib types by name in a method called from here, and apply the layout quirks that way.

Similar to how MethodTableBuilder::CheckForSystemTypes is done. I expect we will need most quirks from this method eventually.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at MethodTableBuilder::CheckForSystemTypes - will we need anything else besides the Decimal addition? We already treat Vector<T> and the other vectors the way I propose for Decimal as well. I don't particularly care about the mechanism, but I do care about consistency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Pro: Checking type in ReadyToRunCompilerContext allows to don't move InteropTypes to ILC.TypeSystem.R2R and allows to use existing InteropTypes.IsDecimal

  2. Pro: Checking type in ReadyToRunCompilerContext removes checking of platform via ReadyToRunCompilerContext constructor

  3. Contra: Checking type in ReadyToRunCompilerContext requires to write many code to implement other methods of FieldLayoutAlgorithm

  4. Pro: checking type in MetadataFieldLayoutAlgorithm.cs exists already and is almost one-liner

  5. Contra: copy-paste of isDecimal required or moving of InteropTypes to ILC.TypeSystem.R2R

  6. Pro: Moving is already done

  7. Contra: More changes in a branch for static struct will be to mitigate second half of the 10w5d.cs test.

  8. Contra: Colleagues say, more types with broken alignment on ARM32 may arise.

So, Where? Please tell me the consensus!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also faced with similar issues in Nullable types. If there are already exceptions for vectors and other types in ReadyToRunCompilerContext I think it would be preferred place.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the problem with Nullable types? Nullable types should not have any layout quirks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a nullable long enum isn't handled correctly, that is a bug in the general purpose type layout logic, not something that should be special cased.

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LongE & so seem are fixed by #32663 / #32664.
But I afraid it is only masking of bug:
When (before) jit/methodical/fp/exgen/10w5d.csFunc_* don't compiled by cg2 but delayed to jit while ctors is done by cg2. So some difference in placement of static structs was (IMHO) between cg2 & jit compilers was and leaded to missfunction.
Right now placement of statics and using of it is by cg2 and you can't see difference with jit's ideas.

So bug (10w5d, LongE and so) may resurrect when cg2 and jit will be used. For example in eval case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Decimal, I am wondering whether we should fix the Decimal implementation to make this special casing unncessary. I will take a look at what it would take today.

LongE & so seem are fixed by #32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

With 4e5c8c0 (today's (20200629) master) the jit/methodical/fp/exgen/10w5d.cs is ok.

How to force 'delayed compilation' back I don't know (eval does not exists, imho). Sorry.
If you mean mass testing clr tests with cg2 please look to our night test results:
results.tar.gz

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#38603 should be a cleaner fix for the Decimal issue.

@pavel-orekhov

Copy link
Copy Markdown
Author

cc @alpencolt

* Fix passing Decimal to funcs. It fixes the JIT/Methodical/MDArray/DataTypes/decimal.cs
@pavel-orekhov

Copy link
Copy Markdown
Author

def2b07 is @jkotas's style rewriting ;)

jkotas added a commit to jkotas/runtime that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artifically inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixesdotnet#38390
jkotas added a commit that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artificially inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixes#38390
@jkotas

Copy link
Copy Markdown
Member

@pavel-orekhov I have fixed decimal implementation to not require this quirk. Give it a try

@t-mustafin

Copy link
Copy Markdown
Contributor

I hit on different input argument r1 usage generated by crossgen2 and Tier-1 compilation on lambda-expression containing decimal variable:

varsoldOutProducts=products.Where(p =>p.UnitsInStock>0&&p.UnitPrice>60.00M);

from Linq test.
Code by crossgen2 loads input decimal from r1+20:

00001A 6948 ldr r0,[r1+20]00001C 9004str r0,[sp+0x10]00001E 6988 ldr r0,[r1+24]0000209005str r0,[sp+0x14]000022 69C8 ldr r0,[r1+28]0000249006str r0,[sp+0x18]000026 6A08 ldr r0,[r1+32]0000289007str r0,[sp+0x1c]

meanwhile tier-1 compiled code loads input decimal from r1+24:

00001A F101 0018add r0, r1,2400001E 6804 ldr r4,[r0]0000206845 ldr r5,[r0+4]0000223008 adds r0,80000246801 ldr r1,[r0]0000266840 ldr r0,[r0+4]0000289108str r1,[sp+0x20]00002A 9009str r0,[sp+0x24]

That incompatibility leads to test fail after tier compilation of that lambda-expression. Test with turned off tier compilation finish successfully.
Linq.cg2.dump.txt
Linq.cg2.S.txt
Linq.cg2-tier.dump.txt
Linq.cg2-tier.S.txt

@jkotas

Copy link
Copy Markdown
Member

This bug should not be specific to Decimal. Are you able to reproduce it with a custom type with long field? E.g. struct MyStruct { int a; int b; long c; }.

Looks like getClassAlignmentRequirement in crossgen2 may be returning wrong value on arm.

@t-mustafin

Copy link
Copy Markdown
Contributor

Yes, it is reproduced with MyStruct { int a; int b; long c; }. Lambda-expression:

varsoldOutProducts=products.Where(p =>p.MyStruct.a>0x300||p.MyStruct.b>0x50000||p.MyStruct.c>0x6000000);

Offsets generated by crossgen2:

000006 6A48 ldr r0,[r1+36]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6A88 ldr r0,[r1+40]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 002C add r0, r1,4400001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

Offsets generated by Tier-1:

G_M16079_IG02:000006 6A88 ldr r0,[r1+40]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6AC8 ldr r0,[r1+44]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 0030add r0, r1,4800001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

@mangod9

Copy link
Copy Markdown
Member

@t-mustafin since this particular issues is closed, could you please create a new issue for this ? Thx.

@ghostghost locked as resolved and limited conversation to collaborators Dec 8, 2020
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants

@pavel-orekhov@dnfadmin@MichalStrehovsky@jkotas@t-mustafin@mangod9@alpencolt@davidwrighton@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Fix System.Decimal aligment on ARM32 with crossgen2 - #38390

Closed
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master
Closed

Fix System.Decimal aligment on ARM32 with crossgen2#38390
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master

Conversation

@pavel-orekhov

Copy link
Copy Markdown
  • Fix passing Decimal to funcs. Fixes tests/.../decimal.cs and half of tests/.../10w5d.cs

@dnfadmin

dnfadmin commented Jun 25, 2020

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

@MichalStrehovsky

Copy link
Copy Markdown
Member

This should be done similar to how other special types handled:

  • Make a new layout algorithm class deriving from FieldLayoutAlgorithm that does layout for System.Decimal specifically
  • Add it to the if block here:

if(type.IsObject)
return_systemObjectFieldLayoutAlgorithm;
elseif(type==UniversalCanonType)
thrownewNotImplementedException();
elseif(type.IsRuntimeDeterminedType)
thrownewNotImplementedException();
elseif(VectorOfTFieldLayoutAlgorithm.IsVectorOfTType(type))
{
return_vectorOfTFieldLayoutAlgorithm;
}
elseif(VectorFieldLayoutAlgorithm.IsVectorType(type))
{
return_vectorFieldLayoutAlgorithm;
}
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm!=null);
return_r2rFieldLayoutAlgorithm;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The helpers in InteropTypes.cs are specifically for interop marshallers. We don't have a central location with a "is this type X?" functionality. We place the functionality locally where it's needed.

@pavel-orekhovpavel-orekhovJun 25, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks to you!
Ok. Moving modules to another assembly is not very good. So, is it right you suggests to copy-paste InteropTypes::IsSystemDecimal and IsCoreNamedType to R2RCompilerContext.FieldLayoutAlgorithm() / _r2rFieldLayoutAlgorithm and compare strings like

diff --git a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
index bbe1274cae3..d3210f2c5c7 100644
--- a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
+++ b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
@@ -60,6 +60,10 @@ public override FieldLayoutAlgorithm GetLayoutAlgorithmForType(DefType type)
{
return _vectorFieldLayoutAlgorithm;
}
+ else if (type.IsValueType && type.Context.Target.Architecture == TargetArchitecture.ARM && IsSystemDecimal(type.Context, type))
+ {
+ return _decimalOnARM32FieldLayoutAlgorithm;
+ }
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm != null);
return _r2rFieldLayoutAlgorithm;
}

?

Or i can use InteropTypes.IsSystemDecimal() directly here?

May be you suggests to create the flag DefType.isSystemDecimal in order to compare bits, not strings? But I don't know where to set this flag, where the Decimal class loads. :( Do you know it? Can you tell me that?

@MichalStrehovsky

Copy link
Copy Markdown
Member

cc @dotnet/crossgen-contrib

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easiest to check for CoreLib types by name in a method called from here, and apply the layout quirks that way.

Similar to how MethodTableBuilder::CheckForSystemTypes is done. I expect we will need most quirks from this method eventually.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at MethodTableBuilder::CheckForSystemTypes - will we need anything else besides the Decimal addition? We already treat Vector<T> and the other vectors the way I propose for Decimal as well. I don't particularly care about the mechanism, but I do care about consistency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Pro: Checking type in ReadyToRunCompilerContext allows to don't move InteropTypes to ILC.TypeSystem.R2R and allows to use existing InteropTypes.IsDecimal

  2. Pro: Checking type in ReadyToRunCompilerContext removes checking of platform via ReadyToRunCompilerContext constructor

  3. Contra: Checking type in ReadyToRunCompilerContext requires to write many code to implement other methods of FieldLayoutAlgorithm

  4. Pro: checking type in MetadataFieldLayoutAlgorithm.cs exists already and is almost one-liner

  5. Contra: copy-paste of isDecimal required or moving of InteropTypes to ILC.TypeSystem.R2R

  6. Pro: Moving is already done

  7. Contra: More changes in a branch for static struct will be to mitigate second half of the 10w5d.cs test.

  8. Contra: Colleagues say, more types with broken alignment on ARM32 may arise.

So, Where? Please tell me the consensus!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also faced with similar issues in Nullable types. If there are already exceptions for vectors and other types in ReadyToRunCompilerContext I think it would be preferred place.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the problem with Nullable types? Nullable types should not have any layout quirks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a nullable long enum isn't handled correctly, that is a bug in the general purpose type layout logic, not something that should be special cased.

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LongE & so seem are fixed by #32663 / #32664.
But I afraid it is only masking of bug:
When (before) jit/methodical/fp/exgen/10w5d.csFunc_* don't compiled by cg2 but delayed to jit while ctors is done by cg2. So some difference in placement of static structs was (IMHO) between cg2 & jit compilers was and leaded to missfunction.
Right now placement of statics and using of it is by cg2 and you can't see difference with jit's ideas.

So bug (10w5d, LongE and so) may resurrect when cg2 and jit will be used. For example in eval case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Decimal, I am wondering whether we should fix the Decimal implementation to make this special casing unncessary. I will take a look at what it would take today.

LongE & so seem are fixed by #32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

With 4e5c8c0 (today's (20200629) master) the jit/methodical/fp/exgen/10w5d.cs is ok.

How to force 'delayed compilation' back I don't know (eval does not exists, imho). Sorry.
If you mean mass testing clr tests with cg2 please look to our night test results:
results.tar.gz

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#38603 should be a cleaner fix for the Decimal issue.

@pavel-orekhov

Copy link
Copy Markdown
Author

cc @alpencolt

* Fix passing Decimal to funcs. It fixes the JIT/Methodical/MDArray/DataTypes/decimal.cs
@pavel-orekhov

Copy link
Copy Markdown
Author

def2b07 is @jkotas's style rewriting ;)

jkotas added a commit to jkotas/runtime that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artifically inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixesdotnet#38390
jkotas added a commit that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artificially inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixes#38390
@jkotas

Copy link
Copy Markdown
Member

@pavel-orekhov I have fixed decimal implementation to not require this quirk. Give it a try

@t-mustafin

Copy link
Copy Markdown
Contributor

I hit on different input argument r1 usage generated by crossgen2 and Tier-1 compilation on lambda-expression containing decimal variable:

varsoldOutProducts=products.Where(p =>p.UnitsInStock>0&&p.UnitPrice>60.00M);

from Linq test.
Code by crossgen2 loads input decimal from r1+20:

00001A 6948 ldr r0,[r1+20]00001C 9004str r0,[sp+0x10]00001E 6988 ldr r0,[r1+24]0000209005str r0,[sp+0x14]000022 69C8 ldr r0,[r1+28]0000249006str r0,[sp+0x18]000026 6A08 ldr r0,[r1+32]0000289007str r0,[sp+0x1c]

meanwhile tier-1 compiled code loads input decimal from r1+24:

00001A F101 0018add r0, r1,2400001E 6804 ldr r4,[r0]0000206845 ldr r5,[r0+4]0000223008 adds r0,80000246801 ldr r1,[r0]0000266840 ldr r0,[r0+4]0000289108str r1,[sp+0x20]00002A 9009str r0,[sp+0x24]

That incompatibility leads to test fail after tier compilation of that lambda-expression. Test with turned off tier compilation finish successfully.
Linq.cg2.dump.txt
Linq.cg2.S.txt
Linq.cg2-tier.dump.txt
Linq.cg2-tier.S.txt

@jkotas

Copy link
Copy Markdown
Member

This bug should not be specific to Decimal. Are you able to reproduce it with a custom type with long field? E.g. struct MyStruct { int a; int b; long c; }.

Looks like getClassAlignmentRequirement in crossgen2 may be returning wrong value on arm.

@t-mustafin

Copy link
Copy Markdown
Contributor

Yes, it is reproduced with MyStruct { int a; int b; long c; }. Lambda-expression:

varsoldOutProducts=products.Where(p =>p.MyStruct.a>0x300||p.MyStruct.b>0x50000||p.MyStruct.c>0x6000000);

Offsets generated by crossgen2:

000006 6A48 ldr r0,[r1+36]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6A88 ldr r0,[r1+40]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 002C add r0, r1,4400001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

Offsets generated by Tier-1:

G_M16079_IG02:000006 6A88 ldr r0,[r1+40]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6AC8 ldr r0,[r1+44]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 0030add r0, r1,4800001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

@mangod9

Copy link
Copy Markdown
Member

@t-mustafin since this particular issues is closed, could you please create a new issue for this ? Thx.

@ghostghost locked as resolved and limited conversation to collaborators Dec 8, 2020
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants

@pavel-orekhov@dnfadmin@MichalStrehovsky@jkotas@t-mustafin@mangod9@alpencolt@davidwrighton@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix System.Decimal aligment on ARM32 with crossgen2 - #38390

Closed
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master
Closed

Fix System.Decimal aligment on ARM32 with crossgen2#38390
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master

Conversation

@pavel-orekhov

Copy link
Copy Markdown
  • Fix passing Decimal to funcs. Fixes tests/.../decimal.cs and half of tests/.../10w5d.cs

@dnfadmin

dnfadmin commented Jun 25, 2020

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

@MichalStrehovsky

Copy link
Copy Markdown
Member

This should be done similar to how other special types handled:

  • Make a new layout algorithm class deriving from FieldLayoutAlgorithm that does layout for System.Decimal specifically
  • Add it to the if block here:

if(type.IsObject)
return_systemObjectFieldLayoutAlgorithm;
elseif(type==UniversalCanonType)
thrownewNotImplementedException();
elseif(type.IsRuntimeDeterminedType)
thrownewNotImplementedException();
elseif(VectorOfTFieldLayoutAlgorithm.IsVectorOfTType(type))
{
return_vectorOfTFieldLayoutAlgorithm;
}
elseif(VectorFieldLayoutAlgorithm.IsVectorType(type))
{
return_vectorFieldLayoutAlgorithm;
}
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm!=null);
return_r2rFieldLayoutAlgorithm;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The helpers in InteropTypes.cs are specifically for interop marshallers. We don't have a central location with a "is this type X?" functionality. We place the functionality locally where it's needed.

@pavel-orekhovpavel-orekhovJun 25, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks to you!
Ok. Moving modules to another assembly is not very good. So, is it right you suggests to copy-paste InteropTypes::IsSystemDecimal and IsCoreNamedType to R2RCompilerContext.FieldLayoutAlgorithm() / _r2rFieldLayoutAlgorithm and compare strings like

diff --git a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
index bbe1274cae3..d3210f2c5c7 100644
--- a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
+++ b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
@@ -60,6 +60,10 @@ public override FieldLayoutAlgorithm GetLayoutAlgorithmForType(DefType type)
{
return _vectorFieldLayoutAlgorithm;
}
+ else if (type.IsValueType && type.Context.Target.Architecture == TargetArchitecture.ARM && IsSystemDecimal(type.Context, type))
+ {
+ return _decimalOnARM32FieldLayoutAlgorithm;
+ }
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm != null);
return _r2rFieldLayoutAlgorithm;
}

?

Or i can use InteropTypes.IsSystemDecimal() directly here?

May be you suggests to create the flag DefType.isSystemDecimal in order to compare bits, not strings? But I don't know where to set this flag, where the Decimal class loads. :( Do you know it? Can you tell me that?

@MichalStrehovsky

Copy link
Copy Markdown
Member

cc @dotnet/crossgen-contrib

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easiest to check for CoreLib types by name in a method called from here, and apply the layout quirks that way.

Similar to how MethodTableBuilder::CheckForSystemTypes is done. I expect we will need most quirks from this method eventually.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at MethodTableBuilder::CheckForSystemTypes - will we need anything else besides the Decimal addition? We already treat Vector<T> and the other vectors the way I propose for Decimal as well. I don't particularly care about the mechanism, but I do care about consistency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Pro: Checking type in ReadyToRunCompilerContext allows to don't move InteropTypes to ILC.TypeSystem.R2R and allows to use existing InteropTypes.IsDecimal

  2. Pro: Checking type in ReadyToRunCompilerContext removes checking of platform via ReadyToRunCompilerContext constructor

  3. Contra: Checking type in ReadyToRunCompilerContext requires to write many code to implement other methods of FieldLayoutAlgorithm

  4. Pro: checking type in MetadataFieldLayoutAlgorithm.cs exists already and is almost one-liner

  5. Contra: copy-paste of isDecimal required or moving of InteropTypes to ILC.TypeSystem.R2R

  6. Pro: Moving is already done

  7. Contra: More changes in a branch for static struct will be to mitigate second half of the 10w5d.cs test.

  8. Contra: Colleagues say, more types with broken alignment on ARM32 may arise.

So, Where? Please tell me the consensus!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also faced with similar issues in Nullable types. If there are already exceptions for vectors and other types in ReadyToRunCompilerContext I think it would be preferred place.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the problem with Nullable types? Nullable types should not have any layout quirks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a nullable long enum isn't handled correctly, that is a bug in the general purpose type layout logic, not something that should be special cased.

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LongE & so seem are fixed by #32663 / #32664.
But I afraid it is only masking of bug:
When (before) jit/methodical/fp/exgen/10w5d.csFunc_* don't compiled by cg2 but delayed to jit while ctors is done by cg2. So some difference in placement of static structs was (IMHO) between cg2 & jit compilers was and leaded to missfunction.
Right now placement of statics and using of it is by cg2 and you can't see difference with jit's ideas.

So bug (10w5d, LongE and so) may resurrect when cg2 and jit will be used. For example in eval case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Decimal, I am wondering whether we should fix the Decimal implementation to make this special casing unncessary. I will take a look at what it would take today.

LongE & so seem are fixed by #32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

With 4e5c8c0 (today's (20200629) master) the jit/methodical/fp/exgen/10w5d.cs is ok.

How to force 'delayed compilation' back I don't know (eval does not exists, imho). Sorry.
If you mean mass testing clr tests with cg2 please look to our night test results:
results.tar.gz

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#38603 should be a cleaner fix for the Decimal issue.

@pavel-orekhov

Copy link
Copy Markdown
Author

cc @alpencolt

* Fix passing Decimal to funcs. It fixes the JIT/Methodical/MDArray/DataTypes/decimal.cs
@pavel-orekhov

Copy link
Copy Markdown
Author

def2b07 is @jkotas's style rewriting ;)

jkotas added a commit to jkotas/runtime that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artifically inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixesdotnet#38390
jkotas added a commit that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artificially inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixes#38390
@jkotas

Copy link
Copy Markdown
Member

@pavel-orekhov I have fixed decimal implementation to not require this quirk. Give it a try

@t-mustafin

Copy link
Copy Markdown
Contributor

I hit on different input argument r1 usage generated by crossgen2 and Tier-1 compilation on lambda-expression containing decimal variable:

varsoldOutProducts=products.Where(p =>p.UnitsInStock>0&&p.UnitPrice>60.00M);

from Linq test.
Code by crossgen2 loads input decimal from r1+20:

00001A 6948 ldr r0,[r1+20]00001C 9004str r0,[sp+0x10]00001E 6988 ldr r0,[r1+24]0000209005str r0,[sp+0x14]000022 69C8 ldr r0,[r1+28]0000249006str r0,[sp+0x18]000026 6A08 ldr r0,[r1+32]0000289007str r0,[sp+0x1c]

meanwhile tier-1 compiled code loads input decimal from r1+24:

00001A F101 0018add r0, r1,2400001E 6804 ldr r4,[r0]0000206845 ldr r5,[r0+4]0000223008 adds r0,80000246801 ldr r1,[r0]0000266840 ldr r0,[r0+4]0000289108str r1,[sp+0x20]00002A 9009str r0,[sp+0x24]

That incompatibility leads to test fail after tier compilation of that lambda-expression. Test with turned off tier compilation finish successfully.
Linq.cg2.dump.txt
Linq.cg2.S.txt
Linq.cg2-tier.dump.txt
Linq.cg2-tier.S.txt

@jkotas

Copy link
Copy Markdown
Member

This bug should not be specific to Decimal. Are you able to reproduce it with a custom type with long field? E.g. struct MyStruct { int a; int b; long c; }.

Looks like getClassAlignmentRequirement in crossgen2 may be returning wrong value on arm.

@t-mustafin

Copy link
Copy Markdown
Contributor

Yes, it is reproduced with MyStruct { int a; int b; long c; }. Lambda-expression:

varsoldOutProducts=products.Where(p =>p.MyStruct.a>0x300||p.MyStruct.b>0x50000||p.MyStruct.c>0x6000000);

Offsets generated by crossgen2:

000006 6A48 ldr r0,[r1+36]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6A88 ldr r0,[r1+40]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 002C add r0, r1,4400001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

Offsets generated by Tier-1:

G_M16079_IG02:000006 6A88 ldr r0,[r1+40]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6AC8 ldr r0,[r1+44]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 0030add r0, r1,4800001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

@mangod9

Copy link
Copy Markdown
Member

@t-mustafin since this particular issues is closed, could you please create a new issue for this ? Thx.

@ghostghost locked as resolved and limited conversation to collaborators Dec 8, 2020
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants

@pavel-orekhov@dnfadmin@MichalStrehovsky@jkotas@t-mustafin@mangod9@alpencolt@davidwrighton@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix System.Decimal aligment on ARM32 with crossgen2 - #38390

Closed
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master
Closed

Fix System.Decimal aligment on ARM32 with crossgen2#38390
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master

Conversation

@pavel-orekhov

Copy link
Copy Markdown
  • Fix passing Decimal to funcs. Fixes tests/.../decimal.cs and half of tests/.../10w5d.cs

@dnfadmin

dnfadmin commented Jun 25, 2020

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

@MichalStrehovsky

Copy link
Copy Markdown
Member

This should be done similar to how other special types handled:

  • Make a new layout algorithm class deriving from FieldLayoutAlgorithm that does layout for System.Decimal specifically
  • Add it to the if block here:

if(type.IsObject)
return_systemObjectFieldLayoutAlgorithm;
elseif(type==UniversalCanonType)
thrownewNotImplementedException();
elseif(type.IsRuntimeDeterminedType)
thrownewNotImplementedException();
elseif(VectorOfTFieldLayoutAlgorithm.IsVectorOfTType(type))
{
return_vectorOfTFieldLayoutAlgorithm;
}
elseif(VectorFieldLayoutAlgorithm.IsVectorType(type))
{
return_vectorFieldLayoutAlgorithm;
}
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm!=null);
return_r2rFieldLayoutAlgorithm;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The helpers in InteropTypes.cs are specifically for interop marshallers. We don't have a central location with a "is this type X?" functionality. We place the functionality locally where it's needed.

@pavel-orekhovpavel-orekhovJun 25, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks to you!
Ok. Moving modules to another assembly is not very good. So, is it right you suggests to copy-paste InteropTypes::IsSystemDecimal and IsCoreNamedType to R2RCompilerContext.FieldLayoutAlgorithm() / _r2rFieldLayoutAlgorithm and compare strings like

diff --git a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
index bbe1274cae3..d3210f2c5c7 100644
--- a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
+++ b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
@@ -60,6 +60,10 @@ public override FieldLayoutAlgorithm GetLayoutAlgorithmForType(DefType type)
{
return _vectorFieldLayoutAlgorithm;
}
+ else if (type.IsValueType && type.Context.Target.Architecture == TargetArchitecture.ARM && IsSystemDecimal(type.Context, type))
+ {
+ return _decimalOnARM32FieldLayoutAlgorithm;
+ }
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm != null);
return _r2rFieldLayoutAlgorithm;
}

?

Or i can use InteropTypes.IsSystemDecimal() directly here?

May be you suggests to create the flag DefType.isSystemDecimal in order to compare bits, not strings? But I don't know where to set this flag, where the Decimal class loads. :( Do you know it? Can you tell me that?

@MichalStrehovsky

Copy link
Copy Markdown
Member

cc @dotnet/crossgen-contrib

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easiest to check for CoreLib types by name in a method called from here, and apply the layout quirks that way.

Similar to how MethodTableBuilder::CheckForSystemTypes is done. I expect we will need most quirks from this method eventually.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at MethodTableBuilder::CheckForSystemTypes - will we need anything else besides the Decimal addition? We already treat Vector<T> and the other vectors the way I propose for Decimal as well. I don't particularly care about the mechanism, but I do care about consistency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Pro: Checking type in ReadyToRunCompilerContext allows to don't move InteropTypes to ILC.TypeSystem.R2R and allows to use existing InteropTypes.IsDecimal

  2. Pro: Checking type in ReadyToRunCompilerContext removes checking of platform via ReadyToRunCompilerContext constructor

  3. Contra: Checking type in ReadyToRunCompilerContext requires to write many code to implement other methods of FieldLayoutAlgorithm

  4. Pro: checking type in MetadataFieldLayoutAlgorithm.cs exists already and is almost one-liner

  5. Contra: copy-paste of isDecimal required or moving of InteropTypes to ILC.TypeSystem.R2R

  6. Pro: Moving is already done

  7. Contra: More changes in a branch for static struct will be to mitigate second half of the 10w5d.cs test.

  8. Contra: Colleagues say, more types with broken alignment on ARM32 may arise.

So, Where? Please tell me the consensus!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also faced with similar issues in Nullable types. If there are already exceptions for vectors and other types in ReadyToRunCompilerContext I think it would be preferred place.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the problem with Nullable types? Nullable types should not have any layout quirks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a nullable long enum isn't handled correctly, that is a bug in the general purpose type layout logic, not something that should be special cased.

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LongE & so seem are fixed by #32663 / #32664.
But I afraid it is only masking of bug:
When (before) jit/methodical/fp/exgen/10w5d.csFunc_* don't compiled by cg2 but delayed to jit while ctors is done by cg2. So some difference in placement of static structs was (IMHO) between cg2 & jit compilers was and leaded to missfunction.
Right now placement of statics and using of it is by cg2 and you can't see difference with jit's ideas.

So bug (10w5d, LongE and so) may resurrect when cg2 and jit will be used. For example in eval case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Decimal, I am wondering whether we should fix the Decimal implementation to make this special casing unncessary. I will take a look at what it would take today.

LongE & so seem are fixed by #32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

With 4e5c8c0 (today's (20200629) master) the jit/methodical/fp/exgen/10w5d.cs is ok.

How to force 'delayed compilation' back I don't know (eval does not exists, imho). Sorry.
If you mean mass testing clr tests with cg2 please look to our night test results:
results.tar.gz

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#38603 should be a cleaner fix for the Decimal issue.

@pavel-orekhov

Copy link
Copy Markdown
Author

cc @alpencolt

* Fix passing Decimal to funcs. It fixes the JIT/Methodical/MDArray/DataTypes/decimal.cs
@pavel-orekhov

Copy link
Copy Markdown
Author

def2b07 is @jkotas's style rewriting ;)

jkotas added a commit to jkotas/runtime that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artifically inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixesdotnet#38390
jkotas added a commit that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artificially inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixes#38390
@jkotas

Copy link
Copy Markdown
Member

@pavel-orekhov I have fixed decimal implementation to not require this quirk. Give it a try

@t-mustafin

Copy link
Copy Markdown
Contributor

I hit on different input argument r1 usage generated by crossgen2 and Tier-1 compilation on lambda-expression containing decimal variable:

varsoldOutProducts=products.Where(p =>p.UnitsInStock>0&&p.UnitPrice>60.00M);

from Linq test.
Code by crossgen2 loads input decimal from r1+20:

00001A 6948 ldr r0,[r1+20]00001C 9004str r0,[sp+0x10]00001E 6988 ldr r0,[r1+24]0000209005str r0,[sp+0x14]000022 69C8 ldr r0,[r1+28]0000249006str r0,[sp+0x18]000026 6A08 ldr r0,[r1+32]0000289007str r0,[sp+0x1c]

meanwhile tier-1 compiled code loads input decimal from r1+24:

00001A F101 0018add r0, r1,2400001E 6804 ldr r4,[r0]0000206845 ldr r5,[r0+4]0000223008 adds r0,80000246801 ldr r1,[r0]0000266840 ldr r0,[r0+4]0000289108str r1,[sp+0x20]00002A 9009str r0,[sp+0x24]

That incompatibility leads to test fail after tier compilation of that lambda-expression. Test with turned off tier compilation finish successfully.
Linq.cg2.dump.txt
Linq.cg2.S.txt
Linq.cg2-tier.dump.txt
Linq.cg2-tier.S.txt

@jkotas

Copy link
Copy Markdown
Member

This bug should not be specific to Decimal. Are you able to reproduce it with a custom type with long field? E.g. struct MyStruct { int a; int b; long c; }.

Looks like getClassAlignmentRequirement in crossgen2 may be returning wrong value on arm.

@t-mustafin

Copy link
Copy Markdown
Contributor

Yes, it is reproduced with MyStruct { int a; int b; long c; }. Lambda-expression:

varsoldOutProducts=products.Where(p =>p.MyStruct.a>0x300||p.MyStruct.b>0x50000||p.MyStruct.c>0x6000000);

Offsets generated by crossgen2:

000006 6A48 ldr r0,[r1+36]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6A88 ldr r0,[r1+40]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 002C add r0, r1,4400001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

Offsets generated by Tier-1:

G_M16079_IG02:000006 6A88 ldr r0,[r1+40]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6AC8 ldr r0,[r1+44]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 0030add r0, r1,4800001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

@mangod9

Copy link
Copy Markdown
Member

@t-mustafin since this particular issues is closed, could you please create a new issue for this ? Thx.

@ghostghost locked as resolved and limited conversation to collaborators Dec 8, 2020
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants

@pavel-orekhov@dnfadmin@MichalStrehovsky@jkotas@t-mustafin@mangod9@alpencolt@davidwrighton@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Fix System.Decimal aligment on ARM32 with crossgen2 - #38390

Closed
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master
Closed

Fix System.Decimal aligment on ARM32 with crossgen2#38390
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master

Conversation

@pavel-orekhov

Copy link
Copy Markdown
  • Fix passing Decimal to funcs. Fixes tests/.../decimal.cs and half of tests/.../10w5d.cs

@dnfadmin

dnfadmin commented Jun 25, 2020

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

@MichalStrehovsky

Copy link
Copy Markdown
Member

This should be done similar to how other special types handled:

  • Make a new layout algorithm class deriving from FieldLayoutAlgorithm that does layout for System.Decimal specifically
  • Add it to the if block here:

if(type.IsObject)
return_systemObjectFieldLayoutAlgorithm;
elseif(type==UniversalCanonType)
thrownewNotImplementedException();
elseif(type.IsRuntimeDeterminedType)
thrownewNotImplementedException();
elseif(VectorOfTFieldLayoutAlgorithm.IsVectorOfTType(type))
{
return_vectorOfTFieldLayoutAlgorithm;
}
elseif(VectorFieldLayoutAlgorithm.IsVectorType(type))
{
return_vectorFieldLayoutAlgorithm;
}
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm!=null);
return_r2rFieldLayoutAlgorithm;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The helpers in InteropTypes.cs are specifically for interop marshallers. We don't have a central location with a "is this type X?" functionality. We place the functionality locally where it's needed.

@pavel-orekhovpavel-orekhovJun 25, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks to you!
Ok. Moving modules to another assembly is not very good. So, is it right you suggests to copy-paste InteropTypes::IsSystemDecimal and IsCoreNamedType to R2RCompilerContext.FieldLayoutAlgorithm() / _r2rFieldLayoutAlgorithm and compare strings like

diff --git a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
index bbe1274cae3..d3210f2c5c7 100644
--- a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
+++ b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
@@ -60,6 +60,10 @@ public override FieldLayoutAlgorithm GetLayoutAlgorithmForType(DefType type)
{
return _vectorFieldLayoutAlgorithm;
}
+ else if (type.IsValueType && type.Context.Target.Architecture == TargetArchitecture.ARM && IsSystemDecimal(type.Context, type))
+ {
+ return _decimalOnARM32FieldLayoutAlgorithm;
+ }
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm != null);
return _r2rFieldLayoutAlgorithm;
}

?

Or i can use InteropTypes.IsSystemDecimal() directly here?

May be you suggests to create the flag DefType.isSystemDecimal in order to compare bits, not strings? But I don't know where to set this flag, where the Decimal class loads. :( Do you know it? Can you tell me that?

@MichalStrehovsky

Copy link
Copy Markdown
Member

cc @dotnet/crossgen-contrib

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easiest to check for CoreLib types by name in a method called from here, and apply the layout quirks that way.

Similar to how MethodTableBuilder::CheckForSystemTypes is done. I expect we will need most quirks from this method eventually.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at MethodTableBuilder::CheckForSystemTypes - will we need anything else besides the Decimal addition? We already treat Vector<T> and the other vectors the way I propose for Decimal as well. I don't particularly care about the mechanism, but I do care about consistency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Pro: Checking type in ReadyToRunCompilerContext allows to don't move InteropTypes to ILC.TypeSystem.R2R and allows to use existing InteropTypes.IsDecimal

  2. Pro: Checking type in ReadyToRunCompilerContext removes checking of platform via ReadyToRunCompilerContext constructor

  3. Contra: Checking type in ReadyToRunCompilerContext requires to write many code to implement other methods of FieldLayoutAlgorithm

  4. Pro: checking type in MetadataFieldLayoutAlgorithm.cs exists already and is almost one-liner

  5. Contra: copy-paste of isDecimal required or moving of InteropTypes to ILC.TypeSystem.R2R

  6. Pro: Moving is already done

  7. Contra: More changes in a branch for static struct will be to mitigate second half of the 10w5d.cs test.

  8. Contra: Colleagues say, more types with broken alignment on ARM32 may arise.

So, Where? Please tell me the consensus!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also faced with similar issues in Nullable types. If there are already exceptions for vectors and other types in ReadyToRunCompilerContext I think it would be preferred place.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the problem with Nullable types? Nullable types should not have any layout quirks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a nullable long enum isn't handled correctly, that is a bug in the general purpose type layout logic, not something that should be special cased.

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LongE & so seem are fixed by #32663 / #32664.
But I afraid it is only masking of bug:
When (before) jit/methodical/fp/exgen/10w5d.csFunc_* don't compiled by cg2 but delayed to jit while ctors is done by cg2. So some difference in placement of static structs was (IMHO) between cg2 & jit compilers was and leaded to missfunction.
Right now placement of statics and using of it is by cg2 and you can't see difference with jit's ideas.

So bug (10w5d, LongE and so) may resurrect when cg2 and jit will be used. For example in eval case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Decimal, I am wondering whether we should fix the Decimal implementation to make this special casing unncessary. I will take a look at what it would take today.

LongE & so seem are fixed by #32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

With 4e5c8c0 (today's (20200629) master) the jit/methodical/fp/exgen/10w5d.cs is ok.

How to force 'delayed compilation' back I don't know (eval does not exists, imho). Sorry.
If you mean mass testing clr tests with cg2 please look to our night test results:
results.tar.gz

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#38603 should be a cleaner fix for the Decimal issue.

@pavel-orekhov

Copy link
Copy Markdown
Author

cc @alpencolt

* Fix passing Decimal to funcs. It fixes the JIT/Methodical/MDArray/DataTypes/decimal.cs
@pavel-orekhov

Copy link
Copy Markdown
Author

def2b07 is @jkotas's style rewriting ;)

jkotas added a commit to jkotas/runtime that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artifically inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixesdotnet#38390
jkotas added a commit that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artificially inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixes#38390
@jkotas

Copy link
Copy Markdown
Member

@pavel-orekhov I have fixed decimal implementation to not require this quirk. Give it a try

@t-mustafin

Copy link
Copy Markdown
Contributor

I hit on different input argument r1 usage generated by crossgen2 and Tier-1 compilation on lambda-expression containing decimal variable:

varsoldOutProducts=products.Where(p =>p.UnitsInStock>0&&p.UnitPrice>60.00M);

from Linq test.
Code by crossgen2 loads input decimal from r1+20:

00001A 6948 ldr r0,[r1+20]00001C 9004str r0,[sp+0x10]00001E 6988 ldr r0,[r1+24]0000209005str r0,[sp+0x14]000022 69C8 ldr r0,[r1+28]0000249006str r0,[sp+0x18]000026 6A08 ldr r0,[r1+32]0000289007str r0,[sp+0x1c]

meanwhile tier-1 compiled code loads input decimal from r1+24:

00001A F101 0018add r0, r1,2400001E 6804 ldr r4,[r0]0000206845 ldr r5,[r0+4]0000223008 adds r0,80000246801 ldr r1,[r0]0000266840 ldr r0,[r0+4]0000289108str r1,[sp+0x20]00002A 9009str r0,[sp+0x24]

That incompatibility leads to test fail after tier compilation of that lambda-expression. Test with turned off tier compilation finish successfully.
Linq.cg2.dump.txt
Linq.cg2.S.txt
Linq.cg2-tier.dump.txt
Linq.cg2-tier.S.txt

@jkotas

Copy link
Copy Markdown
Member

This bug should not be specific to Decimal. Are you able to reproduce it with a custom type with long field? E.g. struct MyStruct { int a; int b; long c; }.

Looks like getClassAlignmentRequirement in crossgen2 may be returning wrong value on arm.

@t-mustafin

Copy link
Copy Markdown
Contributor

Yes, it is reproduced with MyStruct { int a; int b; long c; }. Lambda-expression:

varsoldOutProducts=products.Where(p =>p.MyStruct.a>0x300||p.MyStruct.b>0x50000||p.MyStruct.c>0x6000000);

Offsets generated by crossgen2:

000006 6A48 ldr r0,[r1+36]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6A88 ldr r0,[r1+40]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 002C add r0, r1,4400001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

Offsets generated by Tier-1:

G_M16079_IG02:000006 6A88 ldr r0,[r1+40]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6AC8 ldr r0,[r1+44]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 0030add r0, r1,4800001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

@mangod9

Copy link
Copy Markdown
Member

@t-mustafin since this particular issues is closed, could you please create a new issue for this ? Thx.

@ghostghost locked as resolved and limited conversation to collaborators Dec 8, 2020
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants

@pavel-orekhov@dnfadmin@MichalStrehovsky@jkotas@t-mustafin@mangod9@alpencolt@davidwrighton@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix System.Decimal aligment on ARM32 with crossgen2 - #38390

Closed
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master
Closed

Fix System.Decimal aligment on ARM32 with crossgen2#38390
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master

Conversation

@pavel-orekhov

Copy link
Copy Markdown
  • Fix passing Decimal to funcs. Fixes tests/.../decimal.cs and half of tests/.../10w5d.cs

@dnfadmin

dnfadmin commented Jun 25, 2020

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

@MichalStrehovsky

Copy link
Copy Markdown
Member

This should be done similar to how other special types handled:

  • Make a new layout algorithm class deriving from FieldLayoutAlgorithm that does layout for System.Decimal specifically
  • Add it to the if block here:

if(type.IsObject)
return_systemObjectFieldLayoutAlgorithm;
elseif(type==UniversalCanonType)
thrownewNotImplementedException();
elseif(type.IsRuntimeDeterminedType)
thrownewNotImplementedException();
elseif(VectorOfTFieldLayoutAlgorithm.IsVectorOfTType(type))
{
return_vectorOfTFieldLayoutAlgorithm;
}
elseif(VectorFieldLayoutAlgorithm.IsVectorType(type))
{
return_vectorFieldLayoutAlgorithm;
}
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm!=null);
return_r2rFieldLayoutAlgorithm;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The helpers in InteropTypes.cs are specifically for interop marshallers. We don't have a central location with a "is this type X?" functionality. We place the functionality locally where it's needed.

@pavel-orekhovpavel-orekhovJun 25, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks to you!
Ok. Moving modules to another assembly is not very good. So, is it right you suggests to copy-paste InteropTypes::IsSystemDecimal and IsCoreNamedType to R2RCompilerContext.FieldLayoutAlgorithm() / _r2rFieldLayoutAlgorithm and compare strings like

diff --git a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
index bbe1274cae3..d3210f2c5c7 100644
--- a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
+++ b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
@@ -60,6 +60,10 @@ public override FieldLayoutAlgorithm GetLayoutAlgorithmForType(DefType type)
{
return _vectorFieldLayoutAlgorithm;
}
+ else if (type.IsValueType && type.Context.Target.Architecture == TargetArchitecture.ARM && IsSystemDecimal(type.Context, type))
+ {
+ return _decimalOnARM32FieldLayoutAlgorithm;
+ }
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm != null);
return _r2rFieldLayoutAlgorithm;
}

?

Or i can use InteropTypes.IsSystemDecimal() directly here?

May be you suggests to create the flag DefType.isSystemDecimal in order to compare bits, not strings? But I don't know where to set this flag, where the Decimal class loads. :( Do you know it? Can you tell me that?

@MichalStrehovsky

Copy link
Copy Markdown
Member

cc @dotnet/crossgen-contrib

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easiest to check for CoreLib types by name in a method called from here, and apply the layout quirks that way.

Similar to how MethodTableBuilder::CheckForSystemTypes is done. I expect we will need most quirks from this method eventually.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at MethodTableBuilder::CheckForSystemTypes - will we need anything else besides the Decimal addition? We already treat Vector<T> and the other vectors the way I propose for Decimal as well. I don't particularly care about the mechanism, but I do care about consistency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Pro: Checking type in ReadyToRunCompilerContext allows to don't move InteropTypes to ILC.TypeSystem.R2R and allows to use existing InteropTypes.IsDecimal

  2. Pro: Checking type in ReadyToRunCompilerContext removes checking of platform via ReadyToRunCompilerContext constructor

  3. Contra: Checking type in ReadyToRunCompilerContext requires to write many code to implement other methods of FieldLayoutAlgorithm

  4. Pro: checking type in MetadataFieldLayoutAlgorithm.cs exists already and is almost one-liner

  5. Contra: copy-paste of isDecimal required or moving of InteropTypes to ILC.TypeSystem.R2R

  6. Pro: Moving is already done

  7. Contra: More changes in a branch for static struct will be to mitigate second half of the 10w5d.cs test.

  8. Contra: Colleagues say, more types with broken alignment on ARM32 may arise.

So, Where? Please tell me the consensus!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also faced with similar issues in Nullable types. If there are already exceptions for vectors and other types in ReadyToRunCompilerContext I think it would be preferred place.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the problem with Nullable types? Nullable types should not have any layout quirks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a nullable long enum isn't handled correctly, that is a bug in the general purpose type layout logic, not something that should be special cased.

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LongE & so seem are fixed by #32663 / #32664.
But I afraid it is only masking of bug:
When (before) jit/methodical/fp/exgen/10w5d.csFunc_* don't compiled by cg2 but delayed to jit while ctors is done by cg2. So some difference in placement of static structs was (IMHO) between cg2 & jit compilers was and leaded to missfunction.
Right now placement of statics and using of it is by cg2 and you can't see difference with jit's ideas.

So bug (10w5d, LongE and so) may resurrect when cg2 and jit will be used. For example in eval case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Decimal, I am wondering whether we should fix the Decimal implementation to make this special casing unncessary. I will take a look at what it would take today.

LongE & so seem are fixed by #32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

With 4e5c8c0 (today's (20200629) master) the jit/methodical/fp/exgen/10w5d.cs is ok.

How to force 'delayed compilation' back I don't know (eval does not exists, imho). Sorry.
If you mean mass testing clr tests with cg2 please look to our night test results:
results.tar.gz

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#38603 should be a cleaner fix for the Decimal issue.

@pavel-orekhov

Copy link
Copy Markdown
Author

cc @alpencolt

* Fix passing Decimal to funcs. It fixes the JIT/Methodical/MDArray/DataTypes/decimal.cs
@pavel-orekhov

Copy link
Copy Markdown
Author

def2b07 is @jkotas's style rewriting ;)

jkotas added a commit to jkotas/runtime that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artifically inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixesdotnet#38390
jkotas added a commit that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artificially inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixes#38390
@jkotas

Copy link
Copy Markdown
Member

@pavel-orekhov I have fixed decimal implementation to not require this quirk. Give it a try

@t-mustafin

Copy link
Copy Markdown
Contributor

I hit on different input argument r1 usage generated by crossgen2 and Tier-1 compilation on lambda-expression containing decimal variable:

varsoldOutProducts=products.Where(p =>p.UnitsInStock>0&&p.UnitPrice>60.00M);

from Linq test.
Code by crossgen2 loads input decimal from r1+20:

00001A 6948 ldr r0,[r1+20]00001C 9004str r0,[sp+0x10]00001E 6988 ldr r0,[r1+24]0000209005str r0,[sp+0x14]000022 69C8 ldr r0,[r1+28]0000249006str r0,[sp+0x18]000026 6A08 ldr r0,[r1+32]0000289007str r0,[sp+0x1c]

meanwhile tier-1 compiled code loads input decimal from r1+24:

00001A F101 0018add r0, r1,2400001E 6804 ldr r4,[r0]0000206845 ldr r5,[r0+4]0000223008 adds r0,80000246801 ldr r1,[r0]0000266840 ldr r0,[r0+4]0000289108str r1,[sp+0x20]00002A 9009str r0,[sp+0x24]

That incompatibility leads to test fail after tier compilation of that lambda-expression. Test with turned off tier compilation finish successfully.
Linq.cg2.dump.txt
Linq.cg2.S.txt
Linq.cg2-tier.dump.txt
Linq.cg2-tier.S.txt

@jkotas

Copy link
Copy Markdown
Member

This bug should not be specific to Decimal. Are you able to reproduce it with a custom type with long field? E.g. struct MyStruct { int a; int b; long c; }.

Looks like getClassAlignmentRequirement in crossgen2 may be returning wrong value on arm.

@t-mustafin

Copy link
Copy Markdown
Contributor

Yes, it is reproduced with MyStruct { int a; int b; long c; }. Lambda-expression:

varsoldOutProducts=products.Where(p =>p.MyStruct.a>0x300||p.MyStruct.b>0x50000||p.MyStruct.c>0x6000000);

Offsets generated by crossgen2:

000006 6A48 ldr r0,[r1+36]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6A88 ldr r0,[r1+40]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 002C add r0, r1,4400001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

Offsets generated by Tier-1:

G_M16079_IG02:000006 6A88 ldr r0,[r1+40]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6AC8 ldr r0,[r1+44]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 0030add r0, r1,4800001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

@mangod9

Copy link
Copy Markdown
Member

@t-mustafin since this particular issues is closed, could you please create a new issue for this ? Thx.

@ghostghost locked as resolved and limited conversation to collaborators Dec 8, 2020
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants

@pavel-orekhov@dnfadmin@MichalStrehovsky@jkotas@t-mustafin@mangod9@alpencolt@davidwrighton@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Fix System.Decimal aligment on ARM32 with crossgen2 - #38390

Closed
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master
Closed

Fix System.Decimal aligment on ARM32 with crossgen2#38390
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master

Conversation

@pavel-orekhov

Copy link
Copy Markdown
  • Fix passing Decimal to funcs. Fixes tests/.../decimal.cs and half of tests/.../10w5d.cs

@dnfadmin

dnfadmin commented Jun 25, 2020

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

@MichalStrehovsky

Copy link
Copy Markdown
Member

This should be done similar to how other special types handled:

  • Make a new layout algorithm class deriving from FieldLayoutAlgorithm that does layout for System.Decimal specifically
  • Add it to the if block here:

if(type.IsObject)
return_systemObjectFieldLayoutAlgorithm;
elseif(type==UniversalCanonType)
thrownewNotImplementedException();
elseif(type.IsRuntimeDeterminedType)
thrownewNotImplementedException();
elseif(VectorOfTFieldLayoutAlgorithm.IsVectorOfTType(type))
{
return_vectorOfTFieldLayoutAlgorithm;
}
elseif(VectorFieldLayoutAlgorithm.IsVectorType(type))
{
return_vectorFieldLayoutAlgorithm;
}
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm!=null);
return_r2rFieldLayoutAlgorithm;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The helpers in InteropTypes.cs are specifically for interop marshallers. We don't have a central location with a "is this type X?" functionality. We place the functionality locally where it's needed.

@pavel-orekhovpavel-orekhovJun 25, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks to you!
Ok. Moving modules to another assembly is not very good. So, is it right you suggests to copy-paste InteropTypes::IsSystemDecimal and IsCoreNamedType to R2RCompilerContext.FieldLayoutAlgorithm() / _r2rFieldLayoutAlgorithm and compare strings like

diff --git a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
index bbe1274cae3..d3210f2c5c7 100644
--- a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
+++ b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
@@ -60,6 +60,10 @@ public override FieldLayoutAlgorithm GetLayoutAlgorithmForType(DefType type)
{
return _vectorFieldLayoutAlgorithm;
}
+ else if (type.IsValueType && type.Context.Target.Architecture == TargetArchitecture.ARM && IsSystemDecimal(type.Context, type))
+ {
+ return _decimalOnARM32FieldLayoutAlgorithm;
+ }
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm != null);
return _r2rFieldLayoutAlgorithm;
}

?

Or i can use InteropTypes.IsSystemDecimal() directly here?

May be you suggests to create the flag DefType.isSystemDecimal in order to compare bits, not strings? But I don't know where to set this flag, where the Decimal class loads. :( Do you know it? Can you tell me that?

@MichalStrehovsky

Copy link
Copy Markdown
Member

cc @dotnet/crossgen-contrib

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easiest to check for CoreLib types by name in a method called from here, and apply the layout quirks that way.

Similar to how MethodTableBuilder::CheckForSystemTypes is done. I expect we will need most quirks from this method eventually.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at MethodTableBuilder::CheckForSystemTypes - will we need anything else besides the Decimal addition? We already treat Vector<T> and the other vectors the way I propose for Decimal as well. I don't particularly care about the mechanism, but I do care about consistency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Pro: Checking type in ReadyToRunCompilerContext allows to don't move InteropTypes to ILC.TypeSystem.R2R and allows to use existing InteropTypes.IsDecimal

  2. Pro: Checking type in ReadyToRunCompilerContext removes checking of platform via ReadyToRunCompilerContext constructor

  3. Contra: Checking type in ReadyToRunCompilerContext requires to write many code to implement other methods of FieldLayoutAlgorithm

  4. Pro: checking type in MetadataFieldLayoutAlgorithm.cs exists already and is almost one-liner

  5. Contra: copy-paste of isDecimal required or moving of InteropTypes to ILC.TypeSystem.R2R

  6. Pro: Moving is already done

  7. Contra: More changes in a branch for static struct will be to mitigate second half of the 10w5d.cs test.

  8. Contra: Colleagues say, more types with broken alignment on ARM32 may arise.

So, Where? Please tell me the consensus!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also faced with similar issues in Nullable types. If there are already exceptions for vectors and other types in ReadyToRunCompilerContext I think it would be preferred place.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the problem with Nullable types? Nullable types should not have any layout quirks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a nullable long enum isn't handled correctly, that is a bug in the general purpose type layout logic, not something that should be special cased.

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LongE & so seem are fixed by #32663 / #32664.
But I afraid it is only masking of bug:
When (before) jit/methodical/fp/exgen/10w5d.csFunc_* don't compiled by cg2 but delayed to jit while ctors is done by cg2. So some difference in placement of static structs was (IMHO) between cg2 & jit compilers was and leaded to missfunction.
Right now placement of statics and using of it is by cg2 and you can't see difference with jit's ideas.

So bug (10w5d, LongE and so) may resurrect when cg2 and jit will be used. For example in eval case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Decimal, I am wondering whether we should fix the Decimal implementation to make this special casing unncessary. I will take a look at what it would take today.

LongE & so seem are fixed by #32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

With 4e5c8c0 (today's (20200629) master) the jit/methodical/fp/exgen/10w5d.cs is ok.

How to force 'delayed compilation' back I don't know (eval does not exists, imho). Sorry.
If you mean mass testing clr tests with cg2 please look to our night test results:
results.tar.gz

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#38603 should be a cleaner fix for the Decimal issue.

@pavel-orekhov

Copy link
Copy Markdown
Author

cc @alpencolt

* Fix passing Decimal to funcs. It fixes the JIT/Methodical/MDArray/DataTypes/decimal.cs
@pavel-orekhov

Copy link
Copy Markdown
Author

def2b07 is @jkotas's style rewriting ;)

jkotas added a commit to jkotas/runtime that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artifically inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixesdotnet#38390
jkotas added a commit that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artificially inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixes#38390
@jkotas

Copy link
Copy Markdown
Member

@pavel-orekhov I have fixed decimal implementation to not require this quirk. Give it a try

@t-mustafin

Copy link
Copy Markdown
Contributor

I hit on different input argument r1 usage generated by crossgen2 and Tier-1 compilation on lambda-expression containing decimal variable:

varsoldOutProducts=products.Where(p =>p.UnitsInStock>0&&p.UnitPrice>60.00M);

from Linq test.
Code by crossgen2 loads input decimal from r1+20:

00001A 6948 ldr r0,[r1+20]00001C 9004str r0,[sp+0x10]00001E 6988 ldr r0,[r1+24]0000209005str r0,[sp+0x14]000022 69C8 ldr r0,[r1+28]0000249006str r0,[sp+0x18]000026 6A08 ldr r0,[r1+32]0000289007str r0,[sp+0x1c]

meanwhile tier-1 compiled code loads input decimal from r1+24:

00001A F101 0018add r0, r1,2400001E 6804 ldr r4,[r0]0000206845 ldr r5,[r0+4]0000223008 adds r0,80000246801 ldr r1,[r0]0000266840 ldr r0,[r0+4]0000289108str r1,[sp+0x20]00002A 9009str r0,[sp+0x24]

That incompatibility leads to test fail after tier compilation of that lambda-expression. Test with turned off tier compilation finish successfully.
Linq.cg2.dump.txt
Linq.cg2.S.txt
Linq.cg2-tier.dump.txt
Linq.cg2-tier.S.txt

@jkotas

Copy link
Copy Markdown
Member

This bug should not be specific to Decimal. Are you able to reproduce it with a custom type with long field? E.g. struct MyStruct { int a; int b; long c; }.

Looks like getClassAlignmentRequirement in crossgen2 may be returning wrong value on arm.

@t-mustafin

Copy link
Copy Markdown
Contributor

Yes, it is reproduced with MyStruct { int a; int b; long c; }. Lambda-expression:

varsoldOutProducts=products.Where(p =>p.MyStruct.a>0x300||p.MyStruct.b>0x50000||p.MyStruct.c>0x6000000);

Offsets generated by crossgen2:

000006 6A48 ldr r0,[r1+36]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6A88 ldr r0,[r1+40]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 002C add r0, r1,4400001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

Offsets generated by Tier-1:

G_M16079_IG02:000006 6A88 ldr r0,[r1+40]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6AC8 ldr r0,[r1+44]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 0030add r0, r1,4800001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

@mangod9

Copy link
Copy Markdown
Member

@t-mustafin since this particular issues is closed, could you please create a new issue for this ? Thx.

@ghostghost locked as resolved and limited conversation to collaborators Dec 8, 2020
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants

@pavel-orekhov@dnfadmin@MichalStrehovsky@jkotas@t-mustafin@mangod9@alpencolt@davidwrighton@Dotnet-GitSync-Bot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Fix System.Decimal aligment on ARM32 with crossgen2 - #38390

Closed
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master
Closed

Fix System.Decimal aligment on ARM32 with crossgen2#38390
pavel-orekhov wants to merge 1 commit into
dotnet:masterfrom
pavel-orekhov:master

Conversation

@pavel-orekhov

Copy link
Copy Markdown
  • Fix passing Decimal to funcs. Fixes tests/.../decimal.cs and half of tests/.../10w5d.cs

@dnfadmin

dnfadmin commented Jun 25, 2020

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

@MichalStrehovsky

Copy link
Copy Markdown
Member

This should be done similar to how other special types handled:

  • Make a new layout algorithm class deriving from FieldLayoutAlgorithm that does layout for System.Decimal specifically
  • Add it to the if block here:

if(type.IsObject)
return_systemObjectFieldLayoutAlgorithm;
elseif(type==UniversalCanonType)
thrownewNotImplementedException();
elseif(type.IsRuntimeDeterminedType)
thrownewNotImplementedException();
elseif(VectorOfTFieldLayoutAlgorithm.IsVectorOfTType(type))
{
return_vectorOfTFieldLayoutAlgorithm;
}
elseif(VectorFieldLayoutAlgorithm.IsVectorType(type))
{
return_vectorFieldLayoutAlgorithm;
}
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm!=null);
return_r2rFieldLayoutAlgorithm;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The helpers in InteropTypes.cs are specifically for interop marshallers. We don't have a central location with a "is this type X?" functionality. We place the functionality locally where it's needed.

@pavel-orekhovpavel-orekhovJun 25, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks to you!
Ok. Moving modules to another assembly is not very good. So, is it right you suggests to copy-paste InteropTypes::IsSystemDecimal and IsCoreNamedType to R2RCompilerContext.FieldLayoutAlgorithm() / _r2rFieldLayoutAlgorithm and compare strings like

diff --git a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
index bbe1274cae3..d3210f2c5c7 100644
--- a/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
+++ b/src/coreclr/src/tools/crossgen2/ILCompiler.ReadyToRun/Compiler/ReadyToRunCompilerContext.cs
@@ -60,6 +60,10 @@ public override FieldLayoutAlgorithm GetLayoutAlgorithmForType(DefType type)
{
return _vectorFieldLayoutAlgorithm;
}
+ else if (type.IsValueType && type.Context.Target.Architecture == TargetArchitecture.ARM && IsSystemDecimal(type.Context, type))
+ {
+ return _decimalOnARM32FieldLayoutAlgorithm;
+ }
else
{
Debug.Assert(_r2rFieldLayoutAlgorithm != null);
return _r2rFieldLayoutAlgorithm;
}

?

Or i can use InteropTypes.IsSystemDecimal() directly here?

May be you suggests to create the flag DefType.isSystemDecimal in order to compare bits, not strings? But I don't know where to set this flag, where the Decimal class loads. :( Do you know it? Can you tell me that?

@MichalStrehovsky

Copy link
Copy Markdown
Member

cc @dotnet/crossgen-contrib

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be easiest to check for CoreLib types by name in a method called from here, and apply the layout quirks that way.

Similar to how MethodTableBuilder::CheckForSystemTypes is done. I expect we will need most quirks from this method eventually.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at MethodTableBuilder::CheckForSystemTypes - will we need anything else besides the Decimal addition? We already treat Vector<T> and the other vectors the way I propose for Decimal as well. I don't particularly care about the mechanism, but I do care about consistency.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Pro: Checking type in ReadyToRunCompilerContext allows to don't move InteropTypes to ILC.TypeSystem.R2R and allows to use existing InteropTypes.IsDecimal

  2. Pro: Checking type in ReadyToRunCompilerContext removes checking of platform via ReadyToRunCompilerContext constructor

  3. Contra: Checking type in ReadyToRunCompilerContext requires to write many code to implement other methods of FieldLayoutAlgorithm

  4. Pro: checking type in MetadataFieldLayoutAlgorithm.cs exists already and is almost one-liner

  5. Contra: copy-paste of isDecimal required or moving of InteropTypes to ILC.TypeSystem.R2R

  6. Pro: Moving is already done

  7. Contra: More changes in a branch for static struct will be to mitigate second half of the 10w5d.cs test.

  8. Contra: Colleagues say, more types with broken alignment on ARM32 may arise.

So, Where? Please tell me the consensus!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also faced with similar issues in Nullable types. If there are already exceptions for vectors and other types in ReadyToRunCompilerContext I think it would be preferred place.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the problem with Nullable types? Nullable types should not have any layout quirks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a nullable long enum isn't handled correctly, that is a bug in the general purpose type layout logic, not something that should be special cased.

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LongE & so seem are fixed by #32663 / #32664.
But I afraid it is only masking of bug:
When (before) jit/methodical/fp/exgen/10w5d.csFunc_* don't compiled by cg2 but delayed to jit while ctors is done by cg2. So some difference in placement of static structs was (IMHO) between cg2 & jit compilers was and leaded to missfunction.
Right now placement of statics and using of it is by cg2 and you can't see difference with jit's ideas.

So bug (10w5d, LongE and so) may resurrect when cg2 and jit will be used. For example in eval case.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Decimal, I am wondering whether we should fix the Decimal implementation to make this special casing unncessary. I will take a look at what it would take today.

LongE & so seem are fixed by #32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

@pavel-orekhovpavel-orekhovJun 29, 2020

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#32663 / #32664.

I have reverted this workaround a few days ago. Could you please check whether it is still a problem on current master?

With 4e5c8c0 (today's (20200629) master) the jit/methodical/fp/exgen/10w5d.cs is ok.

How to force 'delayed compilation' back I don't know (eval does not exists, imho). Sorry.
If you mean mass testing clr tests with cg2 please look to our night test results:
results.tar.gz

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#38603 should be a cleaner fix for the Decimal issue.

@pavel-orekhov

Copy link
Copy Markdown
Author

cc @alpencolt

* Fix passing Decimal to funcs. It fixes the JIT/Methodical/MDArray/DataTypes/decimal.cs
@pavel-orekhov

Copy link
Copy Markdown
Author

def2b07 is @jkotas's style rewriting ;)

jkotas added a commit to jkotas/runtime that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artifically inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixesdotnet#38390
jkotas added a commit that referenced this pull request Jun 30, 2020
System.Decimal fields did not match Win32 DECIMAL type for historic reasons. It required type loader to have a quirk to artificially inflate System.Decimal alignment to match the alignment of Win32 DECIMAL type to make interop work well.
This change is fixing the System.Decimal fields to match Win32 DECIMAL type and removing the quirk from the type loader since it does not belong there.
The downsides are:
- Slightly lower code quality on 32-bit platforms. 32-bit platforms are not our high performance targets anymore.
- Explicit implementation of ISerializable is required on System.Decimal for binary serialization compatibility.
Fixes#38390
@jkotas

Copy link
Copy Markdown
Member

@pavel-orekhov I have fixed decimal implementation to not require this quirk. Give it a try

@t-mustafin

Copy link
Copy Markdown
Contributor

I hit on different input argument r1 usage generated by crossgen2 and Tier-1 compilation on lambda-expression containing decimal variable:

varsoldOutProducts=products.Where(p =>p.UnitsInStock>0&&p.UnitPrice>60.00M);

from Linq test.
Code by crossgen2 loads input decimal from r1+20:

00001A 6948 ldr r0,[r1+20]00001C 9004str r0,[sp+0x10]00001E 6988 ldr r0,[r1+24]0000209005str r0,[sp+0x14]000022 69C8 ldr r0,[r1+28]0000249006str r0,[sp+0x18]000026 6A08 ldr r0,[r1+32]0000289007str r0,[sp+0x1c]

meanwhile tier-1 compiled code loads input decimal from r1+24:

00001A F101 0018add r0, r1,2400001E 6804 ldr r4,[r0]0000206845 ldr r5,[r0+4]0000223008 adds r0,80000246801 ldr r1,[r0]0000266840 ldr r0,[r0+4]0000289108str r1,[sp+0x20]00002A 9009str r0,[sp+0x24]

That incompatibility leads to test fail after tier compilation of that lambda-expression. Test with turned off tier compilation finish successfully.
Linq.cg2.dump.txt
Linq.cg2.S.txt
Linq.cg2-tier.dump.txt
Linq.cg2-tier.S.txt

@jkotas

Copy link
Copy Markdown
Member

This bug should not be specific to Decimal. Are you able to reproduce it with a custom type with long field? E.g. struct MyStruct { int a; int b; long c; }.

Looks like getClassAlignmentRequirement in crossgen2 may be returning wrong value on arm.

@t-mustafin

Copy link
Copy Markdown
Contributor

Yes, it is reproduced with MyStruct { int a; int b; long c; }. Lambda-expression:

varsoldOutProducts=products.Where(p =>p.MyStruct.a>0x300||p.MyStruct.b>0x50000||p.MyStruct.c>0x6000000);

Offsets generated by crossgen2:

000006 6A48 ldr r0,[r1+36]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6A88 ldr r0,[r1+40]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 002C add r0, r1,4400001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

Offsets generated by Tier-1:

G_M16079_IG02:000006 6A88 ldr r0,[r1+40]000008 F5B0 7F40 cmp r0,76800000C DC14 bgt SHORT G_M16079_IG06 ;; bbWeight=1 PerfScore 3.00G_M16079_IG03:00000E 6AC8 ldr r0,[r1+44]000010 F5B0 2FA0 cmp r0,0x50000000014 DC10 bgt SHORT G_M16079_IG06000016 F101 0030add r0, r1,4800001A 6803 ldr r3,[r0]00001C 6840 ldr r0,[r0+4]

@mangod9

Copy link
Copy Markdown
Member

@t-mustafin since this particular issues is closed, could you please create a new issue for this ? Thx.

@ghostghost locked as resolved and limited conversation to collaborators Dec 8, 2020
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants

@pavel-orekhov@dnfadmin@MichalStrehovsky@jkotas@t-mustafin@mangod9@alpencolt@davidwrighton@Dotnet-GitSync-Bot