Precisely track reflected on fields - #70546

Merged
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection
Jun 13, 2022
Merged

Precisely track reflected on fields#70546
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection

Conversation

@MichalStrehovsky

Copy link
Copy Markdown
Member

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.

With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.

We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided Class<int>.Foo should be reflection accessible and Class<double> is used elsewhere in the program, Class<double>.Foo is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.

I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter in the past.

Added a couple new tests that test the various invariants.

This saves 2% in size on a dotnet new webapi template project with IlcTrimMetadata on. It is a small regression with IlcTrimMetadata off because we actually now track more things for the reflectable field: previously we would not make sure the field is reflection-accessible at runtime. We need a TypeHandle for the field type for it to be usable. As a potential future optimization, we could look into allowing reflection to see "unconstructed" TypeHandles (and use that one). Reflection is currently not allowed to see unconstructed TypeHandles to prevent people falling into a RuntimeHelpers.AllocateUninitializedObject(someField.FieldType.TypeHandle) trap that has a bad failure mode right now. Once we fix the failure mode, we could potentially allow it.

Cc @dotnet/ilc-contrib

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.
With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.
We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided `Class<int>.Foo` should be reflection accessible and `Class<double>` is used elsewhere in the program, `Class<double>.Foo` is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.
I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter.
Added a couple new tests that test the various invariants.

@vitek-karasvitek-karas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - but I don't know enough of the runtime side to tell if this will keep enough info around for the runtime to work correctly (I mean the tests work, but for a review).

Comment on lines +74 to +78
else
{
dependencies.Add(factory.TypeNonGCStaticsSymbol((MetadataType)_field.OwningType), "NonGC static base of a reflectable field");
needsNonGcStaticBase = false;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just remove this branch and let the if immediately below do this? It seems to be doing the same thing.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's part of a if/else cascade - we only need to add the TypeNonGCStaticsSymbol if there's a class constructor, or the field is non-GC static. There's no quick way to check if a field is non-GC static - one has to ask the "is it RVA/ThreadStatic/GCStatic" questions first.

The best we could do here is change this else block to needsNonGcStaticBase = true; and delete the dependencies.Add line but then the reason string will be wrong.

// so for enums also include their MethodTable.
dependencies.Add(factory.MaximallyConstructableType(_type), "Reflectable enum");

// Enums are not useful without their literal fields

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the comment, but my question is why do we need to explicitly preserve the metadata for the fields - I would expect that if the app uses them we would see their usage directly. Or is this because annotations/suppressions in framework assume this? (If so I think it's worth a comment about that).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields are never referenced from code. Basically, an enum looks like this:

classMyEnum:Enum{privateint_value;publicconstintValue1=1;publicconstintValue2=2;}

One cannot do ldsfld MyEnum.Value1 in IL. It's lowered into ldc.i4.1.

Added comment.

Comment on lines +124 to +125
var reflectableFieldNode = obj as ReflectableFieldNode;
if (reflectableFieldNode != null)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is based on the existing code, but it would look "nicer" like this:

if(objisReflectableFieldNodereflectableFieldNode)

Maybe even change the whole thing into a switch?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to follow the existing style in the file and don't change surrounding style as part of unrelated changes. It makes history/git blame tidier. We could reformat in a single pass if it's a problem.

Comment on lines +419 to +421
// Tiny optimization: no get/set for literal fields
if (field.IsLiteral)
continue;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So literal fields have special handling in reflection code? Meaning they don't need any runtime support, it's all done via metadata, right?
(and we assume that since it's literal its type is primitive/string and thus we don't need to explicitly preserve its type either).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literal fields (const/enum values) only exist in metadata. They don't have a "value" that is accessible through IL. Reflection GetValue reads the value from the metadata instead of reading some location in memory (like we do for other fields). There's no location in memory to worry about.

@MichalStrehovsky
MichalStrehovsky merged commit f54c15a into dotnet:mainJun 13, 2022
@MichalStrehovsky
MichalStrehovsky deleted the fieldReflection branch June 13, 2022 01:42
@ghostghost locked as resolved and limited conversation to collaborators Jul 13, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@MichalStrehovsky@vitek-karas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Precisely track reflected on fields - #70546

Merged
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection
Jun 13, 2022
Merged

Precisely track reflected on fields#70546
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection

Conversation

@MichalStrehovsky

Copy link
Copy Markdown
Member

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.

With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.

We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided Class<int>.Foo should be reflection accessible and Class<double> is used elsewhere in the program, Class<double>.Foo is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.

I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter in the past.

Added a couple new tests that test the various invariants.

This saves 2% in size on a dotnet new webapi template project with IlcTrimMetadata on. It is a small regression with IlcTrimMetadata off because we actually now track more things for the reflectable field: previously we would not make sure the field is reflection-accessible at runtime. We need a TypeHandle for the field type for it to be usable. As a potential future optimization, we could look into allowing reflection to see "unconstructed" TypeHandles (and use that one). Reflection is currently not allowed to see unconstructed TypeHandles to prevent people falling into a RuntimeHelpers.AllocateUninitializedObject(someField.FieldType.TypeHandle) trap that has a bad failure mode right now. Once we fix the failure mode, we could potentially allow it.

Cc @dotnet/ilc-contrib

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.
With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.
We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided `Class<int>.Foo` should be reflection accessible and `Class<double>` is used elsewhere in the program, `Class<double>.Foo` is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.
I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter.
Added a couple new tests that test the various invariants.

@vitek-karasvitek-karas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - but I don't know enough of the runtime side to tell if this will keep enough info around for the runtime to work correctly (I mean the tests work, but for a review).

Comment on lines +74 to +78
else
{
dependencies.Add(factory.TypeNonGCStaticsSymbol((MetadataType)_field.OwningType), "NonGC static base of a reflectable field");
needsNonGcStaticBase = false;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just remove this branch and let the if immediately below do this? It seems to be doing the same thing.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's part of a if/else cascade - we only need to add the TypeNonGCStaticsSymbol if there's a class constructor, or the field is non-GC static. There's no quick way to check if a field is non-GC static - one has to ask the "is it RVA/ThreadStatic/GCStatic" questions first.

The best we could do here is change this else block to needsNonGcStaticBase = true; and delete the dependencies.Add line but then the reason string will be wrong.

// so for enums also include their MethodTable.
dependencies.Add(factory.MaximallyConstructableType(_type), "Reflectable enum");

// Enums are not useful without their literal fields

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the comment, but my question is why do we need to explicitly preserve the metadata for the fields - I would expect that if the app uses them we would see their usage directly. Or is this because annotations/suppressions in framework assume this? (If so I think it's worth a comment about that).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields are never referenced from code. Basically, an enum looks like this:

classMyEnum:Enum{privateint_value;publicconstintValue1=1;publicconstintValue2=2;}

One cannot do ldsfld MyEnum.Value1 in IL. It's lowered into ldc.i4.1.

Added comment.

Comment on lines +124 to +125
var reflectableFieldNode = obj as ReflectableFieldNode;
if (reflectableFieldNode != null)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is based on the existing code, but it would look "nicer" like this:

if(objisReflectableFieldNodereflectableFieldNode)

Maybe even change the whole thing into a switch?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to follow the existing style in the file and don't change surrounding style as part of unrelated changes. It makes history/git blame tidier. We could reformat in a single pass if it's a problem.

Comment on lines +419 to +421
// Tiny optimization: no get/set for literal fields
if (field.IsLiteral)
continue;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So literal fields have special handling in reflection code? Meaning they don't need any runtime support, it's all done via metadata, right?
(and we assume that since it's literal its type is primitive/string and thus we don't need to explicitly preserve its type either).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literal fields (const/enum values) only exist in metadata. They don't have a "value" that is accessible through IL. Reflection GetValue reads the value from the metadata instead of reading some location in memory (like we do for other fields). There's no location in memory to worry about.

@MichalStrehovsky
MichalStrehovsky merged commit f54c15a into dotnet:mainJun 13, 2022
@MichalStrehovsky
MichalStrehovsky deleted the fieldReflection branch June 13, 2022 01:42
@ghostghost locked as resolved and limited conversation to collaborators Jul 13, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@MichalStrehovsky@vitek-karas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Precisely track reflected on fields - #70546

Merged
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection
Jun 13, 2022
Merged

Precisely track reflected on fields#70546
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection

Conversation

@MichalStrehovsky

Copy link
Copy Markdown
Member

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.

With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.

We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided Class<int>.Foo should be reflection accessible and Class<double> is used elsewhere in the program, Class<double>.Foo is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.

I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter in the past.

Added a couple new tests that test the various invariants.

This saves 2% in size on a dotnet new webapi template project with IlcTrimMetadata on. It is a small regression with IlcTrimMetadata off because we actually now track more things for the reflectable field: previously we would not make sure the field is reflection-accessible at runtime. We need a TypeHandle for the field type for it to be usable. As a potential future optimization, we could look into allowing reflection to see "unconstructed" TypeHandles (and use that one). Reflection is currently not allowed to see unconstructed TypeHandles to prevent people falling into a RuntimeHelpers.AllocateUninitializedObject(someField.FieldType.TypeHandle) trap that has a bad failure mode right now. Once we fix the failure mode, we could potentially allow it.

Cc @dotnet/ilc-contrib

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.
With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.
We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided `Class<int>.Foo` should be reflection accessible and `Class<double>` is used elsewhere in the program, `Class<double>.Foo` is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.
I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter.
Added a couple new tests that test the various invariants.

@vitek-karasvitek-karas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - but I don't know enough of the runtime side to tell if this will keep enough info around for the runtime to work correctly (I mean the tests work, but for a review).

Comment on lines +74 to +78
else
{
dependencies.Add(factory.TypeNonGCStaticsSymbol((MetadataType)_field.OwningType), "NonGC static base of a reflectable field");
needsNonGcStaticBase = false;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just remove this branch and let the if immediately below do this? It seems to be doing the same thing.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's part of a if/else cascade - we only need to add the TypeNonGCStaticsSymbol if there's a class constructor, or the field is non-GC static. There's no quick way to check if a field is non-GC static - one has to ask the "is it RVA/ThreadStatic/GCStatic" questions first.

The best we could do here is change this else block to needsNonGcStaticBase = true; and delete the dependencies.Add line but then the reason string will be wrong.

// so for enums also include their MethodTable.
dependencies.Add(factory.MaximallyConstructableType(_type), "Reflectable enum");

// Enums are not useful without their literal fields

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the comment, but my question is why do we need to explicitly preserve the metadata for the fields - I would expect that if the app uses them we would see their usage directly. Or is this because annotations/suppressions in framework assume this? (If so I think it's worth a comment about that).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields are never referenced from code. Basically, an enum looks like this:

classMyEnum:Enum{privateint_value;publicconstintValue1=1;publicconstintValue2=2;}

One cannot do ldsfld MyEnum.Value1 in IL. It's lowered into ldc.i4.1.

Added comment.

Comment on lines +124 to +125
var reflectableFieldNode = obj as ReflectableFieldNode;
if (reflectableFieldNode != null)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is based on the existing code, but it would look "nicer" like this:

if(objisReflectableFieldNodereflectableFieldNode)

Maybe even change the whole thing into a switch?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to follow the existing style in the file and don't change surrounding style as part of unrelated changes. It makes history/git blame tidier. We could reformat in a single pass if it's a problem.

Comment on lines +419 to +421
// Tiny optimization: no get/set for literal fields
if (field.IsLiteral)
continue;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So literal fields have special handling in reflection code? Meaning they don't need any runtime support, it's all done via metadata, right?
(and we assume that since it's literal its type is primitive/string and thus we don't need to explicitly preserve its type either).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literal fields (const/enum values) only exist in metadata. They don't have a "value" that is accessible through IL. Reflection GetValue reads the value from the metadata instead of reading some location in memory (like we do for other fields). There's no location in memory to worry about.

@MichalStrehovsky
MichalStrehovsky merged commit f54c15a into dotnet:mainJun 13, 2022
@MichalStrehovsky
MichalStrehovsky deleted the fieldReflection branch June 13, 2022 01:42
@ghostghost locked as resolved and limited conversation to collaborators Jul 13, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@MichalStrehovsky@vitek-karas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Precisely track reflected on fields - #70546

Merged
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection
Jun 13, 2022
Merged

Precisely track reflected on fields#70546
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection

Conversation

@MichalStrehovsky

Copy link
Copy Markdown
Member

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.

With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.

We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided Class<int>.Foo should be reflection accessible and Class<double> is used elsewhere in the program, Class<double>.Foo is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.

I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter in the past.

Added a couple new tests that test the various invariants.

This saves 2% in size on a dotnet new webapi template project with IlcTrimMetadata on. It is a small regression with IlcTrimMetadata off because we actually now track more things for the reflectable field: previously we would not make sure the field is reflection-accessible at runtime. We need a TypeHandle for the field type for it to be usable. As a potential future optimization, we could look into allowing reflection to see "unconstructed" TypeHandles (and use that one). Reflection is currently not allowed to see unconstructed TypeHandles to prevent people falling into a RuntimeHelpers.AllocateUninitializedObject(someField.FieldType.TypeHandle) trap that has a bad failure mode right now. Once we fix the failure mode, we could potentially allow it.

Cc @dotnet/ilc-contrib

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.
With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.
We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided `Class<int>.Foo` should be reflection accessible and `Class<double>` is used elsewhere in the program, `Class<double>.Foo` is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.
I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter.
Added a couple new tests that test the various invariants.

@vitek-karasvitek-karas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - but I don't know enough of the runtime side to tell if this will keep enough info around for the runtime to work correctly (I mean the tests work, but for a review).

Comment on lines +74 to +78
else
{
dependencies.Add(factory.TypeNonGCStaticsSymbol((MetadataType)_field.OwningType), "NonGC static base of a reflectable field");
needsNonGcStaticBase = false;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just remove this branch and let the if immediately below do this? It seems to be doing the same thing.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's part of a if/else cascade - we only need to add the TypeNonGCStaticsSymbol if there's a class constructor, or the field is non-GC static. There's no quick way to check if a field is non-GC static - one has to ask the "is it RVA/ThreadStatic/GCStatic" questions first.

The best we could do here is change this else block to needsNonGcStaticBase = true; and delete the dependencies.Add line but then the reason string will be wrong.

// so for enums also include their MethodTable.
dependencies.Add(factory.MaximallyConstructableType(_type), "Reflectable enum");

// Enums are not useful without their literal fields

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the comment, but my question is why do we need to explicitly preserve the metadata for the fields - I would expect that if the app uses them we would see their usage directly. Or is this because annotations/suppressions in framework assume this? (If so I think it's worth a comment about that).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields are never referenced from code. Basically, an enum looks like this:

classMyEnum:Enum{privateint_value;publicconstintValue1=1;publicconstintValue2=2;}

One cannot do ldsfld MyEnum.Value1 in IL. It's lowered into ldc.i4.1.

Added comment.

Comment on lines +124 to +125
var reflectableFieldNode = obj as ReflectableFieldNode;
if (reflectableFieldNode != null)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is based on the existing code, but it would look "nicer" like this:

if(objisReflectableFieldNodereflectableFieldNode)

Maybe even change the whole thing into a switch?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to follow the existing style in the file and don't change surrounding style as part of unrelated changes. It makes history/git blame tidier. We could reformat in a single pass if it's a problem.

Comment on lines +419 to +421
// Tiny optimization: no get/set for literal fields
if (field.IsLiteral)
continue;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So literal fields have special handling in reflection code? Meaning they don't need any runtime support, it's all done via metadata, right?
(and we assume that since it's literal its type is primitive/string and thus we don't need to explicitly preserve its type either).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literal fields (const/enum values) only exist in metadata. They don't have a "value" that is accessible through IL. Reflection GetValue reads the value from the metadata instead of reading some location in memory (like we do for other fields). There's no location in memory to worry about.

@MichalStrehovsky
MichalStrehovsky merged commit f54c15a into dotnet:mainJun 13, 2022
@MichalStrehovsky
MichalStrehovsky deleted the fieldReflection branch June 13, 2022 01:42
@ghostghost locked as resolved and limited conversation to collaborators Jul 13, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@MichalStrehovsky@vitek-karas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Precisely track reflected on fields - #70546

Merged
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection
Jun 13, 2022
Merged

Precisely track reflected on fields#70546
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection

Conversation

@MichalStrehovsky

Copy link
Copy Markdown
Member

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.

With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.

We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided Class<int>.Foo should be reflection accessible and Class<double> is used elsewhere in the program, Class<double>.Foo is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.

I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter in the past.

Added a couple new tests that test the various invariants.

This saves 2% in size on a dotnet new webapi template project with IlcTrimMetadata on. It is a small regression with IlcTrimMetadata off because we actually now track more things for the reflectable field: previously we would not make sure the field is reflection-accessible at runtime. We need a TypeHandle for the field type for it to be usable. As a potential future optimization, we could look into allowing reflection to see "unconstructed" TypeHandles (and use that one). Reflection is currently not allowed to see unconstructed TypeHandles to prevent people falling into a RuntimeHelpers.AllocateUninitializedObject(someField.FieldType.TypeHandle) trap that has a bad failure mode right now. Once we fix the failure mode, we could potentially allow it.

Cc @dotnet/ilc-contrib

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.
With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.
We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided `Class<int>.Foo` should be reflection accessible and `Class<double>` is used elsewhere in the program, `Class<double>.Foo` is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.
I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter.
Added a couple new tests that test the various invariants.

@vitek-karasvitek-karas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - but I don't know enough of the runtime side to tell if this will keep enough info around for the runtime to work correctly (I mean the tests work, but for a review).

Comment on lines +74 to +78
else
{
dependencies.Add(factory.TypeNonGCStaticsSymbol((MetadataType)_field.OwningType), "NonGC static base of a reflectable field");
needsNonGcStaticBase = false;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just remove this branch and let the if immediately below do this? It seems to be doing the same thing.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's part of a if/else cascade - we only need to add the TypeNonGCStaticsSymbol if there's a class constructor, or the field is non-GC static. There's no quick way to check if a field is non-GC static - one has to ask the "is it RVA/ThreadStatic/GCStatic" questions first.

The best we could do here is change this else block to needsNonGcStaticBase = true; and delete the dependencies.Add line but then the reason string will be wrong.

// so for enums also include their MethodTable.
dependencies.Add(factory.MaximallyConstructableType(_type), "Reflectable enum");

// Enums are not useful without their literal fields

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the comment, but my question is why do we need to explicitly preserve the metadata for the fields - I would expect that if the app uses them we would see their usage directly. Or is this because annotations/suppressions in framework assume this? (If so I think it's worth a comment about that).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields are never referenced from code. Basically, an enum looks like this:

classMyEnum:Enum{privateint_value;publicconstintValue1=1;publicconstintValue2=2;}

One cannot do ldsfld MyEnum.Value1 in IL. It's lowered into ldc.i4.1.

Added comment.

Comment on lines +124 to +125
var reflectableFieldNode = obj as ReflectableFieldNode;
if (reflectableFieldNode != null)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is based on the existing code, but it would look "nicer" like this:

if(objisReflectableFieldNodereflectableFieldNode)

Maybe even change the whole thing into a switch?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to follow the existing style in the file and don't change surrounding style as part of unrelated changes. It makes history/git blame tidier. We could reformat in a single pass if it's a problem.

Comment on lines +419 to +421
// Tiny optimization: no get/set for literal fields
if (field.IsLiteral)
continue;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So literal fields have special handling in reflection code? Meaning they don't need any runtime support, it's all done via metadata, right?
(and we assume that since it's literal its type is primitive/string and thus we don't need to explicitly preserve its type either).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literal fields (const/enum values) only exist in metadata. They don't have a "value" that is accessible through IL. Reflection GetValue reads the value from the metadata instead of reading some location in memory (like we do for other fields). There's no location in memory to worry about.

@MichalStrehovsky
MichalStrehovsky merged commit f54c15a into dotnet:mainJun 13, 2022
@MichalStrehovsky
MichalStrehovsky deleted the fieldReflection branch June 13, 2022 01:42
@ghostghost locked as resolved and limited conversation to collaborators Jul 13, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@MichalStrehovsky@vitek-karas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Precisely track reflected on fields - #70546

Merged
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection
Jun 13, 2022
Merged

Precisely track reflected on fields#70546
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection

Conversation

@MichalStrehovsky

Copy link
Copy Markdown
Member

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.

With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.

We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided Class<int>.Foo should be reflection accessible and Class<double> is used elsewhere in the program, Class<double>.Foo is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.

I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter in the past.

Added a couple new tests that test the various invariants.

This saves 2% in size on a dotnet new webapi template project with IlcTrimMetadata on. It is a small regression with IlcTrimMetadata off because we actually now track more things for the reflectable field: previously we would not make sure the field is reflection-accessible at runtime. We need a TypeHandle for the field type for it to be usable. As a potential future optimization, we could look into allowing reflection to see "unconstructed" TypeHandles (and use that one). Reflection is currently not allowed to see unconstructed TypeHandles to prevent people falling into a RuntimeHelpers.AllocateUninitializedObject(someField.FieldType.TypeHandle) trap that has a bad failure mode right now. Once we fix the failure mode, we could potentially allow it.

Cc @dotnet/ilc-contrib

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.
With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.
We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided `Class<int>.Foo` should be reflection accessible and `Class<double>` is used elsewhere in the program, `Class<double>.Foo` is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.
I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter.
Added a couple new tests that test the various invariants.

@vitek-karasvitek-karas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - but I don't know enough of the runtime side to tell if this will keep enough info around for the runtime to work correctly (I mean the tests work, but for a review).

Comment on lines +74 to +78
else
{
dependencies.Add(factory.TypeNonGCStaticsSymbol((MetadataType)_field.OwningType), "NonGC static base of a reflectable field");
needsNonGcStaticBase = false;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just remove this branch and let the if immediately below do this? It seems to be doing the same thing.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's part of a if/else cascade - we only need to add the TypeNonGCStaticsSymbol if there's a class constructor, or the field is non-GC static. There's no quick way to check if a field is non-GC static - one has to ask the "is it RVA/ThreadStatic/GCStatic" questions first.

The best we could do here is change this else block to needsNonGcStaticBase = true; and delete the dependencies.Add line but then the reason string will be wrong.

// so for enums also include their MethodTable.
dependencies.Add(factory.MaximallyConstructableType(_type), "Reflectable enum");

// Enums are not useful without their literal fields

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the comment, but my question is why do we need to explicitly preserve the metadata for the fields - I would expect that if the app uses them we would see their usage directly. Or is this because annotations/suppressions in framework assume this? (If so I think it's worth a comment about that).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields are never referenced from code. Basically, an enum looks like this:

classMyEnum:Enum{privateint_value;publicconstintValue1=1;publicconstintValue2=2;}

One cannot do ldsfld MyEnum.Value1 in IL. It's lowered into ldc.i4.1.

Added comment.

Comment on lines +124 to +125
var reflectableFieldNode = obj as ReflectableFieldNode;
if (reflectableFieldNode != null)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is based on the existing code, but it would look "nicer" like this:

if(objisReflectableFieldNodereflectableFieldNode)

Maybe even change the whole thing into a switch?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to follow the existing style in the file and don't change surrounding style as part of unrelated changes. It makes history/git blame tidier. We could reformat in a single pass if it's a problem.

Comment on lines +419 to +421
// Tiny optimization: no get/set for literal fields
if (field.IsLiteral)
continue;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So literal fields have special handling in reflection code? Meaning they don't need any runtime support, it's all done via metadata, right?
(and we assume that since it's literal its type is primitive/string and thus we don't need to explicitly preserve its type either).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literal fields (const/enum values) only exist in metadata. They don't have a "value" that is accessible through IL. Reflection GetValue reads the value from the metadata instead of reading some location in memory (like we do for other fields). There's no location in memory to worry about.

@MichalStrehovsky
MichalStrehovsky merged commit f54c15a into dotnet:mainJun 13, 2022
@MichalStrehovsky
MichalStrehovsky deleted the fieldReflection branch June 13, 2022 01:42
@ghostghost locked as resolved and limited conversation to collaborators Jul 13, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@MichalStrehovsky@vitek-karas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Precisely track reflected on fields - #70546

Merged
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection
Jun 13, 2022
Merged

Precisely track reflected on fields#70546
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection

Conversation

@MichalStrehovsky

Copy link
Copy Markdown
Member

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.

With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.

We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided Class<int>.Foo should be reflection accessible and Class<double> is used elsewhere in the program, Class<double>.Foo is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.

I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter in the past.

Added a couple new tests that test the various invariants.

This saves 2% in size on a dotnet new webapi template project with IlcTrimMetadata on. It is a small regression with IlcTrimMetadata off because we actually now track more things for the reflectable field: previously we would not make sure the field is reflection-accessible at runtime. We need a TypeHandle for the field type for it to be usable. As a potential future optimization, we could look into allowing reflection to see "unconstructed" TypeHandles (and use that one). Reflection is currently not allowed to see unconstructed TypeHandles to prevent people falling into a RuntimeHelpers.AllocateUninitializedObject(someField.FieldType.TypeHandle) trap that has a bad failure mode right now. Once we fix the failure mode, we could potentially allow it.

Cc @dotnet/ilc-contrib

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.
With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.
We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided `Class<int>.Foo` should be reflection accessible and `Class<double>` is used elsewhere in the program, `Class<double>.Foo` is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.
I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter.
Added a couple new tests that test the various invariants.

@vitek-karasvitek-karas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - but I don't know enough of the runtime side to tell if this will keep enough info around for the runtime to work correctly (I mean the tests work, but for a review).

Comment on lines +74 to +78
else
{
dependencies.Add(factory.TypeNonGCStaticsSymbol((MetadataType)_field.OwningType), "NonGC static base of a reflectable field");
needsNonGcStaticBase = false;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just remove this branch and let the if immediately below do this? It seems to be doing the same thing.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's part of a if/else cascade - we only need to add the TypeNonGCStaticsSymbol if there's a class constructor, or the field is non-GC static. There's no quick way to check if a field is non-GC static - one has to ask the "is it RVA/ThreadStatic/GCStatic" questions first.

The best we could do here is change this else block to needsNonGcStaticBase = true; and delete the dependencies.Add line but then the reason string will be wrong.

// so for enums also include their MethodTable.
dependencies.Add(factory.MaximallyConstructableType(_type), "Reflectable enum");

// Enums are not useful without their literal fields

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the comment, but my question is why do we need to explicitly preserve the metadata for the fields - I would expect that if the app uses them we would see their usage directly. Or is this because annotations/suppressions in framework assume this? (If so I think it's worth a comment about that).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields are never referenced from code. Basically, an enum looks like this:

classMyEnum:Enum{privateint_value;publicconstintValue1=1;publicconstintValue2=2;}

One cannot do ldsfld MyEnum.Value1 in IL. It's lowered into ldc.i4.1.

Added comment.

Comment on lines +124 to +125
var reflectableFieldNode = obj as ReflectableFieldNode;
if (reflectableFieldNode != null)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is based on the existing code, but it would look "nicer" like this:

if(objisReflectableFieldNodereflectableFieldNode)

Maybe even change the whole thing into a switch?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to follow the existing style in the file and don't change surrounding style as part of unrelated changes. It makes history/git blame tidier. We could reformat in a single pass if it's a problem.

Comment on lines +419 to +421
// Tiny optimization: no get/set for literal fields
if (field.IsLiteral)
continue;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So literal fields have special handling in reflection code? Meaning they don't need any runtime support, it's all done via metadata, right?
(and we assume that since it's literal its type is primitive/string and thus we don't need to explicitly preserve its type either).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literal fields (const/enum values) only exist in metadata. They don't have a "value" that is accessible through IL. Reflection GetValue reads the value from the metadata instead of reading some location in memory (like we do for other fields). There's no location in memory to worry about.

@MichalStrehovsky
MichalStrehovsky merged commit f54c15a into dotnet:mainJun 13, 2022
@MichalStrehovsky
MichalStrehovsky deleted the fieldReflection branch June 13, 2022 01:42
@ghostghost locked as resolved and limited conversation to collaborators Jul 13, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@MichalStrehovsky@vitek-karas
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Precisely track reflected on fields - #70546

Merged
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection
Jun 13, 2022
Merged

Precisely track reflected on fields#70546
MichalStrehovsky merged 2 commits into
dotnet:mainfrom
MichalStrehovsky:fieldReflection

Conversation

@MichalStrehovsky

Copy link
Copy Markdown
Member

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.

With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.

We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided Class<int>.Foo should be reflection accessible and Class<double> is used elsewhere in the program, Class<double>.Foo is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.

I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter in the past.

Added a couple new tests that test the various invariants.

This saves 2% in size on a dotnet new webapi template project with IlcTrimMetadata on. It is a small regression with IlcTrimMetadata off because we actually now track more things for the reflectable field: previously we would not make sure the field is reflection-accessible at runtime. We need a TypeHandle for the field type for it to be usable. As a potential future optimization, we could look into allowing reflection to see "unconstructed" TypeHandles (and use that one). Reflection is currently not allowed to see unconstructed TypeHandles to prevent people falling into a RuntimeHelpers.AllocateUninitializedObject(someField.FieldType.TypeHandle) trap that has a bad failure mode right now. Once we fix the failure mode, we could potentially allow it.

Cc @dotnet/ilc-contrib

The AOT compiler used simple logic to decide what fields should be kept/generated: if a type is considered constructed, generate all fields. This works, but it's not very efficient.
With this change I'm introducing precise tracking of each field that should be accessible from reflection at runtime. The fields are represented by new nodes within the dependency graph.
We track fields on individual generic instantiations and ensure we end up in a consistent state where if e.g. we decided `Class<int>.Foo` should be reflection accessible and `Class<double>` is used elsewhere in the program, `Class<double>.Foo` is also reflection accessible. This matches how IL Linker thinks about reflectability where genericness doesn't matter. We could be more optimal here, but various suppressions in framework rely on this logic. Additional reflectable fields only cost tens of bytes.
I had to update various places within the compiler that didn't bother specifying field dependencies because they didn't matter.
Added a couple new tests that test the various invariants.

@vitek-karasvitek-karas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - but I don't know enough of the runtime side to tell if this will keep enough info around for the runtime to work correctly (I mean the tests work, but for a review).

Comment on lines +74 to +78
else
{
dependencies.Add(factory.TypeNonGCStaticsSymbol((MetadataType)_field.OwningType), "NonGC static base of a reflectable field");
needsNonGcStaticBase = false;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just remove this branch and let the if immediately below do this? It seems to be doing the same thing.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's part of a if/else cascade - we only need to add the TypeNonGCStaticsSymbol if there's a class constructor, or the field is non-GC static. There's no quick way to check if a field is non-GC static - one has to ask the "is it RVA/ThreadStatic/GCStatic" questions first.

The best we could do here is change this else block to needsNonGcStaticBase = true; and delete the dependencies.Add line but then the reason string will be wrong.

// so for enums also include their MethodTable.
dependencies.Add(factory.MaximallyConstructableType(_type), "Reflectable enum");

// Enums are not useful without their literal fields

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the comment, but my question is why do we need to explicitly preserve the metadata for the fields - I would expect that if the app uses them we would see their usage directly. Or is this because annotations/suppressions in framework assume this? (If so I think it's worth a comment about that).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields are never referenced from code. Basically, an enum looks like this:

classMyEnum:Enum{privateint_value;publicconstintValue1=1;publicconstintValue2=2;}

One cannot do ldsfld MyEnum.Value1 in IL. It's lowered into ldc.i4.1.

Added comment.

Comment on lines +124 to +125
var reflectableFieldNode = obj as ReflectableFieldNode;
if (reflectableFieldNode != null)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is based on the existing code, but it would look "nicer" like this:

if(objisReflectableFieldNodereflectableFieldNode)

Maybe even change the whole thing into a switch?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to follow the existing style in the file and don't change surrounding style as part of unrelated changes. It makes history/git blame tidier. We could reformat in a single pass if it's a problem.

Comment on lines +419 to +421
// Tiny optimization: no get/set for literal fields
if (field.IsLiteral)
continue;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So literal fields have special handling in reflection code? Meaning they don't need any runtime support, it's all done via metadata, right?
(and we assume that since it's literal its type is primitive/string and thus we don't need to explicitly preserve its type either).

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literal fields (const/enum values) only exist in metadata. They don't have a "value" that is accessible through IL. Reflection GetValue reads the value from the metadata instead of reading some location in memory (like we do for other fields). There's no location in memory to worry about.

@MichalStrehovsky
MichalStrehovsky merged commit f54c15a into dotnet:mainJun 13, 2022
@MichalStrehovsky
MichalStrehovsky deleted the fieldReflection branch June 13, 2022 01:42
@ghostghost locked as resolved and limited conversation to collaborators Jul 13, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@MichalStrehovsky@vitek-karas