Handle MakeGeneric APIs in dataflow by MichalStrehovsky · Pull Request #99037 · dotnet/runtime

MichalStrehovsky · 2024-02-28T11:21:20Z

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Contributes to dotnet#81204. With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code. The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need. If we know what the values are, we elide generating a warning. The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this: * Re-run dataflow analysis on instantiated code. I didn't feel great about doing that. * Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy. * Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works. I went with option 3. I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.

ghost · 2024-02-28T11:21:25Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Author:	MichalStrehovsky
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

MichalStrehovsky · 2024-03-08T07:10:59Z

@yowl @SingleAccretion this will need a matching change in nativeAOT-LLVM - just match the one-line diff in ScannedMethodNode/MethodCodeNode in the node that represents compiled method bodies.

The failure mode you'll see without the matching change is runtime failure in the new tests that do MakeGenericType/Method.

If you wanted to, you could just proactively make the change now, I don't think it'll hurt anything.

MichalStrehovsky · 2024-02-28T11:21:20Z

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Contributes to dotnet#81204. With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code. The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need. If we know what the values are, we elide generating a warning. The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this: * Re-run dataflow analysis on instantiated code. I didn't feel great about doing that. * Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy. * Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works. I went with option 3. I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.

ghost · 2024-02-28T11:21:25Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Author:	MichalStrehovsky
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

MichalStrehovsky · 2024-03-08T07:10:59Z

@yowl @SingleAccretion this will need a matching change in nativeAOT-LLVM - just match the one-line diff in ScannedMethodNode/MethodCodeNode in the node that represents compiled method bodies.

The failure mode you'll see without the matching change is runtime failure in the new tests that do MakeGenericType/Method.

If you wanted to, you could just proactively make the change now, I don't think it'll hurt anything.

MichalStrehovsky · 2024-02-28T11:21:20Z

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Contributes to dotnet#81204. With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code. The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need. If we know what the values are, we elide generating a warning. The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this: * Re-run dataflow analysis on instantiated code. I didn't feel great about doing that. * Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy. * Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works. I went with option 3. I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.

ghost · 2024-02-28T11:21:25Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Author:	MichalStrehovsky
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

MichalStrehovsky · 2024-03-08T07:10:59Z

@yowl @SingleAccretion this will need a matching change in nativeAOT-LLVM - just match the one-line diff in ScannedMethodNode/MethodCodeNode in the node that represents compiled method bodies.

The failure mode you'll see without the matching change is runtime failure in the new tests that do MakeGenericType/Method.

If you wanted to, you could just proactively make the change now, I don't think it'll hurt anything.

MichalStrehovsky · 2024-02-28T11:21:20Z

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Contributes to dotnet#81204. With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code. The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need. If we know what the values are, we elide generating a warning. The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this: * Re-run dataflow analysis on instantiated code. I didn't feel great about doing that. * Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy. * Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works. I went with option 3. I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.

ghost · 2024-02-28T11:21:25Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Author:	MichalStrehovsky
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

MichalStrehovsky · 2024-03-08T07:10:59Z

@yowl @SingleAccretion this will need a matching change in nativeAOT-LLVM - just match the one-line diff in ScannedMethodNode/MethodCodeNode in the node that represents compiled method bodies.

The failure mode you'll see without the matching change is runtime failure in the new tests that do MakeGenericType/Method.

If you wanted to, you could just proactively make the change now, I don't think it'll hurt anything.

MichalStrehovsky · 2024-02-28T11:21:20Z

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Contributes to dotnet#81204. With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code. The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need. If we know what the values are, we elide generating a warning. The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this: * Re-run dataflow analysis on instantiated code. I didn't feel great about doing that. * Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy. * Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works. I went with option 3. I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.

ghost · 2024-02-28T11:21:25Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Author:	MichalStrehovsky
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

MichalStrehovsky · 2024-03-08T07:10:59Z

@yowl @SingleAccretion this will need a matching change in nativeAOT-LLVM - just match the one-line diff in ScannedMethodNode/MethodCodeNode in the node that represents compiled method bodies.

The failure mode you'll see without the matching change is runtime failure in the new tests that do MakeGenericType/Method.

If you wanted to, you could just proactively make the change now, I don't think it'll hurt anything.

MichalStrehovsky · 2024-02-28T11:21:20Z

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Contributes to dotnet#81204. With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code. The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need. If we know what the values are, we elide generating a warning. The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this: * Re-run dataflow analysis on instantiated code. I didn't feel great about doing that. * Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy. * Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works. I went with option 3. I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.

ghost · 2024-02-28T11:21:25Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Author:	MichalStrehovsky
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

MichalStrehovsky · 2024-03-08T07:10:59Z

@yowl @SingleAccretion this will need a matching change in nativeAOT-LLVM - just match the one-line diff in ScannedMethodNode/MethodCodeNode in the node that represents compiled method bodies.

The failure mode you'll see without the matching change is runtime failure in the new tests that do MakeGenericType/Method.

If you wanted to, you could just proactively make the change now, I don't think it'll hurt anything.

MichalStrehovsky · 2024-02-28T11:21:20Z

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Contributes to dotnet#81204. With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code. The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need. If we know what the values are, we elide generating a warning. The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this: * Re-run dataflow analysis on instantiated code. I didn't feel great about doing that. * Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy. * Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works. I went with option 3. I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.

ghost · 2024-02-28T11:21:25Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Author:	MichalStrehovsky
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

MichalStrehovsky · 2024-03-08T07:10:59Z

@yowl @SingleAccretion this will need a matching change in nativeAOT-LLVM - just match the one-line diff in ScannedMethodNode/MethodCodeNode in the node that represents compiled method bodies.

The failure mode you'll see without the matching change is runtime failure in the new tests that do MakeGenericType/Method.

If you wanted to, you could just proactively make the change now, I don't think it'll hurt anything.

MichalStrehovsky · 2024-02-28T11:21:20Z

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Contributes to dotnet#81204. With this, we can handle `typeof(Foo<>).MakeGenericType(typeof(T))` (and `MakeGenericMethod`), including in shared generic code. The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need. If we know what the values are, we elide generating a warning. The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this: * Re-run dataflow analysis on instantiated code. I didn't feel great about doing that. * Introduce a new node `SpecializedDataflowAnalyzedMethodNode` that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on `DataflowAnalyzedMethodNode` being present. This would work, but to obtain a `DataflowAnalyzedMethod` we need the `MethodIL` of the definition (not the `MethodDesc`) and that started to look sketchy. * Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works. I went with option 3. I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new `DiagnosticAnalyzer` similar to the `DynamicallyAccessedMembersAnalyzer`.

ghost · 2024-02-28T11:21:25Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to #81204.

With this, we can handle typeof(Foo<>).MakeGenericType(typeof(T)) (and MakeGenericMethod), including in shared generic code.

The dataflow analysis change should be straightforward (we just look if it's a known array, with all known elements). I scoped this down to "exactly one array" and "exactly one possible value in the element" because the objective is constraint bridging code. We could extend to supporting multiple values, but I don't see a need.

If we know what the values are, we elide generating a warning.

The support for this in the rest of the compiler is a bit annoying due to generics. We only run dataflow on uninstantiated code. So we have a dependency node representing the dataflow analysis. This node obviously cannot know what are all the possible instantiations we saw, so it cannot directly report things about instantiated generic code. Three possible options to solve this:

Re-run dataflow analysis on instantiated code. I didn't feel great about doing that.
Introduce a new node SpecializedDataflowAnalyzedMethodNode that would be injected as a conditional dependency of all scanned/compiled method bodies, conditioned on DataflowAnalyzedMethodNode being present. This would work, but to obtain a DataflowAnalyzedMethod we need the MethodIL of the definition (not the MethodDesc) and that started to look sketchy.
Use dynamic dependencies. We only use these for GVM analysis right now. They give a node the ability to receive callbacks when new nodes are added to the graph. These nodes need to declare they are "interesting" to dynamic dependency analysis. It's not great, but it works.

I went with option 3.

I'm not making this resolve the above issue. We'd ideally want to implement this in the analyzer too. I had a quick look at that and it looks like we need to write a brand new DiagnosticAnalyzer similar to the DynamicallyAccessedMembersAnalyzer.

Cc @dotnet/ilc-contrib

Author:	MichalStrehovsky
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

MichalStrehovsky · 2024-03-08T07:10:59Z

@yowl @SingleAccretion this will need a matching change in nativeAOT-LLVM - just match the one-line diff in ScannedMethodNode/MethodCodeNode in the node that represents compiled method bodies.

The failure mode you'll see without the matching change is runtime failure in the new tests that do MakeGenericType/Method.

If you wanted to, you could just proactively make the change now, I don't think it'll hurt anything.

MichalStrehovsky added the area-NativeAOT-coreclr label Feb 28, 2024

MichalStrehovsky requested a review from sbomer February 28, 2024 11:21

ghost assigned MichalStrehovsky Feb 28, 2024

Didn't we disable the useless analyzer here
8794667

This was referenced Feb 28, 2024
Tracking issue for CI build timeouts #76454
Closed
slow macOS - "##[error]The job running on agent Azure Pipelines 9 ran longer than the maximum time of 60 minutes." dotnet/dnceng#1883
Open

sbomer approved these changes Feb 29, 2024
View reviewed changes

build-analysisBot mentioned this pull request Feb 29, 2024
Profiler test profiler\multiple\multiple\multiple.cmd failing #99091
Closed

MichalStrehovsky added 4 commits February 29, 2024 12:08

CR feedback
06a5105

Make sure tests still test what they should
a94db42

Update Reflection.cs
392f65b

Update ExponentialDataFlow.cs
1d7b032

MichalStrehovsky requested a review from marek-safar as a code owner February 29, 2024 21:27

MichalStrehovsky mentioned this pull request Mar 1, 2024
clang crashing on osx #99150
Closed

MichalStrehovsky merged commit 78452bf into dotnet:mainMar 6, 2024

MichalStrehovsky deleted the makegenericflow branch March 6, 2024 22:40

SingleAccretion mentioned this pull request Mar 8, 2024
[NativeAOT-LLVM] Post-merge tasks dotnet/runtimelab#2472
Open

github-actionsBot locked and limited conversation to collaborators Apr 7, 2024