Run trimming tests as AOT tests

[MichalStrehovsky]

Not everything is passing, so I baselined this. Some we'll probably exclude permanently, others are more concerning and we need to determine if it's test issues or product issues.

Cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[eerhardt]

Should we just get rid of the NativeAotTests.proj extension (there is only 1 currently in the repo) and instead just have TrimmingTests.proj, which are run as both PublishTrimmed and PublishAot?

[MichalStrehovsky]

If you don't have concerns about running the NativeAotTests with trimming as well, that sounds good to me too.

Trimming is generally a subset of AOT.

[eerhardt]

This is the only NativeAotTests today:

runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/NativeAotTests/DiagnosticSourceEventSourceTests.cs

Lines 45 to 84 in 4760cf0

	publicstaticintMain()
	{
	DiagnosticSourcediagnosticSource=newDiagnosticListener("TestDiagnosticListener");
	using(varlistener=newTestEventListener())
	{
	vardata=newEventData()
	{
	Id=Guid.NewGuid(),
	};
	Write(diagnosticSource,"Test.Start",data);
	if(!(listener.LogDataPayload?.Count==3&&
	(string)listener.LogDataPayload[0]=="TestDiagnosticListener"&&
	(string)listener.LogDataPayload[1]=="Test.Start"))
	{
	return-1;
	}
	object[]args=(object[])listener.LogDataPayload[2];
	if(args.Length!=2)
	{
	return-2;
	}
	IDictionary<string,object>arg=(IDictionary<string,object>)args[0];
	if(!((string)arg["Key"]=="Id"&&(string)arg["Value"]==data.Id.ToString()))
	{
	return-3;
	}
	arg=(IDictionary<string,object>)args[1];
	if(!((string)arg["Key"]=="*Enumerate"&&(string)arg["Value"]=="1,2,3"))
	{
	return-4;
	}
	return100;
	}
	}

It should work just fine with PublishTrimmed. It was written to test the MakeGenericType deep inside of DiagnosticSourceEventSource works with NativeAOT.

I think it would be best to just have 1 "kind of test" and it runs for both PublishTrimmed and PublishAot. It will make it easier to write these, and it will add a bit of coverage to both scenarios.

[eerhardt]

Why is this change necessary?

[MichalStrehovsky]

The tests weren't building because they couldn't find AppHost. This is running the Publish target directly, which is very problematic because it bypasses logic in the SDK. We have been having a conversation with the SDK team for months. The fix is to do what real publish does and that's setting _IsPublishing. We do that in multiple places in this repo even though it's used/defined by the SDK.

[eerhardt]

LGTM. Thank you for making this happen.

[eerhardt]

(nit) this comment doesn't align with the code. The comment says one trumps the other. The code says if they are both true it is an error.

[MichalStrehovsky]

/ba-g slow mac

[MichalStrehovsky]

Not everything is passing, so I baselined this. Some we'll probably exclude permanently, others are more concerning and we need to determine if it's test issues or product issues.

Cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[eerhardt]

Should we just get rid of the NativeAotTests.proj extension (there is only 1 currently in the repo) and instead just have TrimmingTests.proj, which are run as both PublishTrimmed and PublishAot?

[MichalStrehovsky]

If you don't have concerns about running the NativeAotTests with trimming as well, that sounds good to me too.

Trimming is generally a subset of AOT.

[eerhardt]

This is the only NativeAotTests today:

runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/NativeAotTests/DiagnosticSourceEventSourceTests.cs

Lines 45 to 84 in 4760cf0

	publicstaticintMain()
	{
	DiagnosticSourcediagnosticSource=newDiagnosticListener("TestDiagnosticListener");
	using(varlistener=newTestEventListener())
	{
	vardata=newEventData()
	{
	Id=Guid.NewGuid(),
	};
	Write(diagnosticSource,"Test.Start",data);
	if(!(listener.LogDataPayload?.Count==3&&
	(string)listener.LogDataPayload[0]=="TestDiagnosticListener"&&
	(string)listener.LogDataPayload[1]=="Test.Start"))
	{
	return-1;
	}
	object[]args=(object[])listener.LogDataPayload[2];
	if(args.Length!=2)
	{
	return-2;
	}
	IDictionary<string,object>arg=(IDictionary<string,object>)args[0];
	if(!((string)arg["Key"]=="Id"&&(string)arg["Value"]==data.Id.ToString()))
	{
	return-3;
	}
	arg=(IDictionary<string,object>)args[1];
	if(!((string)arg["Key"]=="*Enumerate"&&(string)arg["Value"]=="1,2,3"))
	{
	return-4;
	}
	return100;
	}
	}

It should work just fine with PublishTrimmed. It was written to test the MakeGenericType deep inside of DiagnosticSourceEventSource works with NativeAOT.

I think it would be best to just have 1 "kind of test" and it runs for both PublishTrimmed and PublishAot. It will make it easier to write these, and it will add a bit of coverage to both scenarios.

[eerhardt]

Why is this change necessary?

[MichalStrehovsky]

The tests weren't building because they couldn't find AppHost. This is running the Publish target directly, which is very problematic because it bypasses logic in the SDK. We have been having a conversation with the SDK team for months. The fix is to do what real publish does and that's setting _IsPublishing. We do that in multiple places in this repo even though it's used/defined by the SDK.

[eerhardt]

LGTM. Thank you for making this happen.

[eerhardt]

(nit) this comment doesn't align with the code. The comment says one trumps the other. The code says if they are both true it is an error.

[MichalStrehovsky]

/ba-g slow mac

[MichalStrehovsky]

Not everything is passing, so I baselined this. Some we'll probably exclude permanently, others are more concerning and we need to determine if it's test issues or product issues.

Cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[eerhardt]

Should we just get rid of the NativeAotTests.proj extension (there is only 1 currently in the repo) and instead just have TrimmingTests.proj, which are run as both PublishTrimmed and PublishAot?

[MichalStrehovsky]

If you don't have concerns about running the NativeAotTests with trimming as well, that sounds good to me too.

Trimming is generally a subset of AOT.

[eerhardt]

This is the only NativeAotTests today:

runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/NativeAotTests/DiagnosticSourceEventSourceTests.cs

Lines 45 to 84 in 4760cf0

	publicstaticintMain()
	{
	DiagnosticSourcediagnosticSource=newDiagnosticListener("TestDiagnosticListener");
	using(varlistener=newTestEventListener())
	{
	vardata=newEventData()
	{
	Id=Guid.NewGuid(),
	};
	Write(diagnosticSource,"Test.Start",data);
	if(!(listener.LogDataPayload?.Count==3&&
	(string)listener.LogDataPayload[0]=="TestDiagnosticListener"&&
	(string)listener.LogDataPayload[1]=="Test.Start"))
	{
	return-1;
	}
	object[]args=(object[])listener.LogDataPayload[2];
	if(args.Length!=2)
	{
	return-2;
	}
	IDictionary<string,object>arg=(IDictionary<string,object>)args[0];
	if(!((string)arg["Key"]=="Id"&&(string)arg["Value"]==data.Id.ToString()))
	{
	return-3;
	}
	arg=(IDictionary<string,object>)args[1];
	if(!((string)arg["Key"]=="*Enumerate"&&(string)arg["Value"]=="1,2,3"))
	{
	return-4;
	}
	return100;
	}
	}

It should work just fine with PublishTrimmed. It was written to test the MakeGenericType deep inside of DiagnosticSourceEventSource works with NativeAOT.

I think it would be best to just have 1 "kind of test" and it runs for both PublishTrimmed and PublishAot. It will make it easier to write these, and it will add a bit of coverage to both scenarios.

[eerhardt]

Why is this change necessary?

[MichalStrehovsky]

The tests weren't building because they couldn't find AppHost. This is running the Publish target directly, which is very problematic because it bypasses logic in the SDK. We have been having a conversation with the SDK team for months. The fix is to do what real publish does and that's setting _IsPublishing. We do that in multiple places in this repo even though it's used/defined by the SDK.

[eerhardt]

LGTM. Thank you for making this happen.

[eerhardt]

(nit) this comment doesn't align with the code. The comment says one trumps the other. The code says if they are both true it is an error.

[MichalStrehovsky]

/ba-g slow mac

[MichalStrehovsky]

Not everything is passing, so I baselined this. Some we'll probably exclude permanently, others are more concerning and we need to determine if it's test issues or product issues.

Cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[eerhardt]

Should we just get rid of the NativeAotTests.proj extension (there is only 1 currently in the repo) and instead just have TrimmingTests.proj, which are run as both PublishTrimmed and PublishAot?

[MichalStrehovsky]

If you don't have concerns about running the NativeAotTests with trimming as well, that sounds good to me too.

Trimming is generally a subset of AOT.

[eerhardt]

This is the only NativeAotTests today:

runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/NativeAotTests/DiagnosticSourceEventSourceTests.cs

Lines 45 to 84 in 4760cf0

	publicstaticintMain()
	{
	DiagnosticSourcediagnosticSource=newDiagnosticListener("TestDiagnosticListener");
	using(varlistener=newTestEventListener())
	{
	vardata=newEventData()
	{
	Id=Guid.NewGuid(),
	};
	Write(diagnosticSource,"Test.Start",data);
	if(!(listener.LogDataPayload?.Count==3&&
	(string)listener.LogDataPayload[0]=="TestDiagnosticListener"&&
	(string)listener.LogDataPayload[1]=="Test.Start"))
	{
	return-1;
	}
	object[]args=(object[])listener.LogDataPayload[2];
	if(args.Length!=2)
	{
	return-2;
	}
	IDictionary<string,object>arg=(IDictionary<string,object>)args[0];
	if(!((string)arg["Key"]=="Id"&&(string)arg["Value"]==data.Id.ToString()))
	{
	return-3;
	}
	arg=(IDictionary<string,object>)args[1];
	if(!((string)arg["Key"]=="*Enumerate"&&(string)arg["Value"]=="1,2,3"))
	{
	return-4;
	}
	return100;
	}
	}

It should work just fine with PublishTrimmed. It was written to test the MakeGenericType deep inside of DiagnosticSourceEventSource works with NativeAOT.

I think it would be best to just have 1 "kind of test" and it runs for both PublishTrimmed and PublishAot. It will make it easier to write these, and it will add a bit of coverage to both scenarios.

[eerhardt]

Why is this change necessary?

[MichalStrehovsky]

The tests weren't building because they couldn't find AppHost. This is running the Publish target directly, which is very problematic because it bypasses logic in the SDK. We have been having a conversation with the SDK team for months. The fix is to do what real publish does and that's setting _IsPublishing. We do that in multiple places in this repo even though it's used/defined by the SDK.

[eerhardt]

LGTM. Thank you for making this happen.

[eerhardt]

(nit) this comment doesn't align with the code. The comment says one trumps the other. The code says if they are both true it is an error.

[MichalStrehovsky]

/ba-g slow mac

[MichalStrehovsky]

Not everything is passing, so I baselined this. Some we'll probably exclude permanently, others are more concerning and we need to determine if it's test issues or product issues.

Cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[eerhardt]

Should we just get rid of the NativeAotTests.proj extension (there is only 1 currently in the repo) and instead just have TrimmingTests.proj, which are run as both PublishTrimmed and PublishAot?

[MichalStrehovsky]

If you don't have concerns about running the NativeAotTests with trimming as well, that sounds good to me too.

Trimming is generally a subset of AOT.

[eerhardt]

This is the only NativeAotTests today:

runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/NativeAotTests/DiagnosticSourceEventSourceTests.cs

Lines 45 to 84 in 4760cf0

	publicstaticintMain()
	{
	DiagnosticSourcediagnosticSource=newDiagnosticListener("TestDiagnosticListener");
	using(varlistener=newTestEventListener())
	{
	vardata=newEventData()
	{
	Id=Guid.NewGuid(),
	};
	Write(diagnosticSource,"Test.Start",data);
	if(!(listener.LogDataPayload?.Count==3&&
	(string)listener.LogDataPayload[0]=="TestDiagnosticListener"&&
	(string)listener.LogDataPayload[1]=="Test.Start"))
	{
	return-1;
	}
	object[]args=(object[])listener.LogDataPayload[2];
	if(args.Length!=2)
	{
	return-2;
	}
	IDictionary<string,object>arg=(IDictionary<string,object>)args[0];
	if(!((string)arg["Key"]=="Id"&&(string)arg["Value"]==data.Id.ToString()))
	{
	return-3;
	}
	arg=(IDictionary<string,object>)args[1];
	if(!((string)arg["Key"]=="*Enumerate"&&(string)arg["Value"]=="1,2,3"))
	{
	return-4;
	}
	return100;
	}
	}

It should work just fine with PublishTrimmed. It was written to test the MakeGenericType deep inside of DiagnosticSourceEventSource works with NativeAOT.

I think it would be best to just have 1 "kind of test" and it runs for both PublishTrimmed and PublishAot. It will make it easier to write these, and it will add a bit of coverage to both scenarios.

[eerhardt]

Why is this change necessary?

[MichalStrehovsky]

The tests weren't building because they couldn't find AppHost. This is running the Publish target directly, which is very problematic because it bypasses logic in the SDK. We have been having a conversation with the SDK team for months. The fix is to do what real publish does and that's setting _IsPublishing. We do that in multiple places in this repo even though it's used/defined by the SDK.

[eerhardt]

LGTM. Thank you for making this happen.

[eerhardt]

(nit) this comment doesn't align with the code. The comment says one trumps the other. The code says if they are both true it is an error.

[MichalStrehovsky]

/ba-g slow mac

[MichalStrehovsky]

Not everything is passing, so I baselined this. Some we'll probably exclude permanently, others are more concerning and we need to determine if it's test issues or product issues.

Cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[eerhardt]

Should we just get rid of the NativeAotTests.proj extension (there is only 1 currently in the repo) and instead just have TrimmingTests.proj, which are run as both PublishTrimmed and PublishAot?

[MichalStrehovsky]

If you don't have concerns about running the NativeAotTests with trimming as well, that sounds good to me too.

Trimming is generally a subset of AOT.

[eerhardt]

This is the only NativeAotTests today:

runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/NativeAotTests/DiagnosticSourceEventSourceTests.cs

Lines 45 to 84 in 4760cf0

	publicstaticintMain()
	{
	DiagnosticSourcediagnosticSource=newDiagnosticListener("TestDiagnosticListener");
	using(varlistener=newTestEventListener())
	{
	vardata=newEventData()
	{
	Id=Guid.NewGuid(),
	};
	Write(diagnosticSource,"Test.Start",data);
	if(!(listener.LogDataPayload?.Count==3&&
	(string)listener.LogDataPayload[0]=="TestDiagnosticListener"&&
	(string)listener.LogDataPayload[1]=="Test.Start"))
	{
	return-1;
	}
	object[]args=(object[])listener.LogDataPayload[2];
	if(args.Length!=2)
	{
	return-2;
	}
	IDictionary<string,object>arg=(IDictionary<string,object>)args[0];
	if(!((string)arg["Key"]=="Id"&&(string)arg["Value"]==data.Id.ToString()))
	{
	return-3;
	}
	arg=(IDictionary<string,object>)args[1];
	if(!((string)arg["Key"]=="*Enumerate"&&(string)arg["Value"]=="1,2,3"))
	{
	return-4;
	}
	return100;
	}
	}

It should work just fine with PublishTrimmed. It was written to test the MakeGenericType deep inside of DiagnosticSourceEventSource works with NativeAOT.

I think it would be best to just have 1 "kind of test" and it runs for both PublishTrimmed and PublishAot. It will make it easier to write these, and it will add a bit of coverage to both scenarios.

[eerhardt]

Why is this change necessary?

[MichalStrehovsky]

The tests weren't building because they couldn't find AppHost. This is running the Publish target directly, which is very problematic because it bypasses logic in the SDK. We have been having a conversation with the SDK team for months. The fix is to do what real publish does and that's setting _IsPublishing. We do that in multiple places in this repo even though it's used/defined by the SDK.

[eerhardt]

LGTM. Thank you for making this happen.

[eerhardt]

(nit) this comment doesn't align with the code. The comment says one trumps the other. The code says if they are both true it is an error.

[MichalStrehovsky]

/ba-g slow mac

[MichalStrehovsky]

Not everything is passing, so I baselined this. Some we'll probably exclude permanently, others are more concerning and we need to determine if it's test issues or product issues.

Cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[eerhardt]

Should we just get rid of the NativeAotTests.proj extension (there is only 1 currently in the repo) and instead just have TrimmingTests.proj, which are run as both PublishTrimmed and PublishAot?

[MichalStrehovsky]

If you don't have concerns about running the NativeAotTests with trimming as well, that sounds good to me too.

Trimming is generally a subset of AOT.

[eerhardt]

This is the only NativeAotTests today:

runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/NativeAotTests/DiagnosticSourceEventSourceTests.cs

Lines 45 to 84 in 4760cf0

	publicstaticintMain()
	{
	DiagnosticSourcediagnosticSource=newDiagnosticListener("TestDiagnosticListener");
	using(varlistener=newTestEventListener())
	{
	vardata=newEventData()
	{
	Id=Guid.NewGuid(),
	};
	Write(diagnosticSource,"Test.Start",data);
	if(!(listener.LogDataPayload?.Count==3&&
	(string)listener.LogDataPayload[0]=="TestDiagnosticListener"&&
	(string)listener.LogDataPayload[1]=="Test.Start"))
	{
	return-1;
	}
	object[]args=(object[])listener.LogDataPayload[2];
	if(args.Length!=2)
	{
	return-2;
	}
	IDictionary<string,object>arg=(IDictionary<string,object>)args[0];
	if(!((string)arg["Key"]=="Id"&&(string)arg["Value"]==data.Id.ToString()))
	{
	return-3;
	}
	arg=(IDictionary<string,object>)args[1];
	if(!((string)arg["Key"]=="*Enumerate"&&(string)arg["Value"]=="1,2,3"))
	{
	return-4;
	}
	return100;
	}
	}

It should work just fine with PublishTrimmed. It was written to test the MakeGenericType deep inside of DiagnosticSourceEventSource works with NativeAOT.

I think it would be best to just have 1 "kind of test" and it runs for both PublishTrimmed and PublishAot. It will make it easier to write these, and it will add a bit of coverage to both scenarios.

[eerhardt]

Why is this change necessary?

[MichalStrehovsky]

The tests weren't building because they couldn't find AppHost. This is running the Publish target directly, which is very problematic because it bypasses logic in the SDK. We have been having a conversation with the SDK team for months. The fix is to do what real publish does and that's setting _IsPublishing. We do that in multiple places in this repo even though it's used/defined by the SDK.

[eerhardt]

LGTM. Thank you for making this happen.

[eerhardt]

(nit) this comment doesn't align with the code. The comment says one trumps the other. The code says if they are both true it is an error.

[MichalStrehovsky]

/ba-g slow mac

[MichalStrehovsky]

Not everything is passing, so I baselined this. Some we'll probably exclude permanently, others are more concerning and we need to determine if it's test issues or product issues.

Cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

/azp run runtime-nativeaot-outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[eerhardt]

Should we just get rid of the NativeAotTests.proj extension (there is only 1 currently in the repo) and instead just have TrimmingTests.proj, which are run as both PublishTrimmed and PublishAot?

[MichalStrehovsky]

If you don't have concerns about running the NativeAotTests with trimming as well, that sounds good to me too.

Trimming is generally a subset of AOT.

[eerhardt]

This is the only NativeAotTests today:

runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/NativeAotTests/DiagnosticSourceEventSourceTests.cs

Lines 45 to 84 in 4760cf0

	publicstaticintMain()
	{
	DiagnosticSourcediagnosticSource=newDiagnosticListener("TestDiagnosticListener");
	using(varlistener=newTestEventListener())
	{
	vardata=newEventData()
	{
	Id=Guid.NewGuid(),
	};
	Write(diagnosticSource,"Test.Start",data);
	if(!(listener.LogDataPayload?.Count==3&&
	(string)listener.LogDataPayload[0]=="TestDiagnosticListener"&&
	(string)listener.LogDataPayload[1]=="Test.Start"))
	{
	return-1;
	}
	object[]args=(object[])listener.LogDataPayload[2];
	if(args.Length!=2)
	{
	return-2;
	}
	IDictionary<string,object>arg=(IDictionary<string,object>)args[0];
	if(!((string)arg["Key"]=="Id"&&(string)arg["Value"]==data.Id.ToString()))
	{
	return-3;
	}
	arg=(IDictionary<string,object>)args[1];
	if(!((string)arg["Key"]=="*Enumerate"&&(string)arg["Value"]=="1,2,3"))
	{
	return-4;
	}
	return100;
	}
	}

It should work just fine with PublishTrimmed. It was written to test the MakeGenericType deep inside of DiagnosticSourceEventSource works with NativeAOT.

I think it would be best to just have 1 "kind of test" and it runs for both PublishTrimmed and PublishAot. It will make it easier to write these, and it will add a bit of coverage to both scenarios.

[eerhardt]

Why is this change necessary?

[MichalStrehovsky]

The tests weren't building because they couldn't find AppHost. This is running the Publish target directly, which is very problematic because it bypasses logic in the SDK. We have been having a conversation with the SDK team for months. The fix is to do what real publish does and that's setting _IsPublishing. We do that in multiple places in this repo even though it's used/defined by the SDK.

[eerhardt]

LGTM. Thank you for making this happen.

[eerhardt]

(nit) this comment doesn't align with the code. The comment says one trumps the other. The code says if they are both true it is an error.

[MichalStrehovsky]

/ba-g slow mac