WIP: Add installer build/test to the runtime.yml pipeline

[jkoritzinsky]

No description provided.

[jkoritzinsky]

@dagood this is where I've been working on integrating the installer build/test jobs into the runtime.yml pipeline. I have the (non live-live) build working. Below are the next steps I was planning:

• Use the checkout job as the only clone/checkout step and checkout from the git bundle like coreclr and libraries.
• Enable live-live consumption in the installer jobs.
• Port the usage of platform-matrix.yml for the installer jobs in runtime.yml back to installer/azure-pipelines.yml so we don't have to duplicate declaring our build pools.

[dagood]

Status (also reminder for myself where this is at):

Now the installer jobs grab the bits, but fail because Libraries doesn't upload [runtime.*.]Microsoft.Private.CoreFx.NETCoreApp.nuspec. But even if Libraries did upload it, there's more to do there... the nuspec contains absolute paths on the other build agent and would need translation. And it might not be as simple as replacing the old base dir with a new base dir--I think these paths might be to deeper project dirs like ...\runtime\artifacts\bin\System.Private.Xml.Linq\netcoreapp-Windows_NT-Debug\System.Private.Xml.Linq.dll rather than the flattish bin/runtime/{config}/{files} artifact that's uploaded.

It seems like the simplest way to solve this is to have at least the part of "local live-live" (#494 for easy click) where it picks up the Libraries bits by a convention rather than by parsing the nuspec.

@jkoritzinsky did you have something in mind to get the PR past this? Very possible it came up and I'm forgetting.

(FYI I ended up moving Installer onto a base-job template similar to CoreFX since it seemed like the most reasonable way to share the dependency and artifact download logic. In particular generating the name of the target jobs would have been nasty to maintain everywhere.)

[jkoritzinsky]

You came to the same conclusion that I did (picking up the libraries by convention). You'll want to use the "shared framework" paths. There's an MSBuild property to enable binplacing the shared framework to the right folder. I don't remember it off the top of my head, but I'll find it again tomorrow and let you know.

[dagood]

Closing in favor of #705 without "WIP" in title. Would leave it open to see CI results, but the most recent run is stalled out with agent issues.

[jkoritzinsky]

No description provided.

[jkoritzinsky]

@dagood this is where I've been working on integrating the installer build/test jobs into the runtime.yml pipeline. I have the (non live-live) build working. Below are the next steps I was planning:

• Use the checkout job as the only clone/checkout step and checkout from the git bundle like coreclr and libraries.
• Enable live-live consumption in the installer jobs.
• Port the usage of platform-matrix.yml for the installer jobs in runtime.yml back to installer/azure-pipelines.yml so we don't have to duplicate declaring our build pools.

[dagood]

Status (also reminder for myself where this is at):

Now the installer jobs grab the bits, but fail because Libraries doesn't upload [runtime.*.]Microsoft.Private.CoreFx.NETCoreApp.nuspec. But even if Libraries did upload it, there's more to do there... the nuspec contains absolute paths on the other build agent and would need translation. And it might not be as simple as replacing the old base dir with a new base dir--I think these paths might be to deeper project dirs like ...\runtime\artifacts\bin\System.Private.Xml.Linq\netcoreapp-Windows_NT-Debug\System.Private.Xml.Linq.dll rather than the flattish bin/runtime/{config}/{files} artifact that's uploaded.

It seems like the simplest way to solve this is to have at least the part of "local live-live" (#494 for easy click) where it picks up the Libraries bits by a convention rather than by parsing the nuspec.

@jkoritzinsky did you have something in mind to get the PR past this? Very possible it came up and I'm forgetting.

(FYI I ended up moving Installer onto a base-job template similar to CoreFX since it seemed like the most reasonable way to share the dependency and artifact download logic. In particular generating the name of the target jobs would have been nasty to maintain everywhere.)

[jkoritzinsky]

You came to the same conclusion that I did (picking up the libraries by convention). You'll want to use the "shared framework" paths. There's an MSBuild property to enable binplacing the shared framework to the right folder. I don't remember it off the top of my head, but I'll find it again tomorrow and let you know.

[dagood]

Closing in favor of #705 without "WIP" in title. Would leave it open to see CI results, but the most recent run is stalled out with agent issues.

[jkoritzinsky]

No description provided.

[jkoritzinsky]

@dagood this is where I've been working on integrating the installer build/test jobs into the runtime.yml pipeline. I have the (non live-live) build working. Below are the next steps I was planning:

• Use the checkout job as the only clone/checkout step and checkout from the git bundle like coreclr and libraries.
• Enable live-live consumption in the installer jobs.
• Port the usage of platform-matrix.yml for the installer jobs in runtime.yml back to installer/azure-pipelines.yml so we don't have to duplicate declaring our build pools.

[dagood]

Status (also reminder for myself where this is at):

Now the installer jobs grab the bits, but fail because Libraries doesn't upload [runtime.*.]Microsoft.Private.CoreFx.NETCoreApp.nuspec. But even if Libraries did upload it, there's more to do there... the nuspec contains absolute paths on the other build agent and would need translation. And it might not be as simple as replacing the old base dir with a new base dir--I think these paths might be to deeper project dirs like ...\runtime\artifacts\bin\System.Private.Xml.Linq\netcoreapp-Windows_NT-Debug\System.Private.Xml.Linq.dll rather than the flattish bin/runtime/{config}/{files} artifact that's uploaded.

It seems like the simplest way to solve this is to have at least the part of "local live-live" (#494 for easy click) where it picks up the Libraries bits by a convention rather than by parsing the nuspec.

@jkoritzinsky did you have something in mind to get the PR past this? Very possible it came up and I'm forgetting.

(FYI I ended up moving Installer onto a base-job template similar to CoreFX since it seemed like the most reasonable way to share the dependency and artifact download logic. In particular generating the name of the target jobs would have been nasty to maintain everywhere.)

[jkoritzinsky]

You came to the same conclusion that I did (picking up the libraries by convention). You'll want to use the "shared framework" paths. There's an MSBuild property to enable binplacing the shared framework to the right folder. I don't remember it off the top of my head, but I'll find it again tomorrow and let you know.

[dagood]

Closing in favor of #705 without "WIP" in title. Would leave it open to see CI results, but the most recent run is stalled out with agent issues.

[jkoritzinsky]

No description provided.

[jkoritzinsky]

@dagood this is where I've been working on integrating the installer build/test jobs into the runtime.yml pipeline. I have the (non live-live) build working. Below are the next steps I was planning:

• Use the checkout job as the only clone/checkout step and checkout from the git bundle like coreclr and libraries.
• Enable live-live consumption in the installer jobs.
• Port the usage of platform-matrix.yml for the installer jobs in runtime.yml back to installer/azure-pipelines.yml so we don't have to duplicate declaring our build pools.

[dagood]

Status (also reminder for myself where this is at):

Now the installer jobs grab the bits, but fail because Libraries doesn't upload [runtime.*.]Microsoft.Private.CoreFx.NETCoreApp.nuspec. But even if Libraries did upload it, there's more to do there... the nuspec contains absolute paths on the other build agent and would need translation. And it might not be as simple as replacing the old base dir with a new base dir--I think these paths might be to deeper project dirs like ...\runtime\artifacts\bin\System.Private.Xml.Linq\netcoreapp-Windows_NT-Debug\System.Private.Xml.Linq.dll rather than the flattish bin/runtime/{config}/{files} artifact that's uploaded.

It seems like the simplest way to solve this is to have at least the part of "local live-live" (#494 for easy click) where it picks up the Libraries bits by a convention rather than by parsing the nuspec.

@jkoritzinsky did you have something in mind to get the PR past this? Very possible it came up and I'm forgetting.

(FYI I ended up moving Installer onto a base-job template similar to CoreFX since it seemed like the most reasonable way to share the dependency and artifact download logic. In particular generating the name of the target jobs would have been nasty to maintain everywhere.)

[jkoritzinsky]

You came to the same conclusion that I did (picking up the libraries by convention). You'll want to use the "shared framework" paths. There's an MSBuild property to enable binplacing the shared framework to the right folder. I don't remember it off the top of my head, but I'll find it again tomorrow and let you know.

[dagood]

Closing in favor of #705 without "WIP" in title. Would leave it open to see CI results, but the most recent run is stalled out with agent issues.

[jkoritzinsky]

No description provided.

[jkoritzinsky]

@dagood this is where I've been working on integrating the installer build/test jobs into the runtime.yml pipeline. I have the (non live-live) build working. Below are the next steps I was planning:

• Use the checkout job as the only clone/checkout step and checkout from the git bundle like coreclr and libraries.
• Enable live-live consumption in the installer jobs.
• Port the usage of platform-matrix.yml for the installer jobs in runtime.yml back to installer/azure-pipelines.yml so we don't have to duplicate declaring our build pools.

[dagood]

Status (also reminder for myself where this is at):

Now the installer jobs grab the bits, but fail because Libraries doesn't upload [runtime.*.]Microsoft.Private.CoreFx.NETCoreApp.nuspec. But even if Libraries did upload it, there's more to do there... the nuspec contains absolute paths on the other build agent and would need translation. And it might not be as simple as replacing the old base dir with a new base dir--I think these paths might be to deeper project dirs like ...\runtime\artifacts\bin\System.Private.Xml.Linq\netcoreapp-Windows_NT-Debug\System.Private.Xml.Linq.dll rather than the flattish bin/runtime/{config}/{files} artifact that's uploaded.

It seems like the simplest way to solve this is to have at least the part of "local live-live" (#494 for easy click) where it picks up the Libraries bits by a convention rather than by parsing the nuspec.

@jkoritzinsky did you have something in mind to get the PR past this? Very possible it came up and I'm forgetting.

(FYI I ended up moving Installer onto a base-job template similar to CoreFX since it seemed like the most reasonable way to share the dependency and artifact download logic. In particular generating the name of the target jobs would have been nasty to maintain everywhere.)

[jkoritzinsky]

You came to the same conclusion that I did (picking up the libraries by convention). You'll want to use the "shared framework" paths. There's an MSBuild property to enable binplacing the shared framework to the right folder. I don't remember it off the top of my head, but I'll find it again tomorrow and let you know.

[dagood]

Closing in favor of #705 without "WIP" in title. Would leave it open to see CI results, but the most recent run is stalled out with agent issues.

[jkoritzinsky]

No description provided.

[jkoritzinsky]

@dagood this is where I've been working on integrating the installer build/test jobs into the runtime.yml pipeline. I have the (non live-live) build working. Below are the next steps I was planning:

• Use the checkout job as the only clone/checkout step and checkout from the git bundle like coreclr and libraries.
• Enable live-live consumption in the installer jobs.
• Port the usage of platform-matrix.yml for the installer jobs in runtime.yml back to installer/azure-pipelines.yml so we don't have to duplicate declaring our build pools.

[dagood]

Status (also reminder for myself where this is at):

Now the installer jobs grab the bits, but fail because Libraries doesn't upload [runtime.*.]Microsoft.Private.CoreFx.NETCoreApp.nuspec. But even if Libraries did upload it, there's more to do there... the nuspec contains absolute paths on the other build agent and would need translation. And it might not be as simple as replacing the old base dir with a new base dir--I think these paths might be to deeper project dirs like ...\runtime\artifacts\bin\System.Private.Xml.Linq\netcoreapp-Windows_NT-Debug\System.Private.Xml.Linq.dll rather than the flattish bin/runtime/{config}/{files} artifact that's uploaded.

It seems like the simplest way to solve this is to have at least the part of "local live-live" (#494 for easy click) where it picks up the Libraries bits by a convention rather than by parsing the nuspec.

@jkoritzinsky did you have something in mind to get the PR past this? Very possible it came up and I'm forgetting.

(FYI I ended up moving Installer onto a base-job template similar to CoreFX since it seemed like the most reasonable way to share the dependency and artifact download logic. In particular generating the name of the target jobs would have been nasty to maintain everywhere.)

[jkoritzinsky]

You came to the same conclusion that I did (picking up the libraries by convention). You'll want to use the "shared framework" paths. There's an MSBuild property to enable binplacing the shared framework to the right folder. I don't remember it off the top of my head, but I'll find it again tomorrow and let you know.

[dagood]

Closing in favor of #705 without "WIP" in title. Would leave it open to see CI results, but the most recent run is stalled out with agent issues.

[jkoritzinsky]

No description provided.

[jkoritzinsky]

@dagood this is where I've been working on integrating the installer build/test jobs into the runtime.yml pipeline. I have the (non live-live) build working. Below are the next steps I was planning:

• Use the checkout job as the only clone/checkout step and checkout from the git bundle like coreclr and libraries.
• Enable live-live consumption in the installer jobs.
• Port the usage of platform-matrix.yml for the installer jobs in runtime.yml back to installer/azure-pipelines.yml so we don't have to duplicate declaring our build pools.

[dagood]

Status (also reminder for myself where this is at):

Now the installer jobs grab the bits, but fail because Libraries doesn't upload [runtime.*.]Microsoft.Private.CoreFx.NETCoreApp.nuspec. But even if Libraries did upload it, there's more to do there... the nuspec contains absolute paths on the other build agent and would need translation. And it might not be as simple as replacing the old base dir with a new base dir--I think these paths might be to deeper project dirs like ...\runtime\artifacts\bin\System.Private.Xml.Linq\netcoreapp-Windows_NT-Debug\System.Private.Xml.Linq.dll rather than the flattish bin/runtime/{config}/{files} artifact that's uploaded.

It seems like the simplest way to solve this is to have at least the part of "local live-live" (#494 for easy click) where it picks up the Libraries bits by a convention rather than by parsing the nuspec.

@jkoritzinsky did you have something in mind to get the PR past this? Very possible it came up and I'm forgetting.

(FYI I ended up moving Installer onto a base-job template similar to CoreFX since it seemed like the most reasonable way to share the dependency and artifact download logic. In particular generating the name of the target jobs would have been nasty to maintain everywhere.)

[jkoritzinsky]

You came to the same conclusion that I did (picking up the libraries by convention). You'll want to use the "shared framework" paths. There's an MSBuild property to enable binplacing the shared framework to the right folder. I don't remember it off the top of my head, but I'll find it again tomorrow and let you know.

[dagood]

Closing in favor of #705 without "WIP" in title. Would leave it open to see CI results, but the most recent run is stalled out with agent issues.

[jkoritzinsky]

No description provided.

[jkoritzinsky]

@dagood this is where I've been working on integrating the installer build/test jobs into the runtime.yml pipeline. I have the (non live-live) build working. Below are the next steps I was planning:

• Use the checkout job as the only clone/checkout step and checkout from the git bundle like coreclr and libraries.
• Enable live-live consumption in the installer jobs.
• Port the usage of platform-matrix.yml for the installer jobs in runtime.yml back to installer/azure-pipelines.yml so we don't have to duplicate declaring our build pools.

[dagood]

Status (also reminder for myself where this is at):

Now the installer jobs grab the bits, but fail because Libraries doesn't upload [runtime.*.]Microsoft.Private.CoreFx.NETCoreApp.nuspec. But even if Libraries did upload it, there's more to do there... the nuspec contains absolute paths on the other build agent and would need translation. And it might not be as simple as replacing the old base dir with a new base dir--I think these paths might be to deeper project dirs like ...\runtime\artifacts\bin\System.Private.Xml.Linq\netcoreapp-Windows_NT-Debug\System.Private.Xml.Linq.dll rather than the flattish bin/runtime/{config}/{files} artifact that's uploaded.

It seems like the simplest way to solve this is to have at least the part of "local live-live" (#494 for easy click) where it picks up the Libraries bits by a convention rather than by parsing the nuspec.

@jkoritzinsky did you have something in mind to get the PR past this? Very possible it came up and I'm forgetting.

(FYI I ended up moving Installer onto a base-job template similar to CoreFX since it seemed like the most reasonable way to share the dependency and artifact download logic. In particular generating the name of the target jobs would have been nasty to maintain everywhere.)

[jkoritzinsky]

You came to the same conclusion that I did (picking up the libraries by convention). You'll want to use the "shared framework" paths. There's an MSBuild property to enable binplacing the shared framework to the right folder. I don't remember it off the top of my head, but I'll find it again tomorrow and let you know.

[dagood]

Closing in favor of #705 without "WIP" in title. Would leave it open to see CI results, but the most recent run is stalled out with agent issues.