[master] Update dependencies from dotnet/arcade dotnet/roslyn dotnet/runtime

[dotnet-maestro]

This pull request updates the following dependencies

From https://github.com/dotnet/runtime

• Subscription: 8542892c-8237-4ac0-12f9-08d795579ebc
• Build: 20200317.12
• Date Produced: 3/18/2020 9:42 AM
• Commit: c0ddd1c5d1636de873398c8d9544e02289f95fec
• Branch: refs/heads/master
• Updates:
  • Microsoft.NETCore.App.Runtime.win-x64 -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Ref -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Internal -> 5.0.0-preview.3.20167.12

Coherency Updates

The following updates ensure that dependencies with a CoherentParentDependency
attribute were produced in a build used as input to the parent dependency's build.
See Dependency Description Format

• Microsoft.Win32.Registry -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ComponentModel.Annotations -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.DiagnosticSource -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.EventLog -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.IO.Pipelines -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Reflection.Metadata -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Runtime.CompilerServices.Unsafe -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Cng -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Xml -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ServiceProcess.ServiceController -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Encodings.Web -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Json -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• Microsoft.NETCore.Platforms -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)

From https://github.com/dotnet/roslyn

• Subscription: d9874afa-852c-4c5b-12f8-08d795579ebc
• Build: 20200316.3
• Date Produced: 3/16/2020 6:04 PM
• Commit: c0fc88e0ddd03abbdac8fc7e0d280b70d7208382
• Branch: refs/heads/master-vs-deps
• Updates:
  • Microsoft.Net.Compilers.Toolset -> 3.6.0-2.20166.3

From https://github.com/dotnet/arcade

• Subscription: 455063ee-4723-4484-e1e7-08d79559e508
• Build: 20200312.3
• Date Produced: 3/12/2020 5:55 PM
• Commit: 16d4350184cf362cd59807b589d1c93803025abc
• Branch: refs/heads/master
• Updates:
  • Microsoft.DotNet.Arcade.Sdk -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.GenAPI -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.Helix.Sdk -> 5.0.0-beta.20162.3

[ghost]

Auto-approving dependency update.

[pranavkm]

@dagood this error message showed up again:

The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Previous discussions here: dotnet/runtime#1129.

[dagood]

Previous discussions here: dotnet/runtime#1129.

To sum up, if it's the same issue, I would expect to see a data/PlatformManifest.txt in https://dev.azure.com/dnceng/internal/_build/results?buildId=553529&view=resultsPackageArtifacts/Microsoft.NETCore.App.Ref.5.0.0-preview.3.20159.13.nupkg that's missing some data, but I don't see anything obvious missing.

There are some "odd" names in the manifest with lib prefixes like libSystem.Security.Cryptography.Native.OpenSsl.so|Microsoft.NETCore.App.Ref||0.0.0.0 that I remember seeing some commotion over in a dotnet/runtime issue, but I haven't read through that.

@dotnet/runtime-infrastructure PTAL, I might be missing context on recent changes and it might be one of the other causes listed in that issue that I'm not very familiar with anyway (openssl version etc.)

[safern]

cc: @bartonjs@janvorli as they might see something obvious we might be missing here.

[bartonjs]

-4305 is kCCUnimplemented (https://opensource.apple.com/source/CommonCrypto/CommonCrypto-36064/CommonCrypto/CommonCryptor.h.auto.html). The last time we had a problem with that was fixed in 2017. Maybe something is being weird in library binding and we're finding an old copy? Is there a good way to figure out where our shim got loaded from on the CI machines?

AESKeyWrapEncrypt /does/ use ECB mode, so it /could/ be the problems that we fixed in dotnet/corefx#21631 ... if we're finding an old version of the library.

[pranavkm]

We fairly recently changed the OS version for the MacOS version. @dougbu would have more specifics about it. Could that affect this in any way?

[jkoritzinsky]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

[bartonjs]

We fairly recently changed the OS version for the MacOS version. ... Could that affect this in any way?

If you bumped to 10.15, yes (I don't think dotnet/runtime has that in CI yet because the images weren't ready/reliable). If you're 10.14 then the most likely culprit is the shim library rename, since that would be something you'd probably just now be ingesting (and dotnet/runtime isn't failing on 10.14).

[BrennanConroy]

We're on 10.14 now

[safern]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

That seems like a possible issue. I think the easiest would be to pull this and debug locally. Trying now to see what System.Security.Cryptography.Native.so it is loading.

[ericstj]

Agreed with the assesment of @jkoritzinsky and @safern. In 1.x we shipped nuget packages with these shims and rely on conflict resolution to make the framework copies win.

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

[pranavkm]

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

[ericstj]

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

Not exactly. The only way you'd get an old System.Native.* is if you had a reference to a 1.x era package. You'd need to find out which project was bringing that in and add the reference there. System.Security.Cryptography.Xml wouldn't be the culprit, it's probably some 3rd party package with very old dependencies.

[wtgodbe]

Oh, could be - we see transient failures w/ Pdb2Pdb every once in a while: https://github.com/dotnet/aspnetcore-internal/issues/3484

[wtgodbe]

Windows failures are gone on the second attempt, but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

And:

Interop+AppleCrypto+AppleCommonCryptoCryptographicException : The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Which is the same error as before...

[BrennanConroy]

but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

No, those are gone on the second attempt. You're looking at the test tab which will show failures across all retries.

@ericstj@safern Could we get some help with the crypto errors that haven't gone away with the Runtime fix?

[ericstj]

We had discussed that we'll hold off on consuming the extensions from dotnet/runtime to avoid too much churn. @JunTaoLuo can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

[BrennanConroy]

Can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

[ericstj]

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

Right you are, sorry @safern and I had assumed that was going on here, it would have caused similar errors (FileLoadExcetion due to two different assemblies with same identity). I don't think that's the case, so just ignore my comment.

Nevermind, just me, crossed wires. Let me look at crypto failures.

[ericstj]

Here's what I see in the binlog.

The following is referenced by the test project:

Native shim references

/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.apple/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
RuntimeIdentifier = osx.10.10-x64
/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.openssl/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
RuntimeIdentifier = osx.10.10-x64

PackageOverrides

PackageOverrides
NETStandard.Library
OverriddenPackages = Microsoft.Win32.Primitives|4.3.0;
System.AppContext|4.3.0;
System.Collections|4.3.0;
System.Collections.Concurrent|4.3.0;
System.Collections.Immutable|1.4.0;
System.Collections.NonGeneric|4.3.0;
System.Collections.Specialized|4.3.0;
System.ComponentModel|4.3.0;
System.ComponentModel.EventBasedAsync|4.3.0;
System.ComponentModel.Primitives|4.3.0;
System.ComponentModel.TypeConverter|4.3.0;
System.Console|4.3.0;
System.Data.Common|4.3.0;
System.Diagnostics.Contracts|4.3.0;
System.Diagnostics.Debug|4.3.0;
System.Diagnostics.FileVersionInfo|4.3.0;
System.Diagnostics.Process|4.3.0;
System.Diagnostics.StackTrace|4.3.0;
System.Diagnostics.TextWriterTraceListener|4.3.0;
System.Diagnostics.Tools|4.3.0;
System.Diagnostics.TraceSource|4.3.0;
System.Diagnostics.Tracing|4.3.0;
System.Dynamic.Runtime|4.3.0;
System.Globalization|4.3.0;
System.Globalization.Calendars|4.3.0;
System.Globalization.Extensions|4.3.0;
System.IO|4.3.0;
System.IO.Compression|4.3.0;
System.IO.Compression.ZipFile|4.3.0;
System.IO.FileSystem|4.3.0;
System.IO.FileSystem.DriveInfo|4.3.0;
System.IO.FileSystem.Primitives|4.3.0;
System.IO.FileSystem.Watcher|4.3.0;
System.IO.IsolatedStorage|4.3.0;
System.IO.MemoryMappedFiles|4.3.0;
System.IO.Pipes|4.3.0;
System.IO.UnmanagedMemoryStream|4.3.0;
System.Linq|4.3.0;
System.Linq.Expressions|4.3.0;
System.Linq.Queryable|4.3.0;
System.Net.Http|4.3.0;
System.Net.NameResolution|4.3.0;
System.Net.Primitives|4.3.0;
System.Net.Requests|4.3.0;
System.Net.Security|4.3.0;
System.Net.Sockets|4.3.0;
System.Net.WebHeaderCollection|4.3.0;
System.ObjectModel|4.3.0;
System.Private.DataContractSerialization|4.3.0;
System.Reflection|4.3.0;
System.Reflection.Emit|4.3.0;
System.Reflection.Emit.ILGeneration|4.3.0;
System.Reflection.Emit.Lightweight|4.3.0;
System.Reflection.Extensions|4.3.0;
System.Reflection.Primitives|4.3.0;
System.Reflection.TypeExtensions|4.3.0;
System.Resources.ResourceManager|4.3.0;
System.Runtime|4.3.0;
System.Runtime.Extensions|4.3.0;
System.Runtime.Handles|4.3.0;
System.Runtime.InteropServices|4.3.0;
System.Runtime.InteropServices.RuntimeInformation|4.3.0;
System.Runtime.Loader|4.3.0;
System.Runtime.Numerics|4.3.0;
System.Runtime.Serialization.Formatters|4.3.0;
System.Runtime.Serialization.Json|4.3.0;
System.Runtime.Serialization.Primitives|4.3.0;
System.Security.AccessControl|4.4.0;
System.Security.Claims|4.3.0;
System.Security.Cryptography.Algorithms|4.3.0;
System.Security.Cryptography.Csp|4.3.0;
System.Security.Cryptography.Encoding|4.3.0;
System.Security.Cryptography.Primitives|4.3.0;
System.Security.Cryptography.X509Certificates|4.3.0;
System.Security.Cryptography.Xml|4.4.0;
System.Security.Principal|4.3.0;
System.Security.Principal.Windows|4.4.0;
System.Text.Encoding|4.3.0;
System.Text.Encoding.Extensions|4.3.0;
System.Text.RegularExpressions|4.3.0;
System.Threading|4.3.0;
System.Threading.Overlapped|4.3.0;
System.Threading.Tasks|4.3.0;
System.Threading.Tasks.Extensions|4.3.0;
System.Threading.Tasks.Parallel|4.3.0;
System.Threading.Thread|4.3.0;
System.Threading.ThreadPool|4.3.0;
System.Threading.Timer|4.3.0;
System.ValueTuple|4.3.0;
System.Xml.ReaderWriter|4.3.0;
System.Xml.XDocument|4.3.0;
System.Xml.XmlDocument|4.3.0;
System.Xml.XmlSerializer|4.3.0;
System.Xml.XPath|4.3.0;
System.Xml.XPath.XDocument|4.3.0;
Microsoft.NETCore.App.Ref
OverriddenPackages = Microsoft.CSharp|4.4.0Microsoft.Win32.Primitives|4.3.0Microsoft.Win32.Registry|4.4.0runtime.debian.8-x64.runtime.native.System|4.3.0runtime.debian.8-x64.runtime.native.System.IO.Compression|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Http|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Security|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.23-x64.runtime.native.System|4.3.0runtime.fedora.23-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.24-x64.runtime.native.System|4.3.0runtime.fedora.24-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.13.2-x64.runtime.native.System|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.42.1-x64.runtime.native.System|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.osx.10.10-x64.runtime.native.System|4.3.0runtime.osx.10.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Http|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Security|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.rhel.7-x64.runtime.native.System|4.3.0runtime.rhel.7-x64.runtime.native.System.IO.Compression|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Http|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Security|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0System.AppContext|4.3.0System.Buffers|4.4.0System.Collections|4.3.0System.Collections.Concurrent|4.3.0System.Collections.Immutable|1.4.0System.Collections.NonGeneric|4.3.0System.Collections.Specialized|4.3.0System.ComponentModel|4.3.0System.ComponentModel.EventBasedAsync|4.3.0System.ComponentModel.Primitives|4.3.0System.ComponentModel.TypeConverter|4.3.0System.Console|4.3.0System.Data.Common|4.3.0System.Diagnostics.Contracts|4.3.0System.Diagnostics.Debug|4.3.0System.Diagnostics.DiagnosticSource|4.4.0System.Diagnostics.FileVersionInfo|4.3.0System.Diagnostics.Process|4.3.0System.Diagnostics.StackTrace|4.3.0System.Diagnostics.TextWriterTraceListener|4.3.0System.Diagnostics.Tools|4.3.0System.Diagnostics.TraceSource|4.3.0System.Diagnostics.Tracing|4.3.0System.Dynamic.Runtime|4.3.0System.Globalization|4.3.0System.Globalization.Calendars|4.3.0System.Globalization.Extensions|4.3.0System.IO|4.3.0System.IO.Compression|4.3.0System.IO.Compression.ZipFile|4.3.0System.IO.FileSystem|4.3.0System.IO.FileSystem.AccessControl|4.4.0System.IO.FileSystem.DriveInfo|4.3.0System.IO.FileSystem.Primitives|4.3.0System.IO.FileSystem.Watcher|4.3.0System.IO.IsolatedStorage|4.3.0System.IO.MemoryMappedFiles|4.3.0System.IO.Pipes|4.3.0System.IO.UnmanagedMemoryStream|4.3.0System.Linq|4.3.0System.Linq.Expressions|4.3.0System.Linq.Queryable|4.3.0System.Net.Http|4.3.0System.Net.NameResolution|4.3.0System.Net.Primitives|4.3.0System.Net.Requests|4.3.0System.Net.Security|4.3.0System.Net.Sockets|4.3.0System.Net.WebHeaderCollection|4.3.0System.ObjectModel|4.3.0System.Private.DataContractSerialization|4.3.0System.Reflection|4.3.0System.Reflection.Emit|4.3.0System.Reflection.Emit.ILGeneration|4.3.0System.Reflection.Emit.Lightweight|4.3.0System.Reflection.Extensions|4.3.0System.Reflection.Metadata|1.5.0System.Reflection.Primitives|4.3.0System.Reflection.TypeExtensions|4.3.0System.Resources.ResourceManager|4.3.0System.Runtime|4.3.0System.Runtime.Extensions|4.3.0System.Runtime.Handles|4.3.0System.Runtime.InteropServices|4.3.0System.Runtime.InteropServices.RuntimeInformation|4.3.0System.Runtime.Loader|4.3.0System.Runtime.Numerics|4.3.0System.Runtime.Serialization.Formatters|4.3.0System.Runtime.Serialization.Json|4.3.0System.Runtime.Serialization.Primitives|4.3.0System.Security.AccessControl|4.4.0System.Security.Claims|4.3.0System.Security.Cryptography.Algorithms|4.3.0System.Security.Cryptography.Cng|4.4.0System.Security.Cryptography.Csp|4.3.0System.Security.Cryptography.Encoding|4.3.0System.Security.Cryptography.OpenSsl|4.4.0System.Security.Cryptography.Primitives|4.3.0System.Security.Cryptography.X509Certificates|4.3.0System.Security.Cryptography.Xml|4.4.0System.Security.Principal|4.3.0System.Security.Principal.Windows|4.4.0System.Text.Encoding|4.3.0System.Text.Encoding.Extensions|4.3.0System.Text.RegularExpressions|4.3.0System.Threading|4.3.0System.Threading.Overlapped|4.3.0System.Threading.Tasks|4.3.0System.Threading.Tasks.Extensions|4.3.0System.Threading.Tasks.Parallel|4.3.0System.Threading.Thread|4.3.0System.Threading.ThreadPool|4.3.0System.Threading.Timer|4.3.0System.ValueTuple|4.3.0System.Xml.ReaderWriter|4.3.0System.Xml.XDocument|4.3.0System.Xml.XmlDocument|4.3.0System.Xml.XmlSerializer|4.3.0System.Xml.XPath|4.3.0System.Xml.XPath.XDocument|4.3.0

So the problem is that the PackageOverrides.txt file defined by the shared framework is not being used by the SDK /cc @dsplaisted@dagood

Will keep looking for root cause of that and a workaround.

[ericstj]

Ok, it looks like the new info is read, but is garbled. The PackageConflictOverrides returned from the ResolveTargetingPackAssets task has all package IDs running together. This means that all the content in PackageOverrides.txt is effectively ignored.

[ericstj]

Opened the bug here: dotnet/sdk#10904

To workaround you can run a target that fixes it, stay tuned and will share that.

[ericstj]

Ok, I tried to workaround the PackageOverride reading bug, but then encountered another issue, so reactivated dotnet/runtime#33450

So I took another tactic at fixing this: just avoiding the reference to the 1.x era packages. I found these were coming in transitively from a number of packages (Moq, xunit, etc) which reference NETStandard.Library 1.6.0. This is issue dotnet/sdk#3044. I'll workaround it by updating that transitive reference.

[JunTaoLuo]

FYI responding to an earlier conversation, there's no ingestion of dotnet/runtime packages (e.g. primitives) in this PR. I've paused that work until all packages are brought up in dotnet/runtime.

Edit: I see everyone has already figured this out, sorry for the spam

[dotnet-maestro]

Auto-Merge Status

This pull request will be merged because the following merge policies have succeeded.

• ✔️ Standard Merge Policies Succeeded - Successful checks: extensions-ci, extensions-ci (Build Ubuntu 16.04 Release), extensions-ci (Build Ubuntu 16.04 Debug), extensions-ci (Build Test: Linux Source Build), extensions-ci (Build OSX Release), extensions-ci (Build OSX Debug), extensions-ci (Build Windows Release), extensions-ci (Build Windows Debug), extensions-ci (Build Code check)
• ✔️ Standard Merge Policies Succeeded - No reviews have requested changes.

[safern]

Thanks @ericstj

[JunTaoLuo]

🎉

[dotnet-maestro]

This pull request updates the following dependencies

From https://github.com/dotnet/runtime

• Subscription: 8542892c-8237-4ac0-12f9-08d795579ebc
• Build: 20200317.12
• Date Produced: 3/18/2020 9:42 AM
• Commit: c0ddd1c5d1636de873398c8d9544e02289f95fec
• Branch: refs/heads/master
• Updates:
  • Microsoft.NETCore.App.Runtime.win-x64 -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Ref -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Internal -> 5.0.0-preview.3.20167.12

Coherency Updates

The following updates ensure that dependencies with a CoherentParentDependency
attribute were produced in a build used as input to the parent dependency's build.
See Dependency Description Format

• Microsoft.Win32.Registry -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ComponentModel.Annotations -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.DiagnosticSource -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.EventLog -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.IO.Pipelines -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Reflection.Metadata -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Runtime.CompilerServices.Unsafe -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Cng -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Xml -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ServiceProcess.ServiceController -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Encodings.Web -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Json -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• Microsoft.NETCore.Platforms -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)

From https://github.com/dotnet/roslyn

• Subscription: d9874afa-852c-4c5b-12f8-08d795579ebc
• Build: 20200316.3
• Date Produced: 3/16/2020 6:04 PM
• Commit: c0fc88e0ddd03abbdac8fc7e0d280b70d7208382
• Branch: refs/heads/master-vs-deps
• Updates:
  • Microsoft.Net.Compilers.Toolset -> 3.6.0-2.20166.3

From https://github.com/dotnet/arcade

• Subscription: 455063ee-4723-4484-e1e7-08d79559e508
• Build: 20200312.3
• Date Produced: 3/12/2020 5:55 PM
• Commit: 16d4350184cf362cd59807b589d1c93803025abc
• Branch: refs/heads/master
• Updates:
  • Microsoft.DotNet.Arcade.Sdk -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.GenAPI -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.Helix.Sdk -> 5.0.0-beta.20162.3

[ghost]

Auto-approving dependency update.

[pranavkm]

@dagood this error message showed up again:

The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Previous discussions here: dotnet/runtime#1129.

[dagood]

Previous discussions here: dotnet/runtime#1129.

To sum up, if it's the same issue, I would expect to see a data/PlatformManifest.txt in https://dev.azure.com/dnceng/internal/_build/results?buildId=553529&view=resultsPackageArtifacts/Microsoft.NETCore.App.Ref.5.0.0-preview.3.20159.13.nupkg that's missing some data, but I don't see anything obvious missing.

There are some "odd" names in the manifest with lib prefixes like libSystem.Security.Cryptography.Native.OpenSsl.so|Microsoft.NETCore.App.Ref||0.0.0.0 that I remember seeing some commotion over in a dotnet/runtime issue, but I haven't read through that.

@dotnet/runtime-infrastructure PTAL, I might be missing context on recent changes and it might be one of the other causes listed in that issue that I'm not very familiar with anyway (openssl version etc.)

[safern]

cc: @bartonjs@janvorli as they might see something obvious we might be missing here.

[bartonjs]

-4305 is kCCUnimplemented (https://opensource.apple.com/source/CommonCrypto/CommonCrypto-36064/CommonCrypto/CommonCryptor.h.auto.html). The last time we had a problem with that was fixed in 2017. Maybe something is being weird in library binding and we're finding an old copy? Is there a good way to figure out where our shim got loaded from on the CI machines?

AESKeyWrapEncrypt /does/ use ECB mode, so it /could/ be the problems that we fixed in dotnet/corefx#21631 ... if we're finding an old version of the library.

[pranavkm]

We fairly recently changed the OS version for the MacOS version. @dougbu would have more specifics about it. Could that affect this in any way?

[jkoritzinsky]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

[bartonjs]

We fairly recently changed the OS version for the MacOS version. ... Could that affect this in any way?

If you bumped to 10.15, yes (I don't think dotnet/runtime has that in CI yet because the images weren't ready/reliable). If you're 10.14 then the most likely culprit is the shim library rename, since that would be something you'd probably just now be ingesting (and dotnet/runtime isn't failing on 10.14).

[BrennanConroy]

We're on 10.14 now

[safern]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

That seems like a possible issue. I think the easiest would be to pull this and debug locally. Trying now to see what System.Security.Cryptography.Native.so it is loading.

[ericstj]

Agreed with the assesment of @jkoritzinsky and @safern. In 1.x we shipped nuget packages with these shims and rely on conflict resolution to make the framework copies win.

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

[pranavkm]

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

[ericstj]

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

Not exactly. The only way you'd get an old System.Native.* is if you had a reference to a 1.x era package. You'd need to find out which project was bringing that in and add the reference there. System.Security.Cryptography.Xml wouldn't be the culprit, it's probably some 3rd party package with very old dependencies.

[wtgodbe]

Oh, could be - we see transient failures w/ Pdb2Pdb every once in a while: https://github.com/dotnet/aspnetcore-internal/issues/3484

[wtgodbe]

Windows failures are gone on the second attempt, but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

And:

Interop+AppleCrypto+AppleCommonCryptoCryptographicException : The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Which is the same error as before...

[BrennanConroy]

but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

No, those are gone on the second attempt. You're looking at the test tab which will show failures across all retries.

@ericstj@safern Could we get some help with the crypto errors that haven't gone away with the Runtime fix?

[ericstj]

We had discussed that we'll hold off on consuming the extensions from dotnet/runtime to avoid too much churn. @JunTaoLuo can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

[BrennanConroy]

Can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

[ericstj]

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

Right you are, sorry @safern and I had assumed that was going on here, it would have caused similar errors (FileLoadExcetion due to two different assemblies with same identity). I don't think that's the case, so just ignore my comment.

Nevermind, just me, crossed wires. Let me look at crypto failures.

[ericstj]

Here's what I see in the binlog.

The following is referenced by the test project:

Native shim references

/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.apple/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
RuntimeIdentifier = osx.10.10-x64
/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.openssl/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
RuntimeIdentifier = osx.10.10-x64

PackageOverrides

PackageOverrides
NETStandard.Library
OverriddenPackages = Microsoft.Win32.Primitives|4.3.0;
System.AppContext|4.3.0;
System.Collections|4.3.0;
System.Collections.Concurrent|4.3.0;
System.Collections.Immutable|1.4.0;
System.Collections.NonGeneric|4.3.0;
System.Collections.Specialized|4.3.0;
System.ComponentModel|4.3.0;
System.ComponentModel.EventBasedAsync|4.3.0;
System.ComponentModel.Primitives|4.3.0;
System.ComponentModel.TypeConverter|4.3.0;
System.Console|4.3.0;
System.Data.Common|4.3.0;
System.Diagnostics.Contracts|4.3.0;
System.Diagnostics.Debug|4.3.0;
System.Diagnostics.FileVersionInfo|4.3.0;
System.Diagnostics.Process|4.3.0;
System.Diagnostics.StackTrace|4.3.0;
System.Diagnostics.TextWriterTraceListener|4.3.0;
System.Diagnostics.Tools|4.3.0;
System.Diagnostics.TraceSource|4.3.0;
System.Diagnostics.Tracing|4.3.0;
System.Dynamic.Runtime|4.3.0;
System.Globalization|4.3.0;
System.Globalization.Calendars|4.3.0;
System.Globalization.Extensions|4.3.0;
System.IO|4.3.0;
System.IO.Compression|4.3.0;
System.IO.Compression.ZipFile|4.3.0;
System.IO.FileSystem|4.3.0;
System.IO.FileSystem.DriveInfo|4.3.0;
System.IO.FileSystem.Primitives|4.3.0;
System.IO.FileSystem.Watcher|4.3.0;
System.IO.IsolatedStorage|4.3.0;
System.IO.MemoryMappedFiles|4.3.0;
System.IO.Pipes|4.3.0;
System.IO.UnmanagedMemoryStream|4.3.0;
System.Linq|4.3.0;
System.Linq.Expressions|4.3.0;
System.Linq.Queryable|4.3.0;
System.Net.Http|4.3.0;
System.Net.NameResolution|4.3.0;
System.Net.Primitives|4.3.0;
System.Net.Requests|4.3.0;
System.Net.Security|4.3.0;
System.Net.Sockets|4.3.0;
System.Net.WebHeaderCollection|4.3.0;
System.ObjectModel|4.3.0;
System.Private.DataContractSerialization|4.3.0;
System.Reflection|4.3.0;
System.Reflection.Emit|4.3.0;
System.Reflection.Emit.ILGeneration|4.3.0;
System.Reflection.Emit.Lightweight|4.3.0;
System.Reflection.Extensions|4.3.0;
System.Reflection.Primitives|4.3.0;
System.Reflection.TypeExtensions|4.3.0;
System.Resources.ResourceManager|4.3.0;
System.Runtime|4.3.0;
System.Runtime.Extensions|4.3.0;
System.Runtime.Handles|4.3.0;
System.Runtime.InteropServices|4.3.0;
System.Runtime.InteropServices.RuntimeInformation|4.3.0;
System.Runtime.Loader|4.3.0;
System.Runtime.Numerics|4.3.0;
System.Runtime.Serialization.Formatters|4.3.0;
System.Runtime.Serialization.Json|4.3.0;
System.Runtime.Serialization.Primitives|4.3.0;
System.Security.AccessControl|4.4.0;
System.Security.Claims|4.3.0;
System.Security.Cryptography.Algorithms|4.3.0;
System.Security.Cryptography.Csp|4.3.0;
System.Security.Cryptography.Encoding|4.3.0;
System.Security.Cryptography.Primitives|4.3.0;
System.Security.Cryptography.X509Certificates|4.3.0;
System.Security.Cryptography.Xml|4.4.0;
System.Security.Principal|4.3.0;
System.Security.Principal.Windows|4.4.0;
System.Text.Encoding|4.3.0;
System.Text.Encoding.Extensions|4.3.0;
System.Text.RegularExpressions|4.3.0;
System.Threading|4.3.0;
System.Threading.Overlapped|4.3.0;
System.Threading.Tasks|4.3.0;
System.Threading.Tasks.Extensions|4.3.0;
System.Threading.Tasks.Parallel|4.3.0;
System.Threading.Thread|4.3.0;
System.Threading.ThreadPool|4.3.0;
System.Threading.Timer|4.3.0;
System.ValueTuple|4.3.0;
System.Xml.ReaderWriter|4.3.0;
System.Xml.XDocument|4.3.0;
System.Xml.XmlDocument|4.3.0;
System.Xml.XmlSerializer|4.3.0;
System.Xml.XPath|4.3.0;
System.Xml.XPath.XDocument|4.3.0;
Microsoft.NETCore.App.Ref
OverriddenPackages = Microsoft.CSharp|4.4.0Microsoft.Win32.Primitives|4.3.0Microsoft.Win32.Registry|4.4.0runtime.debian.8-x64.runtime.native.System|4.3.0runtime.debian.8-x64.runtime.native.System.IO.Compression|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Http|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Security|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.23-x64.runtime.native.System|4.3.0runtime.fedora.23-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.24-x64.runtime.native.System|4.3.0runtime.fedora.24-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.13.2-x64.runtime.native.System|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.42.1-x64.runtime.native.System|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.osx.10.10-x64.runtime.native.System|4.3.0runtime.osx.10.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Http|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Security|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.rhel.7-x64.runtime.native.System|4.3.0runtime.rhel.7-x64.runtime.native.System.IO.Compression|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Http|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Security|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0System.AppContext|4.3.0System.Buffers|4.4.0System.Collections|4.3.0System.Collections.Concurrent|4.3.0System.Collections.Immutable|1.4.0System.Collections.NonGeneric|4.3.0System.Collections.Specialized|4.3.0System.ComponentModel|4.3.0System.ComponentModel.EventBasedAsync|4.3.0System.ComponentModel.Primitives|4.3.0System.ComponentModel.TypeConverter|4.3.0System.Console|4.3.0System.Data.Common|4.3.0System.Diagnostics.Contracts|4.3.0System.Diagnostics.Debug|4.3.0System.Diagnostics.DiagnosticSource|4.4.0System.Diagnostics.FileVersionInfo|4.3.0System.Diagnostics.Process|4.3.0System.Diagnostics.StackTrace|4.3.0System.Diagnostics.TextWriterTraceListener|4.3.0System.Diagnostics.Tools|4.3.0System.Diagnostics.TraceSource|4.3.0System.Diagnostics.Tracing|4.3.0System.Dynamic.Runtime|4.3.0System.Globalization|4.3.0System.Globalization.Calendars|4.3.0System.Globalization.Extensions|4.3.0System.IO|4.3.0System.IO.Compression|4.3.0System.IO.Compression.ZipFile|4.3.0System.IO.FileSystem|4.3.0System.IO.FileSystem.AccessControl|4.4.0System.IO.FileSystem.DriveInfo|4.3.0System.IO.FileSystem.Primitives|4.3.0System.IO.FileSystem.Watcher|4.3.0System.IO.IsolatedStorage|4.3.0System.IO.MemoryMappedFiles|4.3.0System.IO.Pipes|4.3.0System.IO.UnmanagedMemoryStream|4.3.0System.Linq|4.3.0System.Linq.Expressions|4.3.0System.Linq.Queryable|4.3.0System.Net.Http|4.3.0System.Net.NameResolution|4.3.0System.Net.Primitives|4.3.0System.Net.Requests|4.3.0System.Net.Security|4.3.0System.Net.Sockets|4.3.0System.Net.WebHeaderCollection|4.3.0System.ObjectModel|4.3.0System.Private.DataContractSerialization|4.3.0System.Reflection|4.3.0System.Reflection.Emit|4.3.0System.Reflection.Emit.ILGeneration|4.3.0System.Reflection.Emit.Lightweight|4.3.0System.Reflection.Extensions|4.3.0System.Reflection.Metadata|1.5.0System.Reflection.Primitives|4.3.0System.Reflection.TypeExtensions|4.3.0System.Resources.ResourceManager|4.3.0System.Runtime|4.3.0System.Runtime.Extensions|4.3.0System.Runtime.Handles|4.3.0System.Runtime.InteropServices|4.3.0System.Runtime.InteropServices.RuntimeInformation|4.3.0System.Runtime.Loader|4.3.0System.Runtime.Numerics|4.3.0System.Runtime.Serialization.Formatters|4.3.0System.Runtime.Serialization.Json|4.3.0System.Runtime.Serialization.Primitives|4.3.0System.Security.AccessControl|4.4.0System.Security.Claims|4.3.0System.Security.Cryptography.Algorithms|4.3.0System.Security.Cryptography.Cng|4.4.0System.Security.Cryptography.Csp|4.3.0System.Security.Cryptography.Encoding|4.3.0System.Security.Cryptography.OpenSsl|4.4.0System.Security.Cryptography.Primitives|4.3.0System.Security.Cryptography.X509Certificates|4.3.0System.Security.Cryptography.Xml|4.4.0System.Security.Principal|4.3.0System.Security.Principal.Windows|4.4.0System.Text.Encoding|4.3.0System.Text.Encoding.Extensions|4.3.0System.Text.RegularExpressions|4.3.0System.Threading|4.3.0System.Threading.Overlapped|4.3.0System.Threading.Tasks|4.3.0System.Threading.Tasks.Extensions|4.3.0System.Threading.Tasks.Parallel|4.3.0System.Threading.Thread|4.3.0System.Threading.ThreadPool|4.3.0System.Threading.Timer|4.3.0System.ValueTuple|4.3.0System.Xml.ReaderWriter|4.3.0System.Xml.XDocument|4.3.0System.Xml.XmlDocument|4.3.0System.Xml.XmlSerializer|4.3.0System.Xml.XPath|4.3.0System.Xml.XPath.XDocument|4.3.0

So the problem is that the PackageOverrides.txt file defined by the shared framework is not being used by the SDK /cc @dsplaisted@dagood

Will keep looking for root cause of that and a workaround.

[ericstj]

Ok, it looks like the new info is read, but is garbled. The PackageConflictOverrides returned from the ResolveTargetingPackAssets task has all package IDs running together. This means that all the content in PackageOverrides.txt is effectively ignored.

[ericstj]

Opened the bug here: dotnet/sdk#10904

To workaround you can run a target that fixes it, stay tuned and will share that.

[ericstj]

Ok, I tried to workaround the PackageOverride reading bug, but then encountered another issue, so reactivated dotnet/runtime#33450

So I took another tactic at fixing this: just avoiding the reference to the 1.x era packages. I found these were coming in transitively from a number of packages (Moq, xunit, etc) which reference NETStandard.Library 1.6.0. This is issue dotnet/sdk#3044. I'll workaround it by updating that transitive reference.

[JunTaoLuo]

FYI responding to an earlier conversation, there's no ingestion of dotnet/runtime packages (e.g. primitives) in this PR. I've paused that work until all packages are brought up in dotnet/runtime.

Edit: I see everyone has already figured this out, sorry for the spam

[dotnet-maestro]

Auto-Merge Status

This pull request will be merged because the following merge policies have succeeded.

• ✔️ Standard Merge Policies Succeeded - Successful checks: extensions-ci, extensions-ci (Build Ubuntu 16.04 Release), extensions-ci (Build Ubuntu 16.04 Debug), extensions-ci (Build Test: Linux Source Build), extensions-ci (Build OSX Release), extensions-ci (Build OSX Debug), extensions-ci (Build Windows Release), extensions-ci (Build Windows Debug), extensions-ci (Build Code check)
• ✔️ Standard Merge Policies Succeeded - No reviews have requested changes.

[safern]

Thanks @ericstj

[JunTaoLuo]

🎉

[dotnet-maestro]

This pull request updates the following dependencies

From https://github.com/dotnet/runtime

• Subscription: 8542892c-8237-4ac0-12f9-08d795579ebc
• Build: 20200317.12
• Date Produced: 3/18/2020 9:42 AM
• Commit: c0ddd1c5d1636de873398c8d9544e02289f95fec
• Branch: refs/heads/master
• Updates:
  • Microsoft.NETCore.App.Runtime.win-x64 -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Ref -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Internal -> 5.0.0-preview.3.20167.12

Coherency Updates

The following updates ensure that dependencies with a CoherentParentDependency
attribute were produced in a build used as input to the parent dependency's build.
See Dependency Description Format

• Microsoft.Win32.Registry -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ComponentModel.Annotations -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.DiagnosticSource -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.EventLog -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.IO.Pipelines -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Reflection.Metadata -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Runtime.CompilerServices.Unsafe -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Cng -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Xml -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ServiceProcess.ServiceController -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Encodings.Web -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Json -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• Microsoft.NETCore.Platforms -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)

From https://github.com/dotnet/roslyn

• Subscription: d9874afa-852c-4c5b-12f8-08d795579ebc
• Build: 20200316.3
• Date Produced: 3/16/2020 6:04 PM
• Commit: c0fc88e0ddd03abbdac8fc7e0d280b70d7208382
• Branch: refs/heads/master-vs-deps
• Updates:
  • Microsoft.Net.Compilers.Toolset -> 3.6.0-2.20166.3

From https://github.com/dotnet/arcade

• Subscription: 455063ee-4723-4484-e1e7-08d79559e508
• Build: 20200312.3
• Date Produced: 3/12/2020 5:55 PM
• Commit: 16d4350184cf362cd59807b589d1c93803025abc
• Branch: refs/heads/master
• Updates:
  • Microsoft.DotNet.Arcade.Sdk -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.GenAPI -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.Helix.Sdk -> 5.0.0-beta.20162.3

[ghost]

Auto-approving dependency update.

[pranavkm]

@dagood this error message showed up again:

The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Previous discussions here: dotnet/runtime#1129.

[dagood]

Previous discussions here: dotnet/runtime#1129.

To sum up, if it's the same issue, I would expect to see a data/PlatformManifest.txt in https://dev.azure.com/dnceng/internal/_build/results?buildId=553529&view=resultsPackageArtifacts/Microsoft.NETCore.App.Ref.5.0.0-preview.3.20159.13.nupkg that's missing some data, but I don't see anything obvious missing.

There are some "odd" names in the manifest with lib prefixes like libSystem.Security.Cryptography.Native.OpenSsl.so|Microsoft.NETCore.App.Ref||0.0.0.0 that I remember seeing some commotion over in a dotnet/runtime issue, but I haven't read through that.

@dotnet/runtime-infrastructure PTAL, I might be missing context on recent changes and it might be one of the other causes listed in that issue that I'm not very familiar with anyway (openssl version etc.)

[safern]

cc: @bartonjs@janvorli as they might see something obvious we might be missing here.

[bartonjs]

-4305 is kCCUnimplemented (https://opensource.apple.com/source/CommonCrypto/CommonCrypto-36064/CommonCrypto/CommonCryptor.h.auto.html). The last time we had a problem with that was fixed in 2017. Maybe something is being weird in library binding and we're finding an old copy? Is there a good way to figure out where our shim got loaded from on the CI machines?

AESKeyWrapEncrypt /does/ use ECB mode, so it /could/ be the problems that we fixed in dotnet/corefx#21631 ... if we're finding an old version of the library.

[pranavkm]

We fairly recently changed the OS version for the MacOS version. @dougbu would have more specifics about it. Could that affect this in any way?

[jkoritzinsky]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

[bartonjs]

We fairly recently changed the OS version for the MacOS version. ... Could that affect this in any way?

If you bumped to 10.15, yes (I don't think dotnet/runtime has that in CI yet because the images weren't ready/reliable). If you're 10.14 then the most likely culprit is the shim library rename, since that would be something you'd probably just now be ingesting (and dotnet/runtime isn't failing on 10.14).

[BrennanConroy]

We're on 10.14 now

[safern]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

That seems like a possible issue. I think the easiest would be to pull this and debug locally. Trying now to see what System.Security.Cryptography.Native.so it is loading.

[ericstj]

Agreed with the assesment of @jkoritzinsky and @safern. In 1.x we shipped nuget packages with these shims and rely on conflict resolution to make the framework copies win.

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

[pranavkm]

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

[ericstj]

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

Not exactly. The only way you'd get an old System.Native.* is if you had a reference to a 1.x era package. You'd need to find out which project was bringing that in and add the reference there. System.Security.Cryptography.Xml wouldn't be the culprit, it's probably some 3rd party package with very old dependencies.

[wtgodbe]

Oh, could be - we see transient failures w/ Pdb2Pdb every once in a while: https://github.com/dotnet/aspnetcore-internal/issues/3484

[wtgodbe]

Windows failures are gone on the second attempt, but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

And:

Interop+AppleCrypto+AppleCommonCryptoCryptographicException : The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Which is the same error as before...

[BrennanConroy]

but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

No, those are gone on the second attempt. You're looking at the test tab which will show failures across all retries.

@ericstj@safern Could we get some help with the crypto errors that haven't gone away with the Runtime fix?

[ericstj]

We had discussed that we'll hold off on consuming the extensions from dotnet/runtime to avoid too much churn. @JunTaoLuo can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

[BrennanConroy]

Can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

[ericstj]

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

Right you are, sorry @safern and I had assumed that was going on here, it would have caused similar errors (FileLoadExcetion due to two different assemblies with same identity). I don't think that's the case, so just ignore my comment.

Nevermind, just me, crossed wires. Let me look at crypto failures.

[ericstj]

Here's what I see in the binlog.

The following is referenced by the test project:

Native shim references

/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.apple/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
RuntimeIdentifier = osx.10.10-x64
/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.openssl/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
RuntimeIdentifier = osx.10.10-x64

PackageOverrides

PackageOverrides
NETStandard.Library
OverriddenPackages = Microsoft.Win32.Primitives|4.3.0;
System.AppContext|4.3.0;
System.Collections|4.3.0;
System.Collections.Concurrent|4.3.0;
System.Collections.Immutable|1.4.0;
System.Collections.NonGeneric|4.3.0;
System.Collections.Specialized|4.3.0;
System.ComponentModel|4.3.0;
System.ComponentModel.EventBasedAsync|4.3.0;
System.ComponentModel.Primitives|4.3.0;
System.ComponentModel.TypeConverter|4.3.0;
System.Console|4.3.0;
System.Data.Common|4.3.0;
System.Diagnostics.Contracts|4.3.0;
System.Diagnostics.Debug|4.3.0;
System.Diagnostics.FileVersionInfo|4.3.0;
System.Diagnostics.Process|4.3.0;
System.Diagnostics.StackTrace|4.3.0;
System.Diagnostics.TextWriterTraceListener|4.3.0;
System.Diagnostics.Tools|4.3.0;
System.Diagnostics.TraceSource|4.3.0;
System.Diagnostics.Tracing|4.3.0;
System.Dynamic.Runtime|4.3.0;
System.Globalization|4.3.0;
System.Globalization.Calendars|4.3.0;
System.Globalization.Extensions|4.3.0;
System.IO|4.3.0;
System.IO.Compression|4.3.0;
System.IO.Compression.ZipFile|4.3.0;
System.IO.FileSystem|4.3.0;
System.IO.FileSystem.DriveInfo|4.3.0;
System.IO.FileSystem.Primitives|4.3.0;
System.IO.FileSystem.Watcher|4.3.0;
System.IO.IsolatedStorage|4.3.0;
System.IO.MemoryMappedFiles|4.3.0;
System.IO.Pipes|4.3.0;
System.IO.UnmanagedMemoryStream|4.3.0;
System.Linq|4.3.0;
System.Linq.Expressions|4.3.0;
System.Linq.Queryable|4.3.0;
System.Net.Http|4.3.0;
System.Net.NameResolution|4.3.0;
System.Net.Primitives|4.3.0;
System.Net.Requests|4.3.0;
System.Net.Security|4.3.0;
System.Net.Sockets|4.3.0;
System.Net.WebHeaderCollection|4.3.0;
System.ObjectModel|4.3.0;
System.Private.DataContractSerialization|4.3.0;
System.Reflection|4.3.0;
System.Reflection.Emit|4.3.0;
System.Reflection.Emit.ILGeneration|4.3.0;
System.Reflection.Emit.Lightweight|4.3.0;
System.Reflection.Extensions|4.3.0;
System.Reflection.Primitives|4.3.0;
System.Reflection.TypeExtensions|4.3.0;
System.Resources.ResourceManager|4.3.0;
System.Runtime|4.3.0;
System.Runtime.Extensions|4.3.0;
System.Runtime.Handles|4.3.0;
System.Runtime.InteropServices|4.3.0;
System.Runtime.InteropServices.RuntimeInformation|4.3.0;
System.Runtime.Loader|4.3.0;
System.Runtime.Numerics|4.3.0;
System.Runtime.Serialization.Formatters|4.3.0;
System.Runtime.Serialization.Json|4.3.0;
System.Runtime.Serialization.Primitives|4.3.0;
System.Security.AccessControl|4.4.0;
System.Security.Claims|4.3.0;
System.Security.Cryptography.Algorithms|4.3.0;
System.Security.Cryptography.Csp|4.3.0;
System.Security.Cryptography.Encoding|4.3.0;
System.Security.Cryptography.Primitives|4.3.0;
System.Security.Cryptography.X509Certificates|4.3.0;
System.Security.Cryptography.Xml|4.4.0;
System.Security.Principal|4.3.0;
System.Security.Principal.Windows|4.4.0;
System.Text.Encoding|4.3.0;
System.Text.Encoding.Extensions|4.3.0;
System.Text.RegularExpressions|4.3.0;
System.Threading|4.3.0;
System.Threading.Overlapped|4.3.0;
System.Threading.Tasks|4.3.0;
System.Threading.Tasks.Extensions|4.3.0;
System.Threading.Tasks.Parallel|4.3.0;
System.Threading.Thread|4.3.0;
System.Threading.ThreadPool|4.3.0;
System.Threading.Timer|4.3.0;
System.ValueTuple|4.3.0;
System.Xml.ReaderWriter|4.3.0;
System.Xml.XDocument|4.3.0;
System.Xml.XmlDocument|4.3.0;
System.Xml.XmlSerializer|4.3.0;
System.Xml.XPath|4.3.0;
System.Xml.XPath.XDocument|4.3.0;
Microsoft.NETCore.App.Ref
OverriddenPackages = Microsoft.CSharp|4.4.0Microsoft.Win32.Primitives|4.3.0Microsoft.Win32.Registry|4.4.0runtime.debian.8-x64.runtime.native.System|4.3.0runtime.debian.8-x64.runtime.native.System.IO.Compression|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Http|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Security|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.23-x64.runtime.native.System|4.3.0runtime.fedora.23-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.24-x64.runtime.native.System|4.3.0runtime.fedora.24-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.13.2-x64.runtime.native.System|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.42.1-x64.runtime.native.System|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.osx.10.10-x64.runtime.native.System|4.3.0runtime.osx.10.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Http|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Security|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.rhel.7-x64.runtime.native.System|4.3.0runtime.rhel.7-x64.runtime.native.System.IO.Compression|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Http|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Security|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0System.AppContext|4.3.0System.Buffers|4.4.0System.Collections|4.3.0System.Collections.Concurrent|4.3.0System.Collections.Immutable|1.4.0System.Collections.NonGeneric|4.3.0System.Collections.Specialized|4.3.0System.ComponentModel|4.3.0System.ComponentModel.EventBasedAsync|4.3.0System.ComponentModel.Primitives|4.3.0System.ComponentModel.TypeConverter|4.3.0System.Console|4.3.0System.Data.Common|4.3.0System.Diagnostics.Contracts|4.3.0System.Diagnostics.Debug|4.3.0System.Diagnostics.DiagnosticSource|4.4.0System.Diagnostics.FileVersionInfo|4.3.0System.Diagnostics.Process|4.3.0System.Diagnostics.StackTrace|4.3.0System.Diagnostics.TextWriterTraceListener|4.3.0System.Diagnostics.Tools|4.3.0System.Diagnostics.TraceSource|4.3.0System.Diagnostics.Tracing|4.3.0System.Dynamic.Runtime|4.3.0System.Globalization|4.3.0System.Globalization.Calendars|4.3.0System.Globalization.Extensions|4.3.0System.IO|4.3.0System.IO.Compression|4.3.0System.IO.Compression.ZipFile|4.3.0System.IO.FileSystem|4.3.0System.IO.FileSystem.AccessControl|4.4.0System.IO.FileSystem.DriveInfo|4.3.0System.IO.FileSystem.Primitives|4.3.0System.IO.FileSystem.Watcher|4.3.0System.IO.IsolatedStorage|4.3.0System.IO.MemoryMappedFiles|4.3.0System.IO.Pipes|4.3.0System.IO.UnmanagedMemoryStream|4.3.0System.Linq|4.3.0System.Linq.Expressions|4.3.0System.Linq.Queryable|4.3.0System.Net.Http|4.3.0System.Net.NameResolution|4.3.0System.Net.Primitives|4.3.0System.Net.Requests|4.3.0System.Net.Security|4.3.0System.Net.Sockets|4.3.0System.Net.WebHeaderCollection|4.3.0System.ObjectModel|4.3.0System.Private.DataContractSerialization|4.3.0System.Reflection|4.3.0System.Reflection.Emit|4.3.0System.Reflection.Emit.ILGeneration|4.3.0System.Reflection.Emit.Lightweight|4.3.0System.Reflection.Extensions|4.3.0System.Reflection.Metadata|1.5.0System.Reflection.Primitives|4.3.0System.Reflection.TypeExtensions|4.3.0System.Resources.ResourceManager|4.3.0System.Runtime|4.3.0System.Runtime.Extensions|4.3.0System.Runtime.Handles|4.3.0System.Runtime.InteropServices|4.3.0System.Runtime.InteropServices.RuntimeInformation|4.3.0System.Runtime.Loader|4.3.0System.Runtime.Numerics|4.3.0System.Runtime.Serialization.Formatters|4.3.0System.Runtime.Serialization.Json|4.3.0System.Runtime.Serialization.Primitives|4.3.0System.Security.AccessControl|4.4.0System.Security.Claims|4.3.0System.Security.Cryptography.Algorithms|4.3.0System.Security.Cryptography.Cng|4.4.0System.Security.Cryptography.Csp|4.3.0System.Security.Cryptography.Encoding|4.3.0System.Security.Cryptography.OpenSsl|4.4.0System.Security.Cryptography.Primitives|4.3.0System.Security.Cryptography.X509Certificates|4.3.0System.Security.Cryptography.Xml|4.4.0System.Security.Principal|4.3.0System.Security.Principal.Windows|4.4.0System.Text.Encoding|4.3.0System.Text.Encoding.Extensions|4.3.0System.Text.RegularExpressions|4.3.0System.Threading|4.3.0System.Threading.Overlapped|4.3.0System.Threading.Tasks|4.3.0System.Threading.Tasks.Extensions|4.3.0System.Threading.Tasks.Parallel|4.3.0System.Threading.Thread|4.3.0System.Threading.ThreadPool|4.3.0System.Threading.Timer|4.3.0System.ValueTuple|4.3.0System.Xml.ReaderWriter|4.3.0System.Xml.XDocument|4.3.0System.Xml.XmlDocument|4.3.0System.Xml.XmlSerializer|4.3.0System.Xml.XPath|4.3.0System.Xml.XPath.XDocument|4.3.0

So the problem is that the PackageOverrides.txt file defined by the shared framework is not being used by the SDK /cc @dsplaisted@dagood

Will keep looking for root cause of that and a workaround.

[ericstj]

Ok, it looks like the new info is read, but is garbled. The PackageConflictOverrides returned from the ResolveTargetingPackAssets task has all package IDs running together. This means that all the content in PackageOverrides.txt is effectively ignored.

[ericstj]

Opened the bug here: dotnet/sdk#10904

To workaround you can run a target that fixes it, stay tuned and will share that.

[ericstj]

Ok, I tried to workaround the PackageOverride reading bug, but then encountered another issue, so reactivated dotnet/runtime#33450

So I took another tactic at fixing this: just avoiding the reference to the 1.x era packages. I found these were coming in transitively from a number of packages (Moq, xunit, etc) which reference NETStandard.Library 1.6.0. This is issue dotnet/sdk#3044. I'll workaround it by updating that transitive reference.

[JunTaoLuo]

FYI responding to an earlier conversation, there's no ingestion of dotnet/runtime packages (e.g. primitives) in this PR. I've paused that work until all packages are brought up in dotnet/runtime.

Edit: I see everyone has already figured this out, sorry for the spam

[dotnet-maestro]

Auto-Merge Status

This pull request will be merged because the following merge policies have succeeded.

• ✔️ Standard Merge Policies Succeeded - Successful checks: extensions-ci, extensions-ci (Build Ubuntu 16.04 Release), extensions-ci (Build Ubuntu 16.04 Debug), extensions-ci (Build Test: Linux Source Build), extensions-ci (Build OSX Release), extensions-ci (Build OSX Debug), extensions-ci (Build Windows Release), extensions-ci (Build Windows Debug), extensions-ci (Build Code check)
• ✔️ Standard Merge Policies Succeeded - No reviews have requested changes.

[safern]

Thanks @ericstj

[JunTaoLuo]

🎉

[dotnet-maestro]

This pull request updates the following dependencies

From https://github.com/dotnet/runtime

• Subscription: 8542892c-8237-4ac0-12f9-08d795579ebc
• Build: 20200317.12
• Date Produced: 3/18/2020 9:42 AM
• Commit: c0ddd1c5d1636de873398c8d9544e02289f95fec
• Branch: refs/heads/master
• Updates:
  • Microsoft.NETCore.App.Runtime.win-x64 -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Ref -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Internal -> 5.0.0-preview.3.20167.12

Coherency Updates

The following updates ensure that dependencies with a CoherentParentDependency
attribute were produced in a build used as input to the parent dependency's build.
See Dependency Description Format

• Microsoft.Win32.Registry -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ComponentModel.Annotations -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.DiagnosticSource -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.EventLog -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.IO.Pipelines -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Reflection.Metadata -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Runtime.CompilerServices.Unsafe -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Cng -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Xml -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ServiceProcess.ServiceController -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Encodings.Web -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Json -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• Microsoft.NETCore.Platforms -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)

From https://github.com/dotnet/roslyn

• Subscription: d9874afa-852c-4c5b-12f8-08d795579ebc
• Build: 20200316.3
• Date Produced: 3/16/2020 6:04 PM
• Commit: c0fc88e0ddd03abbdac8fc7e0d280b70d7208382
• Branch: refs/heads/master-vs-deps
• Updates:
  • Microsoft.Net.Compilers.Toolset -> 3.6.0-2.20166.3

From https://github.com/dotnet/arcade

• Subscription: 455063ee-4723-4484-e1e7-08d79559e508
• Build: 20200312.3
• Date Produced: 3/12/2020 5:55 PM
• Commit: 16d4350184cf362cd59807b589d1c93803025abc
• Branch: refs/heads/master
• Updates:
  • Microsoft.DotNet.Arcade.Sdk -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.GenAPI -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.Helix.Sdk -> 5.0.0-beta.20162.3

[ghost]

Auto-approving dependency update.

[pranavkm]

@dagood this error message showed up again:

The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Previous discussions here: dotnet/runtime#1129.

[dagood]

Previous discussions here: dotnet/runtime#1129.

To sum up, if it's the same issue, I would expect to see a data/PlatformManifest.txt in https://dev.azure.com/dnceng/internal/_build/results?buildId=553529&view=resultsPackageArtifacts/Microsoft.NETCore.App.Ref.5.0.0-preview.3.20159.13.nupkg that's missing some data, but I don't see anything obvious missing.

There are some "odd" names in the manifest with lib prefixes like libSystem.Security.Cryptography.Native.OpenSsl.so|Microsoft.NETCore.App.Ref||0.0.0.0 that I remember seeing some commotion over in a dotnet/runtime issue, but I haven't read through that.

@dotnet/runtime-infrastructure PTAL, I might be missing context on recent changes and it might be one of the other causes listed in that issue that I'm not very familiar with anyway (openssl version etc.)

[safern]

cc: @bartonjs@janvorli as they might see something obvious we might be missing here.

[bartonjs]

-4305 is kCCUnimplemented (https://opensource.apple.com/source/CommonCrypto/CommonCrypto-36064/CommonCrypto/CommonCryptor.h.auto.html). The last time we had a problem with that was fixed in 2017. Maybe something is being weird in library binding and we're finding an old copy? Is there a good way to figure out where our shim got loaded from on the CI machines?

AESKeyWrapEncrypt /does/ use ECB mode, so it /could/ be the problems that we fixed in dotnet/corefx#21631 ... if we're finding an old version of the library.

[pranavkm]

We fairly recently changed the OS version for the MacOS version. @dougbu would have more specifics about it. Could that affect this in any way?

[jkoritzinsky]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

[bartonjs]

We fairly recently changed the OS version for the MacOS version. ... Could that affect this in any way?

If you bumped to 10.15, yes (I don't think dotnet/runtime has that in CI yet because the images weren't ready/reliable). If you're 10.14 then the most likely culprit is the shim library rename, since that would be something you'd probably just now be ingesting (and dotnet/runtime isn't failing on 10.14).

[BrennanConroy]

We're on 10.14 now

[safern]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

That seems like a possible issue. I think the easiest would be to pull this and debug locally. Trying now to see what System.Security.Cryptography.Native.so it is loading.

[ericstj]

Agreed with the assesment of @jkoritzinsky and @safern. In 1.x we shipped nuget packages with these shims and rely on conflict resolution to make the framework copies win.

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

[pranavkm]

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

[ericstj]

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

Not exactly. The only way you'd get an old System.Native.* is if you had a reference to a 1.x era package. You'd need to find out which project was bringing that in and add the reference there. System.Security.Cryptography.Xml wouldn't be the culprit, it's probably some 3rd party package with very old dependencies.

[wtgodbe]

Oh, could be - we see transient failures w/ Pdb2Pdb every once in a while: https://github.com/dotnet/aspnetcore-internal/issues/3484

[wtgodbe]

Windows failures are gone on the second attempt, but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

And:

Interop+AppleCrypto+AppleCommonCryptoCryptographicException : The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Which is the same error as before...

[BrennanConroy]

but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

No, those are gone on the second attempt. You're looking at the test tab which will show failures across all retries.

@ericstj@safern Could we get some help with the crypto errors that haven't gone away with the Runtime fix?

[ericstj]

We had discussed that we'll hold off on consuming the extensions from dotnet/runtime to avoid too much churn. @JunTaoLuo can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

[BrennanConroy]

Can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

[ericstj]

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

Right you are, sorry @safern and I had assumed that was going on here, it would have caused similar errors (FileLoadExcetion due to two different assemblies with same identity). I don't think that's the case, so just ignore my comment.

Nevermind, just me, crossed wires. Let me look at crypto failures.

[ericstj]

Here's what I see in the binlog.

The following is referenced by the test project:

Native shim references

/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.apple/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
RuntimeIdentifier = osx.10.10-x64
/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.openssl/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
RuntimeIdentifier = osx.10.10-x64

PackageOverrides

PackageOverrides
NETStandard.Library
OverriddenPackages = Microsoft.Win32.Primitives|4.3.0;
System.AppContext|4.3.0;
System.Collections|4.3.0;
System.Collections.Concurrent|4.3.0;
System.Collections.Immutable|1.4.0;
System.Collections.NonGeneric|4.3.0;
System.Collections.Specialized|4.3.0;
System.ComponentModel|4.3.0;
System.ComponentModel.EventBasedAsync|4.3.0;
System.ComponentModel.Primitives|4.3.0;
System.ComponentModel.TypeConverter|4.3.0;
System.Console|4.3.0;
System.Data.Common|4.3.0;
System.Diagnostics.Contracts|4.3.0;
System.Diagnostics.Debug|4.3.0;
System.Diagnostics.FileVersionInfo|4.3.0;
System.Diagnostics.Process|4.3.0;
System.Diagnostics.StackTrace|4.3.0;
System.Diagnostics.TextWriterTraceListener|4.3.0;
System.Diagnostics.Tools|4.3.0;
System.Diagnostics.TraceSource|4.3.0;
System.Diagnostics.Tracing|4.3.0;
System.Dynamic.Runtime|4.3.0;
System.Globalization|4.3.0;
System.Globalization.Calendars|4.3.0;
System.Globalization.Extensions|4.3.0;
System.IO|4.3.0;
System.IO.Compression|4.3.0;
System.IO.Compression.ZipFile|4.3.0;
System.IO.FileSystem|4.3.0;
System.IO.FileSystem.DriveInfo|4.3.0;
System.IO.FileSystem.Primitives|4.3.0;
System.IO.FileSystem.Watcher|4.3.0;
System.IO.IsolatedStorage|4.3.0;
System.IO.MemoryMappedFiles|4.3.0;
System.IO.Pipes|4.3.0;
System.IO.UnmanagedMemoryStream|4.3.0;
System.Linq|4.3.0;
System.Linq.Expressions|4.3.0;
System.Linq.Queryable|4.3.0;
System.Net.Http|4.3.0;
System.Net.NameResolution|4.3.0;
System.Net.Primitives|4.3.0;
System.Net.Requests|4.3.0;
System.Net.Security|4.3.0;
System.Net.Sockets|4.3.0;
System.Net.WebHeaderCollection|4.3.0;
System.ObjectModel|4.3.0;
System.Private.DataContractSerialization|4.3.0;
System.Reflection|4.3.0;
System.Reflection.Emit|4.3.0;
System.Reflection.Emit.ILGeneration|4.3.0;
System.Reflection.Emit.Lightweight|4.3.0;
System.Reflection.Extensions|4.3.0;
System.Reflection.Primitives|4.3.0;
System.Reflection.TypeExtensions|4.3.0;
System.Resources.ResourceManager|4.3.0;
System.Runtime|4.3.0;
System.Runtime.Extensions|4.3.0;
System.Runtime.Handles|4.3.0;
System.Runtime.InteropServices|4.3.0;
System.Runtime.InteropServices.RuntimeInformation|4.3.0;
System.Runtime.Loader|4.3.0;
System.Runtime.Numerics|4.3.0;
System.Runtime.Serialization.Formatters|4.3.0;
System.Runtime.Serialization.Json|4.3.0;
System.Runtime.Serialization.Primitives|4.3.0;
System.Security.AccessControl|4.4.0;
System.Security.Claims|4.3.0;
System.Security.Cryptography.Algorithms|4.3.0;
System.Security.Cryptography.Csp|4.3.0;
System.Security.Cryptography.Encoding|4.3.0;
System.Security.Cryptography.Primitives|4.3.0;
System.Security.Cryptography.X509Certificates|4.3.0;
System.Security.Cryptography.Xml|4.4.0;
System.Security.Principal|4.3.0;
System.Security.Principal.Windows|4.4.0;
System.Text.Encoding|4.3.0;
System.Text.Encoding.Extensions|4.3.0;
System.Text.RegularExpressions|4.3.0;
System.Threading|4.3.0;
System.Threading.Overlapped|4.3.0;
System.Threading.Tasks|4.3.0;
System.Threading.Tasks.Extensions|4.3.0;
System.Threading.Tasks.Parallel|4.3.0;
System.Threading.Thread|4.3.0;
System.Threading.ThreadPool|4.3.0;
System.Threading.Timer|4.3.0;
System.ValueTuple|4.3.0;
System.Xml.ReaderWriter|4.3.0;
System.Xml.XDocument|4.3.0;
System.Xml.XmlDocument|4.3.0;
System.Xml.XmlSerializer|4.3.0;
System.Xml.XPath|4.3.0;
System.Xml.XPath.XDocument|4.3.0;
Microsoft.NETCore.App.Ref
OverriddenPackages = Microsoft.CSharp|4.4.0Microsoft.Win32.Primitives|4.3.0Microsoft.Win32.Registry|4.4.0runtime.debian.8-x64.runtime.native.System|4.3.0runtime.debian.8-x64.runtime.native.System.IO.Compression|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Http|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Security|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.23-x64.runtime.native.System|4.3.0runtime.fedora.23-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.24-x64.runtime.native.System|4.3.0runtime.fedora.24-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.13.2-x64.runtime.native.System|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.42.1-x64.runtime.native.System|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.osx.10.10-x64.runtime.native.System|4.3.0runtime.osx.10.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Http|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Security|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.rhel.7-x64.runtime.native.System|4.3.0runtime.rhel.7-x64.runtime.native.System.IO.Compression|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Http|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Security|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0System.AppContext|4.3.0System.Buffers|4.4.0System.Collections|4.3.0System.Collections.Concurrent|4.3.0System.Collections.Immutable|1.4.0System.Collections.NonGeneric|4.3.0System.Collections.Specialized|4.3.0System.ComponentModel|4.3.0System.ComponentModel.EventBasedAsync|4.3.0System.ComponentModel.Primitives|4.3.0System.ComponentModel.TypeConverter|4.3.0System.Console|4.3.0System.Data.Common|4.3.0System.Diagnostics.Contracts|4.3.0System.Diagnostics.Debug|4.3.0System.Diagnostics.DiagnosticSource|4.4.0System.Diagnostics.FileVersionInfo|4.3.0System.Diagnostics.Process|4.3.0System.Diagnostics.StackTrace|4.3.0System.Diagnostics.TextWriterTraceListener|4.3.0System.Diagnostics.Tools|4.3.0System.Diagnostics.TraceSource|4.3.0System.Diagnostics.Tracing|4.3.0System.Dynamic.Runtime|4.3.0System.Globalization|4.3.0System.Globalization.Calendars|4.3.0System.Globalization.Extensions|4.3.0System.IO|4.3.0System.IO.Compression|4.3.0System.IO.Compression.ZipFile|4.3.0System.IO.FileSystem|4.3.0System.IO.FileSystem.AccessControl|4.4.0System.IO.FileSystem.DriveInfo|4.3.0System.IO.FileSystem.Primitives|4.3.0System.IO.FileSystem.Watcher|4.3.0System.IO.IsolatedStorage|4.3.0System.IO.MemoryMappedFiles|4.3.0System.IO.Pipes|4.3.0System.IO.UnmanagedMemoryStream|4.3.0System.Linq|4.3.0System.Linq.Expressions|4.3.0System.Linq.Queryable|4.3.0System.Net.Http|4.3.0System.Net.NameResolution|4.3.0System.Net.Primitives|4.3.0System.Net.Requests|4.3.0System.Net.Security|4.3.0System.Net.Sockets|4.3.0System.Net.WebHeaderCollection|4.3.0System.ObjectModel|4.3.0System.Private.DataContractSerialization|4.3.0System.Reflection|4.3.0System.Reflection.Emit|4.3.0System.Reflection.Emit.ILGeneration|4.3.0System.Reflection.Emit.Lightweight|4.3.0System.Reflection.Extensions|4.3.0System.Reflection.Metadata|1.5.0System.Reflection.Primitives|4.3.0System.Reflection.TypeExtensions|4.3.0System.Resources.ResourceManager|4.3.0System.Runtime|4.3.0System.Runtime.Extensions|4.3.0System.Runtime.Handles|4.3.0System.Runtime.InteropServices|4.3.0System.Runtime.InteropServices.RuntimeInformation|4.3.0System.Runtime.Loader|4.3.0System.Runtime.Numerics|4.3.0System.Runtime.Serialization.Formatters|4.3.0System.Runtime.Serialization.Json|4.3.0System.Runtime.Serialization.Primitives|4.3.0System.Security.AccessControl|4.4.0System.Security.Claims|4.3.0System.Security.Cryptography.Algorithms|4.3.0System.Security.Cryptography.Cng|4.4.0System.Security.Cryptography.Csp|4.3.0System.Security.Cryptography.Encoding|4.3.0System.Security.Cryptography.OpenSsl|4.4.0System.Security.Cryptography.Primitives|4.3.0System.Security.Cryptography.X509Certificates|4.3.0System.Security.Cryptography.Xml|4.4.0System.Security.Principal|4.3.0System.Security.Principal.Windows|4.4.0System.Text.Encoding|4.3.0System.Text.Encoding.Extensions|4.3.0System.Text.RegularExpressions|4.3.0System.Threading|4.3.0System.Threading.Overlapped|4.3.0System.Threading.Tasks|4.3.0System.Threading.Tasks.Extensions|4.3.0System.Threading.Tasks.Parallel|4.3.0System.Threading.Thread|4.3.0System.Threading.ThreadPool|4.3.0System.Threading.Timer|4.3.0System.ValueTuple|4.3.0System.Xml.ReaderWriter|4.3.0System.Xml.XDocument|4.3.0System.Xml.XmlDocument|4.3.0System.Xml.XmlSerializer|4.3.0System.Xml.XPath|4.3.0System.Xml.XPath.XDocument|4.3.0

So the problem is that the PackageOverrides.txt file defined by the shared framework is not being used by the SDK /cc @dsplaisted@dagood

Will keep looking for root cause of that and a workaround.

[ericstj]

Ok, it looks like the new info is read, but is garbled. The PackageConflictOverrides returned from the ResolveTargetingPackAssets task has all package IDs running together. This means that all the content in PackageOverrides.txt is effectively ignored.

[ericstj]

Opened the bug here: dotnet/sdk#10904

To workaround you can run a target that fixes it, stay tuned and will share that.

[ericstj]

Ok, I tried to workaround the PackageOverride reading bug, but then encountered another issue, so reactivated dotnet/runtime#33450

So I took another tactic at fixing this: just avoiding the reference to the 1.x era packages. I found these were coming in transitively from a number of packages (Moq, xunit, etc) which reference NETStandard.Library 1.6.0. This is issue dotnet/sdk#3044. I'll workaround it by updating that transitive reference.

[JunTaoLuo]

FYI responding to an earlier conversation, there's no ingestion of dotnet/runtime packages (e.g. primitives) in this PR. I've paused that work until all packages are brought up in dotnet/runtime.

Edit: I see everyone has already figured this out, sorry for the spam

[dotnet-maestro]

Auto-Merge Status

This pull request will be merged because the following merge policies have succeeded.

• ✔️ Standard Merge Policies Succeeded - Successful checks: extensions-ci, extensions-ci (Build Ubuntu 16.04 Release), extensions-ci (Build Ubuntu 16.04 Debug), extensions-ci (Build Test: Linux Source Build), extensions-ci (Build OSX Release), extensions-ci (Build OSX Debug), extensions-ci (Build Windows Release), extensions-ci (Build Windows Debug), extensions-ci (Build Code check)
• ✔️ Standard Merge Policies Succeeded - No reviews have requested changes.

[safern]

Thanks @ericstj

[JunTaoLuo]

🎉

[dotnet-maestro]

This pull request updates the following dependencies

From https://github.com/dotnet/runtime

• Subscription: 8542892c-8237-4ac0-12f9-08d795579ebc
• Build: 20200317.12
• Date Produced: 3/18/2020 9:42 AM
• Commit: c0ddd1c5d1636de873398c8d9544e02289f95fec
• Branch: refs/heads/master
• Updates:
  • Microsoft.NETCore.App.Runtime.win-x64 -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Ref -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Internal -> 5.0.0-preview.3.20167.12

Coherency Updates

The following updates ensure that dependencies with a CoherentParentDependency
attribute were produced in a build used as input to the parent dependency's build.
See Dependency Description Format

• Microsoft.Win32.Registry -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ComponentModel.Annotations -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.DiagnosticSource -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.EventLog -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.IO.Pipelines -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Reflection.Metadata -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Runtime.CompilerServices.Unsafe -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Cng -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Xml -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ServiceProcess.ServiceController -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Encodings.Web -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Json -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• Microsoft.NETCore.Platforms -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)

From https://github.com/dotnet/roslyn

• Subscription: d9874afa-852c-4c5b-12f8-08d795579ebc
• Build: 20200316.3
• Date Produced: 3/16/2020 6:04 PM
• Commit: c0fc88e0ddd03abbdac8fc7e0d280b70d7208382
• Branch: refs/heads/master-vs-deps
• Updates:
  • Microsoft.Net.Compilers.Toolset -> 3.6.0-2.20166.3

From https://github.com/dotnet/arcade

• Subscription: 455063ee-4723-4484-e1e7-08d79559e508
• Build: 20200312.3
• Date Produced: 3/12/2020 5:55 PM
• Commit: 16d4350184cf362cd59807b589d1c93803025abc
• Branch: refs/heads/master
• Updates:
  • Microsoft.DotNet.Arcade.Sdk -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.GenAPI -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.Helix.Sdk -> 5.0.0-beta.20162.3

[ghost]

Auto-approving dependency update.

[pranavkm]

@dagood this error message showed up again:

The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Previous discussions here: dotnet/runtime#1129.

[dagood]

Previous discussions here: dotnet/runtime#1129.

To sum up, if it's the same issue, I would expect to see a data/PlatformManifest.txt in https://dev.azure.com/dnceng/internal/_build/results?buildId=553529&view=resultsPackageArtifacts/Microsoft.NETCore.App.Ref.5.0.0-preview.3.20159.13.nupkg that's missing some data, but I don't see anything obvious missing.

There are some "odd" names in the manifest with lib prefixes like libSystem.Security.Cryptography.Native.OpenSsl.so|Microsoft.NETCore.App.Ref||0.0.0.0 that I remember seeing some commotion over in a dotnet/runtime issue, but I haven't read through that.

@dotnet/runtime-infrastructure PTAL, I might be missing context on recent changes and it might be one of the other causes listed in that issue that I'm not very familiar with anyway (openssl version etc.)

[safern]

cc: @bartonjs@janvorli as they might see something obvious we might be missing here.

[bartonjs]

-4305 is kCCUnimplemented (https://opensource.apple.com/source/CommonCrypto/CommonCrypto-36064/CommonCrypto/CommonCryptor.h.auto.html). The last time we had a problem with that was fixed in 2017. Maybe something is being weird in library binding and we're finding an old copy? Is there a good way to figure out where our shim got loaded from on the CI machines?

AESKeyWrapEncrypt /does/ use ECB mode, so it /could/ be the problems that we fixed in dotnet/corefx#21631 ... if we're finding an old version of the library.

[pranavkm]

We fairly recently changed the OS version for the MacOS version. @dougbu would have more specifics about it. Could that affect this in any way?

[jkoritzinsky]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

[bartonjs]

We fairly recently changed the OS version for the MacOS version. ... Could that affect this in any way?

If you bumped to 10.15, yes (I don't think dotnet/runtime has that in CI yet because the images weren't ready/reliable). If you're 10.14 then the most likely culprit is the shim library rename, since that would be something you'd probably just now be ingesting (and dotnet/runtime isn't failing on 10.14).

[BrennanConroy]

We're on 10.14 now

[safern]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

That seems like a possible issue. I think the easiest would be to pull this and debug locally. Trying now to see what System.Security.Cryptography.Native.so it is loading.

[ericstj]

Agreed with the assesment of @jkoritzinsky and @safern. In 1.x we shipped nuget packages with these shims and rely on conflict resolution to make the framework copies win.

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

[pranavkm]

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

[ericstj]

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

Not exactly. The only way you'd get an old System.Native.* is if you had a reference to a 1.x era package. You'd need to find out which project was bringing that in and add the reference there. System.Security.Cryptography.Xml wouldn't be the culprit, it's probably some 3rd party package with very old dependencies.

[wtgodbe]

Oh, could be - we see transient failures w/ Pdb2Pdb every once in a while: https://github.com/dotnet/aspnetcore-internal/issues/3484

[wtgodbe]

Windows failures are gone on the second attempt, but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

And:

Interop+AppleCrypto+AppleCommonCryptoCryptographicException : The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Which is the same error as before...

[BrennanConroy]

but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

No, those are gone on the second attempt. You're looking at the test tab which will show failures across all retries.

@ericstj@safern Could we get some help with the crypto errors that haven't gone away with the Runtime fix?

[ericstj]

We had discussed that we'll hold off on consuming the extensions from dotnet/runtime to avoid too much churn. @JunTaoLuo can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

[BrennanConroy]

Can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

[ericstj]

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

Right you are, sorry @safern and I had assumed that was going on here, it would have caused similar errors (FileLoadExcetion due to two different assemblies with same identity). I don't think that's the case, so just ignore my comment.

Nevermind, just me, crossed wires. Let me look at crypto failures.

[ericstj]

Here's what I see in the binlog.

The following is referenced by the test project:

Native shim references

/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.apple/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
RuntimeIdentifier = osx.10.10-x64
/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.openssl/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
RuntimeIdentifier = osx.10.10-x64

PackageOverrides

PackageOverrides
NETStandard.Library
OverriddenPackages = Microsoft.Win32.Primitives|4.3.0;
System.AppContext|4.3.0;
System.Collections|4.3.0;
System.Collections.Concurrent|4.3.0;
System.Collections.Immutable|1.4.0;
System.Collections.NonGeneric|4.3.0;
System.Collections.Specialized|4.3.0;
System.ComponentModel|4.3.0;
System.ComponentModel.EventBasedAsync|4.3.0;
System.ComponentModel.Primitives|4.3.0;
System.ComponentModel.TypeConverter|4.3.0;
System.Console|4.3.0;
System.Data.Common|4.3.0;
System.Diagnostics.Contracts|4.3.0;
System.Diagnostics.Debug|4.3.0;
System.Diagnostics.FileVersionInfo|4.3.0;
System.Diagnostics.Process|4.3.0;
System.Diagnostics.StackTrace|4.3.0;
System.Diagnostics.TextWriterTraceListener|4.3.0;
System.Diagnostics.Tools|4.3.0;
System.Diagnostics.TraceSource|4.3.0;
System.Diagnostics.Tracing|4.3.0;
System.Dynamic.Runtime|4.3.0;
System.Globalization|4.3.0;
System.Globalization.Calendars|4.3.0;
System.Globalization.Extensions|4.3.0;
System.IO|4.3.0;
System.IO.Compression|4.3.0;
System.IO.Compression.ZipFile|4.3.0;
System.IO.FileSystem|4.3.0;
System.IO.FileSystem.DriveInfo|4.3.0;
System.IO.FileSystem.Primitives|4.3.0;
System.IO.FileSystem.Watcher|4.3.0;
System.IO.IsolatedStorage|4.3.0;
System.IO.MemoryMappedFiles|4.3.0;
System.IO.Pipes|4.3.0;
System.IO.UnmanagedMemoryStream|4.3.0;
System.Linq|4.3.0;
System.Linq.Expressions|4.3.0;
System.Linq.Queryable|4.3.0;
System.Net.Http|4.3.0;
System.Net.NameResolution|4.3.0;
System.Net.Primitives|4.3.0;
System.Net.Requests|4.3.0;
System.Net.Security|4.3.0;
System.Net.Sockets|4.3.0;
System.Net.WebHeaderCollection|4.3.0;
System.ObjectModel|4.3.0;
System.Private.DataContractSerialization|4.3.0;
System.Reflection|4.3.0;
System.Reflection.Emit|4.3.0;
System.Reflection.Emit.ILGeneration|4.3.0;
System.Reflection.Emit.Lightweight|4.3.0;
System.Reflection.Extensions|4.3.0;
System.Reflection.Primitives|4.3.0;
System.Reflection.TypeExtensions|4.3.0;
System.Resources.ResourceManager|4.3.0;
System.Runtime|4.3.0;
System.Runtime.Extensions|4.3.0;
System.Runtime.Handles|4.3.0;
System.Runtime.InteropServices|4.3.0;
System.Runtime.InteropServices.RuntimeInformation|4.3.0;
System.Runtime.Loader|4.3.0;
System.Runtime.Numerics|4.3.0;
System.Runtime.Serialization.Formatters|4.3.0;
System.Runtime.Serialization.Json|4.3.0;
System.Runtime.Serialization.Primitives|4.3.0;
System.Security.AccessControl|4.4.0;
System.Security.Claims|4.3.0;
System.Security.Cryptography.Algorithms|4.3.0;
System.Security.Cryptography.Csp|4.3.0;
System.Security.Cryptography.Encoding|4.3.0;
System.Security.Cryptography.Primitives|4.3.0;
System.Security.Cryptography.X509Certificates|4.3.0;
System.Security.Cryptography.Xml|4.4.0;
System.Security.Principal|4.3.0;
System.Security.Principal.Windows|4.4.0;
System.Text.Encoding|4.3.0;
System.Text.Encoding.Extensions|4.3.0;
System.Text.RegularExpressions|4.3.0;
System.Threading|4.3.0;
System.Threading.Overlapped|4.3.0;
System.Threading.Tasks|4.3.0;
System.Threading.Tasks.Extensions|4.3.0;
System.Threading.Tasks.Parallel|4.3.0;
System.Threading.Thread|4.3.0;
System.Threading.ThreadPool|4.3.0;
System.Threading.Timer|4.3.0;
System.ValueTuple|4.3.0;
System.Xml.ReaderWriter|4.3.0;
System.Xml.XDocument|4.3.0;
System.Xml.XmlDocument|4.3.0;
System.Xml.XmlSerializer|4.3.0;
System.Xml.XPath|4.3.0;
System.Xml.XPath.XDocument|4.3.0;
Microsoft.NETCore.App.Ref
OverriddenPackages = Microsoft.CSharp|4.4.0Microsoft.Win32.Primitives|4.3.0Microsoft.Win32.Registry|4.4.0runtime.debian.8-x64.runtime.native.System|4.3.0runtime.debian.8-x64.runtime.native.System.IO.Compression|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Http|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Security|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.23-x64.runtime.native.System|4.3.0runtime.fedora.23-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.24-x64.runtime.native.System|4.3.0runtime.fedora.24-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.13.2-x64.runtime.native.System|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.42.1-x64.runtime.native.System|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.osx.10.10-x64.runtime.native.System|4.3.0runtime.osx.10.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Http|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Security|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.rhel.7-x64.runtime.native.System|4.3.0runtime.rhel.7-x64.runtime.native.System.IO.Compression|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Http|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Security|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0System.AppContext|4.3.0System.Buffers|4.4.0System.Collections|4.3.0System.Collections.Concurrent|4.3.0System.Collections.Immutable|1.4.0System.Collections.NonGeneric|4.3.0System.Collections.Specialized|4.3.0System.ComponentModel|4.3.0System.ComponentModel.EventBasedAsync|4.3.0System.ComponentModel.Primitives|4.3.0System.ComponentModel.TypeConverter|4.3.0System.Console|4.3.0System.Data.Common|4.3.0System.Diagnostics.Contracts|4.3.0System.Diagnostics.Debug|4.3.0System.Diagnostics.DiagnosticSource|4.4.0System.Diagnostics.FileVersionInfo|4.3.0System.Diagnostics.Process|4.3.0System.Diagnostics.StackTrace|4.3.0System.Diagnostics.TextWriterTraceListener|4.3.0System.Diagnostics.Tools|4.3.0System.Diagnostics.TraceSource|4.3.0System.Diagnostics.Tracing|4.3.0System.Dynamic.Runtime|4.3.0System.Globalization|4.3.0System.Globalization.Calendars|4.3.0System.Globalization.Extensions|4.3.0System.IO|4.3.0System.IO.Compression|4.3.0System.IO.Compression.ZipFile|4.3.0System.IO.FileSystem|4.3.0System.IO.FileSystem.AccessControl|4.4.0System.IO.FileSystem.DriveInfo|4.3.0System.IO.FileSystem.Primitives|4.3.0System.IO.FileSystem.Watcher|4.3.0System.IO.IsolatedStorage|4.3.0System.IO.MemoryMappedFiles|4.3.0System.IO.Pipes|4.3.0System.IO.UnmanagedMemoryStream|4.3.0System.Linq|4.3.0System.Linq.Expressions|4.3.0System.Linq.Queryable|4.3.0System.Net.Http|4.3.0System.Net.NameResolution|4.3.0System.Net.Primitives|4.3.0System.Net.Requests|4.3.0System.Net.Security|4.3.0System.Net.Sockets|4.3.0System.Net.WebHeaderCollection|4.3.0System.ObjectModel|4.3.0System.Private.DataContractSerialization|4.3.0System.Reflection|4.3.0System.Reflection.Emit|4.3.0System.Reflection.Emit.ILGeneration|4.3.0System.Reflection.Emit.Lightweight|4.3.0System.Reflection.Extensions|4.3.0System.Reflection.Metadata|1.5.0System.Reflection.Primitives|4.3.0System.Reflection.TypeExtensions|4.3.0System.Resources.ResourceManager|4.3.0System.Runtime|4.3.0System.Runtime.Extensions|4.3.0System.Runtime.Handles|4.3.0System.Runtime.InteropServices|4.3.0System.Runtime.InteropServices.RuntimeInformation|4.3.0System.Runtime.Loader|4.3.0System.Runtime.Numerics|4.3.0System.Runtime.Serialization.Formatters|4.3.0System.Runtime.Serialization.Json|4.3.0System.Runtime.Serialization.Primitives|4.3.0System.Security.AccessControl|4.4.0System.Security.Claims|4.3.0System.Security.Cryptography.Algorithms|4.3.0System.Security.Cryptography.Cng|4.4.0System.Security.Cryptography.Csp|4.3.0System.Security.Cryptography.Encoding|4.3.0System.Security.Cryptography.OpenSsl|4.4.0System.Security.Cryptography.Primitives|4.3.0System.Security.Cryptography.X509Certificates|4.3.0System.Security.Cryptography.Xml|4.4.0System.Security.Principal|4.3.0System.Security.Principal.Windows|4.4.0System.Text.Encoding|4.3.0System.Text.Encoding.Extensions|4.3.0System.Text.RegularExpressions|4.3.0System.Threading|4.3.0System.Threading.Overlapped|4.3.0System.Threading.Tasks|4.3.0System.Threading.Tasks.Extensions|4.3.0System.Threading.Tasks.Parallel|4.3.0System.Threading.Thread|4.3.0System.Threading.ThreadPool|4.3.0System.Threading.Timer|4.3.0System.ValueTuple|4.3.0System.Xml.ReaderWriter|4.3.0System.Xml.XDocument|4.3.0System.Xml.XmlDocument|4.3.0System.Xml.XmlSerializer|4.3.0System.Xml.XPath|4.3.0System.Xml.XPath.XDocument|4.3.0

So the problem is that the PackageOverrides.txt file defined by the shared framework is not being used by the SDK /cc @dsplaisted@dagood

Will keep looking for root cause of that and a workaround.

[ericstj]

Ok, it looks like the new info is read, but is garbled. The PackageConflictOverrides returned from the ResolveTargetingPackAssets task has all package IDs running together. This means that all the content in PackageOverrides.txt is effectively ignored.

[ericstj]

Opened the bug here: dotnet/sdk#10904

To workaround you can run a target that fixes it, stay tuned and will share that.

[ericstj]

Ok, I tried to workaround the PackageOverride reading bug, but then encountered another issue, so reactivated dotnet/runtime#33450

So I took another tactic at fixing this: just avoiding the reference to the 1.x era packages. I found these were coming in transitively from a number of packages (Moq, xunit, etc) which reference NETStandard.Library 1.6.0. This is issue dotnet/sdk#3044. I'll workaround it by updating that transitive reference.

[JunTaoLuo]

FYI responding to an earlier conversation, there's no ingestion of dotnet/runtime packages (e.g. primitives) in this PR. I've paused that work until all packages are brought up in dotnet/runtime.

Edit: I see everyone has already figured this out, sorry for the spam

[dotnet-maestro]

Auto-Merge Status

This pull request will be merged because the following merge policies have succeeded.

• ✔️ Standard Merge Policies Succeeded - Successful checks: extensions-ci, extensions-ci (Build Ubuntu 16.04 Release), extensions-ci (Build Ubuntu 16.04 Debug), extensions-ci (Build Test: Linux Source Build), extensions-ci (Build OSX Release), extensions-ci (Build OSX Debug), extensions-ci (Build Windows Release), extensions-ci (Build Windows Debug), extensions-ci (Build Code check)
• ✔️ Standard Merge Policies Succeeded - No reviews have requested changes.

[safern]

Thanks @ericstj

[JunTaoLuo]

🎉

[dotnet-maestro]

This pull request updates the following dependencies

From https://github.com/dotnet/runtime

• Subscription: 8542892c-8237-4ac0-12f9-08d795579ebc
• Build: 20200317.12
• Date Produced: 3/18/2020 9:42 AM
• Commit: c0ddd1c5d1636de873398c8d9544e02289f95fec
• Branch: refs/heads/master
• Updates:
  • Microsoft.NETCore.App.Runtime.win-x64 -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Ref -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Internal -> 5.0.0-preview.3.20167.12

Coherency Updates

The following updates ensure that dependencies with a CoherentParentDependency
attribute were produced in a build used as input to the parent dependency's build.
See Dependency Description Format

• Microsoft.Win32.Registry -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ComponentModel.Annotations -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.DiagnosticSource -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.EventLog -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.IO.Pipelines -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Reflection.Metadata -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Runtime.CompilerServices.Unsafe -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Cng -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Xml -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ServiceProcess.ServiceController -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Encodings.Web -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Json -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• Microsoft.NETCore.Platforms -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)

From https://github.com/dotnet/roslyn

• Subscription: d9874afa-852c-4c5b-12f8-08d795579ebc
• Build: 20200316.3
• Date Produced: 3/16/2020 6:04 PM
• Commit: c0fc88e0ddd03abbdac8fc7e0d280b70d7208382
• Branch: refs/heads/master-vs-deps
• Updates:
  • Microsoft.Net.Compilers.Toolset -> 3.6.0-2.20166.3

From https://github.com/dotnet/arcade

• Subscription: 455063ee-4723-4484-e1e7-08d79559e508
• Build: 20200312.3
• Date Produced: 3/12/2020 5:55 PM
• Commit: 16d4350184cf362cd59807b589d1c93803025abc
• Branch: refs/heads/master
• Updates:
  • Microsoft.DotNet.Arcade.Sdk -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.GenAPI -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.Helix.Sdk -> 5.0.0-beta.20162.3

[ghost]

Auto-approving dependency update.

[pranavkm]

@dagood this error message showed up again:

The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Previous discussions here: dotnet/runtime#1129.

[dagood]

Previous discussions here: dotnet/runtime#1129.

To sum up, if it's the same issue, I would expect to see a data/PlatformManifest.txt in https://dev.azure.com/dnceng/internal/_build/results?buildId=553529&view=resultsPackageArtifacts/Microsoft.NETCore.App.Ref.5.0.0-preview.3.20159.13.nupkg that's missing some data, but I don't see anything obvious missing.

There are some "odd" names in the manifest with lib prefixes like libSystem.Security.Cryptography.Native.OpenSsl.so|Microsoft.NETCore.App.Ref||0.0.0.0 that I remember seeing some commotion over in a dotnet/runtime issue, but I haven't read through that.

@dotnet/runtime-infrastructure PTAL, I might be missing context on recent changes and it might be one of the other causes listed in that issue that I'm not very familiar with anyway (openssl version etc.)

[safern]

cc: @bartonjs@janvorli as they might see something obvious we might be missing here.

[bartonjs]

-4305 is kCCUnimplemented (https://opensource.apple.com/source/CommonCrypto/CommonCrypto-36064/CommonCrypto/CommonCryptor.h.auto.html). The last time we had a problem with that was fixed in 2017. Maybe something is being weird in library binding and we're finding an old copy? Is there a good way to figure out where our shim got loaded from on the CI machines?

AESKeyWrapEncrypt /does/ use ECB mode, so it /could/ be the problems that we fixed in dotnet/corefx#21631 ... if we're finding an old version of the library.

[pranavkm]

We fairly recently changed the OS version for the MacOS version. @dougbu would have more specifics about it. Could that affect this in any way?

[jkoritzinsky]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

[bartonjs]

We fairly recently changed the OS version for the MacOS version. ... Could that affect this in any way?

If you bumped to 10.15, yes (I don't think dotnet/runtime has that in CI yet because the images weren't ready/reliable). If you're 10.14 then the most likely culprit is the shim library rename, since that would be something you'd probably just now be ingesting (and dotnet/runtime isn't failing on 10.14).

[BrennanConroy]

We're on 10.14 now

[safern]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

That seems like a possible issue. I think the easiest would be to pull this and debug locally. Trying now to see what System.Security.Cryptography.Native.so it is loading.

[ericstj]

Agreed with the assesment of @jkoritzinsky and @safern. In 1.x we shipped nuget packages with these shims and rely on conflict resolution to make the framework copies win.

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

[pranavkm]

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

[ericstj]

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

Not exactly. The only way you'd get an old System.Native.* is if you had a reference to a 1.x era package. You'd need to find out which project was bringing that in and add the reference there. System.Security.Cryptography.Xml wouldn't be the culprit, it's probably some 3rd party package with very old dependencies.

[wtgodbe]

Oh, could be - we see transient failures w/ Pdb2Pdb every once in a while: https://github.com/dotnet/aspnetcore-internal/issues/3484

[wtgodbe]

Windows failures are gone on the second attempt, but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

And:

Interop+AppleCrypto+AppleCommonCryptoCryptographicException : The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Which is the same error as before...

[BrennanConroy]

but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

No, those are gone on the second attempt. You're looking at the test tab which will show failures across all retries.

@ericstj@safern Could we get some help with the crypto errors that haven't gone away with the Runtime fix?

[ericstj]

We had discussed that we'll hold off on consuming the extensions from dotnet/runtime to avoid too much churn. @JunTaoLuo can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

[BrennanConroy]

Can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

[ericstj]

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

Right you are, sorry @safern and I had assumed that was going on here, it would have caused similar errors (FileLoadExcetion due to two different assemblies with same identity). I don't think that's the case, so just ignore my comment.

Nevermind, just me, crossed wires. Let me look at crypto failures.

[ericstj]

Here's what I see in the binlog.

The following is referenced by the test project:

Native shim references

/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.apple/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
RuntimeIdentifier = osx.10.10-x64
/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.openssl/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
RuntimeIdentifier = osx.10.10-x64

PackageOverrides

PackageOverrides
NETStandard.Library
OverriddenPackages = Microsoft.Win32.Primitives|4.3.0;
System.AppContext|4.3.0;
System.Collections|4.3.0;
System.Collections.Concurrent|4.3.0;
System.Collections.Immutable|1.4.0;
System.Collections.NonGeneric|4.3.0;
System.Collections.Specialized|4.3.0;
System.ComponentModel|4.3.0;
System.ComponentModel.EventBasedAsync|4.3.0;
System.ComponentModel.Primitives|4.3.0;
System.ComponentModel.TypeConverter|4.3.0;
System.Console|4.3.0;
System.Data.Common|4.3.0;
System.Diagnostics.Contracts|4.3.0;
System.Diagnostics.Debug|4.3.0;
System.Diagnostics.FileVersionInfo|4.3.0;
System.Diagnostics.Process|4.3.0;
System.Diagnostics.StackTrace|4.3.0;
System.Diagnostics.TextWriterTraceListener|4.3.0;
System.Diagnostics.Tools|4.3.0;
System.Diagnostics.TraceSource|4.3.0;
System.Diagnostics.Tracing|4.3.0;
System.Dynamic.Runtime|4.3.0;
System.Globalization|4.3.0;
System.Globalization.Calendars|4.3.0;
System.Globalization.Extensions|4.3.0;
System.IO|4.3.0;
System.IO.Compression|4.3.0;
System.IO.Compression.ZipFile|4.3.0;
System.IO.FileSystem|4.3.0;
System.IO.FileSystem.DriveInfo|4.3.0;
System.IO.FileSystem.Primitives|4.3.0;
System.IO.FileSystem.Watcher|4.3.0;
System.IO.IsolatedStorage|4.3.0;
System.IO.MemoryMappedFiles|4.3.0;
System.IO.Pipes|4.3.0;
System.IO.UnmanagedMemoryStream|4.3.0;
System.Linq|4.3.0;
System.Linq.Expressions|4.3.0;
System.Linq.Queryable|4.3.0;
System.Net.Http|4.3.0;
System.Net.NameResolution|4.3.0;
System.Net.Primitives|4.3.0;
System.Net.Requests|4.3.0;
System.Net.Security|4.3.0;
System.Net.Sockets|4.3.0;
System.Net.WebHeaderCollection|4.3.0;
System.ObjectModel|4.3.0;
System.Private.DataContractSerialization|4.3.0;
System.Reflection|4.3.0;
System.Reflection.Emit|4.3.0;
System.Reflection.Emit.ILGeneration|4.3.0;
System.Reflection.Emit.Lightweight|4.3.0;
System.Reflection.Extensions|4.3.0;
System.Reflection.Primitives|4.3.0;
System.Reflection.TypeExtensions|4.3.0;
System.Resources.ResourceManager|4.3.0;
System.Runtime|4.3.0;
System.Runtime.Extensions|4.3.0;
System.Runtime.Handles|4.3.0;
System.Runtime.InteropServices|4.3.0;
System.Runtime.InteropServices.RuntimeInformation|4.3.0;
System.Runtime.Loader|4.3.0;
System.Runtime.Numerics|4.3.0;
System.Runtime.Serialization.Formatters|4.3.0;
System.Runtime.Serialization.Json|4.3.0;
System.Runtime.Serialization.Primitives|4.3.0;
System.Security.AccessControl|4.4.0;
System.Security.Claims|4.3.0;
System.Security.Cryptography.Algorithms|4.3.0;
System.Security.Cryptography.Csp|4.3.0;
System.Security.Cryptography.Encoding|4.3.0;
System.Security.Cryptography.Primitives|4.3.0;
System.Security.Cryptography.X509Certificates|4.3.0;
System.Security.Cryptography.Xml|4.4.0;
System.Security.Principal|4.3.0;
System.Security.Principal.Windows|4.4.0;
System.Text.Encoding|4.3.0;
System.Text.Encoding.Extensions|4.3.0;
System.Text.RegularExpressions|4.3.0;
System.Threading|4.3.0;
System.Threading.Overlapped|4.3.0;
System.Threading.Tasks|4.3.0;
System.Threading.Tasks.Extensions|4.3.0;
System.Threading.Tasks.Parallel|4.3.0;
System.Threading.Thread|4.3.0;
System.Threading.ThreadPool|4.3.0;
System.Threading.Timer|4.3.0;
System.ValueTuple|4.3.0;
System.Xml.ReaderWriter|4.3.0;
System.Xml.XDocument|4.3.0;
System.Xml.XmlDocument|4.3.0;
System.Xml.XmlSerializer|4.3.0;
System.Xml.XPath|4.3.0;
System.Xml.XPath.XDocument|4.3.0;
Microsoft.NETCore.App.Ref
OverriddenPackages = Microsoft.CSharp|4.4.0Microsoft.Win32.Primitives|4.3.0Microsoft.Win32.Registry|4.4.0runtime.debian.8-x64.runtime.native.System|4.3.0runtime.debian.8-x64.runtime.native.System.IO.Compression|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Http|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Security|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.23-x64.runtime.native.System|4.3.0runtime.fedora.23-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.24-x64.runtime.native.System|4.3.0runtime.fedora.24-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.13.2-x64.runtime.native.System|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.42.1-x64.runtime.native.System|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.osx.10.10-x64.runtime.native.System|4.3.0runtime.osx.10.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Http|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Security|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.rhel.7-x64.runtime.native.System|4.3.0runtime.rhel.7-x64.runtime.native.System.IO.Compression|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Http|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Security|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0System.AppContext|4.3.0System.Buffers|4.4.0System.Collections|4.3.0System.Collections.Concurrent|4.3.0System.Collections.Immutable|1.4.0System.Collections.NonGeneric|4.3.0System.Collections.Specialized|4.3.0System.ComponentModel|4.3.0System.ComponentModel.EventBasedAsync|4.3.0System.ComponentModel.Primitives|4.3.0System.ComponentModel.TypeConverter|4.3.0System.Console|4.3.0System.Data.Common|4.3.0System.Diagnostics.Contracts|4.3.0System.Diagnostics.Debug|4.3.0System.Diagnostics.DiagnosticSource|4.4.0System.Diagnostics.FileVersionInfo|4.3.0System.Diagnostics.Process|4.3.0System.Diagnostics.StackTrace|4.3.0System.Diagnostics.TextWriterTraceListener|4.3.0System.Diagnostics.Tools|4.3.0System.Diagnostics.TraceSource|4.3.0System.Diagnostics.Tracing|4.3.0System.Dynamic.Runtime|4.3.0System.Globalization|4.3.0System.Globalization.Calendars|4.3.0System.Globalization.Extensions|4.3.0System.IO|4.3.0System.IO.Compression|4.3.0System.IO.Compression.ZipFile|4.3.0System.IO.FileSystem|4.3.0System.IO.FileSystem.AccessControl|4.4.0System.IO.FileSystem.DriveInfo|4.3.0System.IO.FileSystem.Primitives|4.3.0System.IO.FileSystem.Watcher|4.3.0System.IO.IsolatedStorage|4.3.0System.IO.MemoryMappedFiles|4.3.0System.IO.Pipes|4.3.0System.IO.UnmanagedMemoryStream|4.3.0System.Linq|4.3.0System.Linq.Expressions|4.3.0System.Linq.Queryable|4.3.0System.Net.Http|4.3.0System.Net.NameResolution|4.3.0System.Net.Primitives|4.3.0System.Net.Requests|4.3.0System.Net.Security|4.3.0System.Net.Sockets|4.3.0System.Net.WebHeaderCollection|4.3.0System.ObjectModel|4.3.0System.Private.DataContractSerialization|4.3.0System.Reflection|4.3.0System.Reflection.Emit|4.3.0System.Reflection.Emit.ILGeneration|4.3.0System.Reflection.Emit.Lightweight|4.3.0System.Reflection.Extensions|4.3.0System.Reflection.Metadata|1.5.0System.Reflection.Primitives|4.3.0System.Reflection.TypeExtensions|4.3.0System.Resources.ResourceManager|4.3.0System.Runtime|4.3.0System.Runtime.Extensions|4.3.0System.Runtime.Handles|4.3.0System.Runtime.InteropServices|4.3.0System.Runtime.InteropServices.RuntimeInformation|4.3.0System.Runtime.Loader|4.3.0System.Runtime.Numerics|4.3.0System.Runtime.Serialization.Formatters|4.3.0System.Runtime.Serialization.Json|4.3.0System.Runtime.Serialization.Primitives|4.3.0System.Security.AccessControl|4.4.0System.Security.Claims|4.3.0System.Security.Cryptography.Algorithms|4.3.0System.Security.Cryptography.Cng|4.4.0System.Security.Cryptography.Csp|4.3.0System.Security.Cryptography.Encoding|4.3.0System.Security.Cryptography.OpenSsl|4.4.0System.Security.Cryptography.Primitives|4.3.0System.Security.Cryptography.X509Certificates|4.3.0System.Security.Cryptography.Xml|4.4.0System.Security.Principal|4.3.0System.Security.Principal.Windows|4.4.0System.Text.Encoding|4.3.0System.Text.Encoding.Extensions|4.3.0System.Text.RegularExpressions|4.3.0System.Threading|4.3.0System.Threading.Overlapped|4.3.0System.Threading.Tasks|4.3.0System.Threading.Tasks.Extensions|4.3.0System.Threading.Tasks.Parallel|4.3.0System.Threading.Thread|4.3.0System.Threading.ThreadPool|4.3.0System.Threading.Timer|4.3.0System.ValueTuple|4.3.0System.Xml.ReaderWriter|4.3.0System.Xml.XDocument|4.3.0System.Xml.XmlDocument|4.3.0System.Xml.XmlSerializer|4.3.0System.Xml.XPath|4.3.0System.Xml.XPath.XDocument|4.3.0

So the problem is that the PackageOverrides.txt file defined by the shared framework is not being used by the SDK /cc @dsplaisted@dagood

Will keep looking for root cause of that and a workaround.

[ericstj]

Ok, it looks like the new info is read, but is garbled. The PackageConflictOverrides returned from the ResolveTargetingPackAssets task has all package IDs running together. This means that all the content in PackageOverrides.txt is effectively ignored.

[ericstj]

Opened the bug here: dotnet/sdk#10904

To workaround you can run a target that fixes it, stay tuned and will share that.

[ericstj]

Ok, I tried to workaround the PackageOverride reading bug, but then encountered another issue, so reactivated dotnet/runtime#33450

So I took another tactic at fixing this: just avoiding the reference to the 1.x era packages. I found these were coming in transitively from a number of packages (Moq, xunit, etc) which reference NETStandard.Library 1.6.0. This is issue dotnet/sdk#3044. I'll workaround it by updating that transitive reference.

[JunTaoLuo]

FYI responding to an earlier conversation, there's no ingestion of dotnet/runtime packages (e.g. primitives) in this PR. I've paused that work until all packages are brought up in dotnet/runtime.

Edit: I see everyone has already figured this out, sorry for the spam

[dotnet-maestro]

Auto-Merge Status

This pull request will be merged because the following merge policies have succeeded.

• ✔️ Standard Merge Policies Succeeded - Successful checks: extensions-ci, extensions-ci (Build Ubuntu 16.04 Release), extensions-ci (Build Ubuntu 16.04 Debug), extensions-ci (Build Test: Linux Source Build), extensions-ci (Build OSX Release), extensions-ci (Build OSX Debug), extensions-ci (Build Windows Release), extensions-ci (Build Windows Debug), extensions-ci (Build Code check)
• ✔️ Standard Merge Policies Succeeded - No reviews have requested changes.

[safern]

Thanks @ericstj

[JunTaoLuo]

🎉

[dotnet-maestro]

This pull request updates the following dependencies

From https://github.com/dotnet/runtime

• Subscription: 8542892c-8237-4ac0-12f9-08d795579ebc
• Build: 20200317.12
• Date Produced: 3/18/2020 9:42 AM
• Commit: c0ddd1c5d1636de873398c8d9544e02289f95fec
• Branch: refs/heads/master
• Updates:
  • Microsoft.NETCore.App.Runtime.win-x64 -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Ref -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Internal -> 5.0.0-preview.3.20167.12

Coherency Updates

The following updates ensure that dependencies with a CoherentParentDependency
attribute were produced in a build used as input to the parent dependency's build.
See Dependency Description Format

• Microsoft.Win32.Registry -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ComponentModel.Annotations -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.DiagnosticSource -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.EventLog -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.IO.Pipelines -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Reflection.Metadata -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Runtime.CompilerServices.Unsafe -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Cng -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Xml -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ServiceProcess.ServiceController -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Encodings.Web -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Json -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• Microsoft.NETCore.Platforms -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)

From https://github.com/dotnet/roslyn

• Subscription: d9874afa-852c-4c5b-12f8-08d795579ebc
• Build: 20200316.3
• Date Produced: 3/16/2020 6:04 PM
• Commit: c0fc88e0ddd03abbdac8fc7e0d280b70d7208382
• Branch: refs/heads/master-vs-deps
• Updates:
  • Microsoft.Net.Compilers.Toolset -> 3.6.0-2.20166.3

From https://github.com/dotnet/arcade

• Subscription: 455063ee-4723-4484-e1e7-08d79559e508
• Build: 20200312.3
• Date Produced: 3/12/2020 5:55 PM
• Commit: 16d4350184cf362cd59807b589d1c93803025abc
• Branch: refs/heads/master
• Updates:
  • Microsoft.DotNet.Arcade.Sdk -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.GenAPI -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.Helix.Sdk -> 5.0.0-beta.20162.3

[ghost]

Auto-approving dependency update.

[pranavkm]

@dagood this error message showed up again:

The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Previous discussions here: dotnet/runtime#1129.

[dagood]

Previous discussions here: dotnet/runtime#1129.

To sum up, if it's the same issue, I would expect to see a data/PlatformManifest.txt in https://dev.azure.com/dnceng/internal/_build/results?buildId=553529&view=resultsPackageArtifacts/Microsoft.NETCore.App.Ref.5.0.0-preview.3.20159.13.nupkg that's missing some data, but I don't see anything obvious missing.

There are some "odd" names in the manifest with lib prefixes like libSystem.Security.Cryptography.Native.OpenSsl.so|Microsoft.NETCore.App.Ref||0.0.0.0 that I remember seeing some commotion over in a dotnet/runtime issue, but I haven't read through that.

@dotnet/runtime-infrastructure PTAL, I might be missing context on recent changes and it might be one of the other causes listed in that issue that I'm not very familiar with anyway (openssl version etc.)

[safern]

cc: @bartonjs@janvorli as they might see something obvious we might be missing here.

[bartonjs]

-4305 is kCCUnimplemented (https://opensource.apple.com/source/CommonCrypto/CommonCrypto-36064/CommonCrypto/CommonCryptor.h.auto.html). The last time we had a problem with that was fixed in 2017. Maybe something is being weird in library binding and we're finding an old copy? Is there a good way to figure out where our shim got loaded from on the CI machines?

AESKeyWrapEncrypt /does/ use ECB mode, so it /could/ be the problems that we fixed in dotnet/corefx#21631 ... if we're finding an old version of the library.

[pranavkm]

We fairly recently changed the OS version for the MacOS version. @dougbu would have more specifics about it. Could that affect this in any way?

[jkoritzinsky]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

[bartonjs]

We fairly recently changed the OS version for the MacOS version. ... Could that affect this in any way?

If you bumped to 10.15, yes (I don't think dotnet/runtime has that in CI yet because the images weren't ready/reliable). If you're 10.14 then the most likely culprit is the shim library rename, since that would be something you'd probably just now be ingesting (and dotnet/runtime isn't failing on 10.14).

[BrennanConroy]

We're on 10.14 now

[safern]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

That seems like a possible issue. I think the easiest would be to pull this and debug locally. Trying now to see what System.Security.Cryptography.Native.so it is loading.

[ericstj]

Agreed with the assesment of @jkoritzinsky and @safern. In 1.x we shipped nuget packages with these shims and rely on conflict resolution to make the framework copies win.

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

[pranavkm]

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

[ericstj]

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

Not exactly. The only way you'd get an old System.Native.* is if you had a reference to a 1.x era package. You'd need to find out which project was bringing that in and add the reference there. System.Security.Cryptography.Xml wouldn't be the culprit, it's probably some 3rd party package with very old dependencies.

[wtgodbe]

Oh, could be - we see transient failures w/ Pdb2Pdb every once in a while: https://github.com/dotnet/aspnetcore-internal/issues/3484

[wtgodbe]

Windows failures are gone on the second attempt, but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

And:

Interop+AppleCrypto+AppleCommonCryptoCryptographicException : The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Which is the same error as before...

[BrennanConroy]

but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

No, those are gone on the second attempt. You're looking at the test tab which will show failures across all retries.

@ericstj@safern Could we get some help with the crypto errors that haven't gone away with the Runtime fix?

[ericstj]

We had discussed that we'll hold off on consuming the extensions from dotnet/runtime to avoid too much churn. @JunTaoLuo can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

[BrennanConroy]

Can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

[ericstj]

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

Right you are, sorry @safern and I had assumed that was going on here, it would have caused similar errors (FileLoadExcetion due to two different assemblies with same identity). I don't think that's the case, so just ignore my comment.

Nevermind, just me, crossed wires. Let me look at crypto failures.

[ericstj]

Here's what I see in the binlog.

The following is referenced by the test project:

Native shim references

/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.apple/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
RuntimeIdentifier = osx.10.10-x64
/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.openssl/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
RuntimeIdentifier = osx.10.10-x64

PackageOverrides

PackageOverrides
NETStandard.Library
OverriddenPackages = Microsoft.Win32.Primitives|4.3.0;
System.AppContext|4.3.0;
System.Collections|4.3.0;
System.Collections.Concurrent|4.3.0;
System.Collections.Immutable|1.4.0;
System.Collections.NonGeneric|4.3.0;
System.Collections.Specialized|4.3.0;
System.ComponentModel|4.3.0;
System.ComponentModel.EventBasedAsync|4.3.0;
System.ComponentModel.Primitives|4.3.0;
System.ComponentModel.TypeConverter|4.3.0;
System.Console|4.3.0;
System.Data.Common|4.3.0;
System.Diagnostics.Contracts|4.3.0;
System.Diagnostics.Debug|4.3.0;
System.Diagnostics.FileVersionInfo|4.3.0;
System.Diagnostics.Process|4.3.0;
System.Diagnostics.StackTrace|4.3.0;
System.Diagnostics.TextWriterTraceListener|4.3.0;
System.Diagnostics.Tools|4.3.0;
System.Diagnostics.TraceSource|4.3.0;
System.Diagnostics.Tracing|4.3.0;
System.Dynamic.Runtime|4.3.0;
System.Globalization|4.3.0;
System.Globalization.Calendars|4.3.0;
System.Globalization.Extensions|4.3.0;
System.IO|4.3.0;
System.IO.Compression|4.3.0;
System.IO.Compression.ZipFile|4.3.0;
System.IO.FileSystem|4.3.0;
System.IO.FileSystem.DriveInfo|4.3.0;
System.IO.FileSystem.Primitives|4.3.0;
System.IO.FileSystem.Watcher|4.3.0;
System.IO.IsolatedStorage|4.3.0;
System.IO.MemoryMappedFiles|4.3.0;
System.IO.Pipes|4.3.0;
System.IO.UnmanagedMemoryStream|4.3.0;
System.Linq|4.3.0;
System.Linq.Expressions|4.3.0;
System.Linq.Queryable|4.3.0;
System.Net.Http|4.3.0;
System.Net.NameResolution|4.3.0;
System.Net.Primitives|4.3.0;
System.Net.Requests|4.3.0;
System.Net.Security|4.3.0;
System.Net.Sockets|4.3.0;
System.Net.WebHeaderCollection|4.3.0;
System.ObjectModel|4.3.0;
System.Private.DataContractSerialization|4.3.0;
System.Reflection|4.3.0;
System.Reflection.Emit|4.3.0;
System.Reflection.Emit.ILGeneration|4.3.0;
System.Reflection.Emit.Lightweight|4.3.0;
System.Reflection.Extensions|4.3.0;
System.Reflection.Primitives|4.3.0;
System.Reflection.TypeExtensions|4.3.0;
System.Resources.ResourceManager|4.3.0;
System.Runtime|4.3.0;
System.Runtime.Extensions|4.3.0;
System.Runtime.Handles|4.3.0;
System.Runtime.InteropServices|4.3.0;
System.Runtime.InteropServices.RuntimeInformation|4.3.0;
System.Runtime.Loader|4.3.0;
System.Runtime.Numerics|4.3.0;
System.Runtime.Serialization.Formatters|4.3.0;
System.Runtime.Serialization.Json|4.3.0;
System.Runtime.Serialization.Primitives|4.3.0;
System.Security.AccessControl|4.4.0;
System.Security.Claims|4.3.0;
System.Security.Cryptography.Algorithms|4.3.0;
System.Security.Cryptography.Csp|4.3.0;
System.Security.Cryptography.Encoding|4.3.0;
System.Security.Cryptography.Primitives|4.3.0;
System.Security.Cryptography.X509Certificates|4.3.0;
System.Security.Cryptography.Xml|4.4.0;
System.Security.Principal|4.3.0;
System.Security.Principal.Windows|4.4.0;
System.Text.Encoding|4.3.0;
System.Text.Encoding.Extensions|4.3.0;
System.Text.RegularExpressions|4.3.0;
System.Threading|4.3.0;
System.Threading.Overlapped|4.3.0;
System.Threading.Tasks|4.3.0;
System.Threading.Tasks.Extensions|4.3.0;
System.Threading.Tasks.Parallel|4.3.0;
System.Threading.Thread|4.3.0;
System.Threading.ThreadPool|4.3.0;
System.Threading.Timer|4.3.0;
System.ValueTuple|4.3.0;
System.Xml.ReaderWriter|4.3.0;
System.Xml.XDocument|4.3.0;
System.Xml.XmlDocument|4.3.0;
System.Xml.XmlSerializer|4.3.0;
System.Xml.XPath|4.3.0;
System.Xml.XPath.XDocument|4.3.0;
Microsoft.NETCore.App.Ref
OverriddenPackages = Microsoft.CSharp|4.4.0Microsoft.Win32.Primitives|4.3.0Microsoft.Win32.Registry|4.4.0runtime.debian.8-x64.runtime.native.System|4.3.0runtime.debian.8-x64.runtime.native.System.IO.Compression|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Http|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Security|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.23-x64.runtime.native.System|4.3.0runtime.fedora.23-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.24-x64.runtime.native.System|4.3.0runtime.fedora.24-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.13.2-x64.runtime.native.System|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.42.1-x64.runtime.native.System|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.osx.10.10-x64.runtime.native.System|4.3.0runtime.osx.10.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Http|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Security|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.rhel.7-x64.runtime.native.System|4.3.0runtime.rhel.7-x64.runtime.native.System.IO.Compression|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Http|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Security|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0System.AppContext|4.3.0System.Buffers|4.4.0System.Collections|4.3.0System.Collections.Concurrent|4.3.0System.Collections.Immutable|1.4.0System.Collections.NonGeneric|4.3.0System.Collections.Specialized|4.3.0System.ComponentModel|4.3.0System.ComponentModel.EventBasedAsync|4.3.0System.ComponentModel.Primitives|4.3.0System.ComponentModel.TypeConverter|4.3.0System.Console|4.3.0System.Data.Common|4.3.0System.Diagnostics.Contracts|4.3.0System.Diagnostics.Debug|4.3.0System.Diagnostics.DiagnosticSource|4.4.0System.Diagnostics.FileVersionInfo|4.3.0System.Diagnostics.Process|4.3.0System.Diagnostics.StackTrace|4.3.0System.Diagnostics.TextWriterTraceListener|4.3.0System.Diagnostics.Tools|4.3.0System.Diagnostics.TraceSource|4.3.0System.Diagnostics.Tracing|4.3.0System.Dynamic.Runtime|4.3.0System.Globalization|4.3.0System.Globalization.Calendars|4.3.0System.Globalization.Extensions|4.3.0System.IO|4.3.0System.IO.Compression|4.3.0System.IO.Compression.ZipFile|4.3.0System.IO.FileSystem|4.3.0System.IO.FileSystem.AccessControl|4.4.0System.IO.FileSystem.DriveInfo|4.3.0System.IO.FileSystem.Primitives|4.3.0System.IO.FileSystem.Watcher|4.3.0System.IO.IsolatedStorage|4.3.0System.IO.MemoryMappedFiles|4.3.0System.IO.Pipes|4.3.0System.IO.UnmanagedMemoryStream|4.3.0System.Linq|4.3.0System.Linq.Expressions|4.3.0System.Linq.Queryable|4.3.0System.Net.Http|4.3.0System.Net.NameResolution|4.3.0System.Net.Primitives|4.3.0System.Net.Requests|4.3.0System.Net.Security|4.3.0System.Net.Sockets|4.3.0System.Net.WebHeaderCollection|4.3.0System.ObjectModel|4.3.0System.Private.DataContractSerialization|4.3.0System.Reflection|4.3.0System.Reflection.Emit|4.3.0System.Reflection.Emit.ILGeneration|4.3.0System.Reflection.Emit.Lightweight|4.3.0System.Reflection.Extensions|4.3.0System.Reflection.Metadata|1.5.0System.Reflection.Primitives|4.3.0System.Reflection.TypeExtensions|4.3.0System.Resources.ResourceManager|4.3.0System.Runtime|4.3.0System.Runtime.Extensions|4.3.0System.Runtime.Handles|4.3.0System.Runtime.InteropServices|4.3.0System.Runtime.InteropServices.RuntimeInformation|4.3.0System.Runtime.Loader|4.3.0System.Runtime.Numerics|4.3.0System.Runtime.Serialization.Formatters|4.3.0System.Runtime.Serialization.Json|4.3.0System.Runtime.Serialization.Primitives|4.3.0System.Security.AccessControl|4.4.0System.Security.Claims|4.3.0System.Security.Cryptography.Algorithms|4.3.0System.Security.Cryptography.Cng|4.4.0System.Security.Cryptography.Csp|4.3.0System.Security.Cryptography.Encoding|4.3.0System.Security.Cryptography.OpenSsl|4.4.0System.Security.Cryptography.Primitives|4.3.0System.Security.Cryptography.X509Certificates|4.3.0System.Security.Cryptography.Xml|4.4.0System.Security.Principal|4.3.0System.Security.Principal.Windows|4.4.0System.Text.Encoding|4.3.0System.Text.Encoding.Extensions|4.3.0System.Text.RegularExpressions|4.3.0System.Threading|4.3.0System.Threading.Overlapped|4.3.0System.Threading.Tasks|4.3.0System.Threading.Tasks.Extensions|4.3.0System.Threading.Tasks.Parallel|4.3.0System.Threading.Thread|4.3.0System.Threading.ThreadPool|4.3.0System.Threading.Timer|4.3.0System.ValueTuple|4.3.0System.Xml.ReaderWriter|4.3.0System.Xml.XDocument|4.3.0System.Xml.XmlDocument|4.3.0System.Xml.XmlSerializer|4.3.0System.Xml.XPath|4.3.0System.Xml.XPath.XDocument|4.3.0

So the problem is that the PackageOverrides.txt file defined by the shared framework is not being used by the SDK /cc @dsplaisted@dagood

Will keep looking for root cause of that and a workaround.

[ericstj]

Ok, it looks like the new info is read, but is garbled. The PackageConflictOverrides returned from the ResolveTargetingPackAssets task has all package IDs running together. This means that all the content in PackageOverrides.txt is effectively ignored.

[ericstj]

Opened the bug here: dotnet/sdk#10904

To workaround you can run a target that fixes it, stay tuned and will share that.

[ericstj]

Ok, I tried to workaround the PackageOverride reading bug, but then encountered another issue, so reactivated dotnet/runtime#33450

So I took another tactic at fixing this: just avoiding the reference to the 1.x era packages. I found these were coming in transitively from a number of packages (Moq, xunit, etc) which reference NETStandard.Library 1.6.0. This is issue dotnet/sdk#3044. I'll workaround it by updating that transitive reference.

[JunTaoLuo]

FYI responding to an earlier conversation, there's no ingestion of dotnet/runtime packages (e.g. primitives) in this PR. I've paused that work until all packages are brought up in dotnet/runtime.

Edit: I see everyone has already figured this out, sorry for the spam

[dotnet-maestro]

Auto-Merge Status

This pull request will be merged because the following merge policies have succeeded.

• ✔️ Standard Merge Policies Succeeded - Successful checks: extensions-ci, extensions-ci (Build Ubuntu 16.04 Release), extensions-ci (Build Ubuntu 16.04 Debug), extensions-ci (Build Test: Linux Source Build), extensions-ci (Build OSX Release), extensions-ci (Build OSX Debug), extensions-ci (Build Windows Release), extensions-ci (Build Windows Debug), extensions-ci (Build Code check)
• ✔️ Standard Merge Policies Succeeded - No reviews have requested changes.

[safern]

Thanks @ericstj

[JunTaoLuo]

🎉

[dotnet-maestro]

This pull request updates the following dependencies

From https://github.com/dotnet/runtime

• Subscription: 8542892c-8237-4ac0-12f9-08d795579ebc
• Build: 20200317.12
• Date Produced: 3/18/2020 9:42 AM
• Commit: c0ddd1c5d1636de873398c8d9544e02289f95fec
• Branch: refs/heads/master
• Updates:
  • Microsoft.NETCore.App.Runtime.win-x64 -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Ref -> 5.0.0-preview.3.20167.12
  • Microsoft.NETCore.App.Internal -> 5.0.0-preview.3.20167.12

Coherency Updates

The following updates ensure that dependencies with a CoherentParentDependency
attribute were produced in a build used as input to the parent dependency's build.
See Dependency Description Format

• Microsoft.Win32.Registry -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ComponentModel.Annotations -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.DiagnosticSource -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Diagnostics.EventLog -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.IO.Pipelines -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Reflection.Metadata -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Runtime.CompilerServices.Unsafe -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Cng -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Security.Cryptography.Xml -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.ServiceProcess.ServiceController -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Encodings.Web -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• System.Text.Json -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)
• Microsoft.NETCore.Platforms -> 5.0.0-preview.3.20167.12 (parent: Microsoft.NETCore.App.Runtime.win-x64)

From https://github.com/dotnet/roslyn

• Subscription: d9874afa-852c-4c5b-12f8-08d795579ebc
• Build: 20200316.3
• Date Produced: 3/16/2020 6:04 PM
• Commit: c0fc88e0ddd03abbdac8fc7e0d280b70d7208382
• Branch: refs/heads/master-vs-deps
• Updates:
  • Microsoft.Net.Compilers.Toolset -> 3.6.0-2.20166.3

From https://github.com/dotnet/arcade

• Subscription: 455063ee-4723-4484-e1e7-08d79559e508
• Build: 20200312.3
• Date Produced: 3/12/2020 5:55 PM
• Commit: 16d4350184cf362cd59807b589d1c93803025abc
• Branch: refs/heads/master
• Updates:
  • Microsoft.DotNet.Arcade.Sdk -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.GenAPI -> 5.0.0-beta.20162.3
  • Microsoft.DotNet.Helix.Sdk -> 5.0.0-beta.20162.3

[ghost]

Auto-approving dependency update.

[pranavkm]

@dagood this error message showed up again:

The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Previous discussions here: dotnet/runtime#1129.

[dagood]

Previous discussions here: dotnet/runtime#1129.

To sum up, if it's the same issue, I would expect to see a data/PlatformManifest.txt in https://dev.azure.com/dnceng/internal/_build/results?buildId=553529&view=resultsPackageArtifacts/Microsoft.NETCore.App.Ref.5.0.0-preview.3.20159.13.nupkg that's missing some data, but I don't see anything obvious missing.

There are some "odd" names in the manifest with lib prefixes like libSystem.Security.Cryptography.Native.OpenSsl.so|Microsoft.NETCore.App.Ref||0.0.0.0 that I remember seeing some commotion over in a dotnet/runtime issue, but I haven't read through that.

@dotnet/runtime-infrastructure PTAL, I might be missing context on recent changes and it might be one of the other causes listed in that issue that I'm not very familiar with anyway (openssl version etc.)

[safern]

cc: @bartonjs@janvorli as they might see something obvious we might be missing here.

[bartonjs]

-4305 is kCCUnimplemented (https://opensource.apple.com/source/CommonCrypto/CommonCrypto-36064/CommonCrypto/CommonCryptor.h.auto.html). The last time we had a problem with that was fixed in 2017. Maybe something is being weird in library binding and we're finding an old copy? Is there a good way to figure out where our shim got loaded from on the CI machines?

AESKeyWrapEncrypt /does/ use ECB mode, so it /could/ be the problems that we fixed in dotnet/corefx#21631 ... if we're finding an old version of the library.

[pranavkm]

We fairly recently changed the OS version for the MacOS version. @dougbu would have more specifics about it. Could that affect this in any way?

[jkoritzinsky]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

[bartonjs]

We fairly recently changed the OS version for the MacOS version. ... Could that affect this in any way?

If you bumped to 10.15, yes (I don't think dotnet/runtime has that in CI yet because the images weren't ready/reliable). If you're 10.14 then the most likely culprit is the shim library rename, since that would be something you'd probably just now be ingesting (and dotnet/runtime isn't failing on 10.14).

[BrennanConroy]

We're on 10.14 now

[safern]

We recently changed the name of the native libraries. Is it possible that the because of the name change and the fact that we are relying on probing to find libSystem.Security.Cryptography.Native.so when it didn’t find System.Security.Cryptography.Native.so that we’re picking up an old version of System.Security.Crpytography.Native.so?

That seems like a possible issue. I think the easiest would be to pull this and debug locally. Trying now to see what System.Security.Cryptography.Native.so it is loading.

[ericstj]

Agreed with the assesment of @jkoritzinsky and @safern. In 1.x we shipped nuget packages with these shims and rely on conflict resolution to make the framework copies win.

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

[pranavkm]

To avoid those old versions of System.Native.* from 1.x you can just reference the latest Microsoft.NETCore.Targets which dummys out all the old runtime.json that brought in the RID-specific System.Native packages.

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

[ericstj]

@ericstj is that a recommendation for this project https://github.com/dotnet/extensions/blob/master/src/Configuration/Config.Xml/src/Microsoft.Extensions.Configuration.Xml.csproj#L19 or for the System.Security.Cryptography.Xml package?

Not exactly. The only way you'd get an old System.Native.* is if you had a reference to a 1.x era package. You'd need to find out which project was bringing that in and add the reference there. System.Security.Cryptography.Xml wouldn't be the culprit, it's probably some 3rd party package with very old dependencies.

[wtgodbe]

Oh, could be - we see transient failures w/ Pdb2Pdb every once in a while: https://github.com/dotnet/aspnetcore-internal/issues/3484

[wtgodbe]

Windows failures are gone on the second attempt, but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

And:

Interop+AppleCrypto+AppleCommonCryptoCryptographicException : The system cryptographic library returned error '-4305' of type 'CCCryptorStatus'

Which is the same error as before...

[BrennanConroy]

but we still have:

System.IO.FileLoadException : Could not load file or assembly 'Microsoft.Extensions.FileProviders.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. An operation is not legal in the current state. (0x80131509)

No, those are gone on the second attempt. You're looking at the test tab which will show failures across all retries.

@ericstj@safern Could we get some help with the crypto errors that haven't gone away with the Runtime fix?

[ericstj]

We had discussed that we'll hold off on consuming the extensions from dotnet/runtime to avoid too much churn. @JunTaoLuo can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

[BrennanConroy]

Can you peel-off the consumption of extensions into a seperate PR so that it doesn't block ingestion of runtime?

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

[ericstj]

I'm not aware of us consuming Extensions from Runtime anywhere yet. Am I missing something here?

Right you are, sorry @safern and I had assumed that was going on here, it would have caused similar errors (FileLoadExcetion due to two different assemblies with same identity). I don't think that's the case, so just ignore my comment.

Nevermind, just me, crossed wires. Let me look at crypto failures.

[ericstj]

Here's what I see in the binlog.

The following is referenced by the test project:

Native shim references

/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.apple/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.Apple.dylib
RuntimeIdentifier = osx.10.10-x64
/Users/runner/runners/2.165.2/work/1/s/.packages/runtime.osx.10.10-x64.runtime.native.system.security.cryptography.openssl/4.3.0/runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
AssetType = native
CopyLocal = true
DestinationSubDirectory = runtimes/osx.10.10-x64/native/
DestinationSubPath = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
NuGetPackageId = runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl
NuGetPackageVersion = 4.3.0
PathInPackage = runtimes/osx.10.10-x64/native/System.Security.Cryptography.Native.OpenSsl.dylib
RuntimeIdentifier = osx.10.10-x64

PackageOverrides

PackageOverrides
NETStandard.Library
OverriddenPackages = Microsoft.Win32.Primitives|4.3.0;
System.AppContext|4.3.0;
System.Collections|4.3.0;
System.Collections.Concurrent|4.3.0;
System.Collections.Immutable|1.4.0;
System.Collections.NonGeneric|4.3.0;
System.Collections.Specialized|4.3.0;
System.ComponentModel|4.3.0;
System.ComponentModel.EventBasedAsync|4.3.0;
System.ComponentModel.Primitives|4.3.0;
System.ComponentModel.TypeConverter|4.3.0;
System.Console|4.3.0;
System.Data.Common|4.3.0;
System.Diagnostics.Contracts|4.3.0;
System.Diagnostics.Debug|4.3.0;
System.Diagnostics.FileVersionInfo|4.3.0;
System.Diagnostics.Process|4.3.0;
System.Diagnostics.StackTrace|4.3.0;
System.Diagnostics.TextWriterTraceListener|4.3.0;
System.Diagnostics.Tools|4.3.0;
System.Diagnostics.TraceSource|4.3.0;
System.Diagnostics.Tracing|4.3.0;
System.Dynamic.Runtime|4.3.0;
System.Globalization|4.3.0;
System.Globalization.Calendars|4.3.0;
System.Globalization.Extensions|4.3.0;
System.IO|4.3.0;
System.IO.Compression|4.3.0;
System.IO.Compression.ZipFile|4.3.0;
System.IO.FileSystem|4.3.0;
System.IO.FileSystem.DriveInfo|4.3.0;
System.IO.FileSystem.Primitives|4.3.0;
System.IO.FileSystem.Watcher|4.3.0;
System.IO.IsolatedStorage|4.3.0;
System.IO.MemoryMappedFiles|4.3.0;
System.IO.Pipes|4.3.0;
System.IO.UnmanagedMemoryStream|4.3.0;
System.Linq|4.3.0;
System.Linq.Expressions|4.3.0;
System.Linq.Queryable|4.3.0;
System.Net.Http|4.3.0;
System.Net.NameResolution|4.3.0;
System.Net.Primitives|4.3.0;
System.Net.Requests|4.3.0;
System.Net.Security|4.3.0;
System.Net.Sockets|4.3.0;
System.Net.WebHeaderCollection|4.3.0;
System.ObjectModel|4.3.0;
System.Private.DataContractSerialization|4.3.0;
System.Reflection|4.3.0;
System.Reflection.Emit|4.3.0;
System.Reflection.Emit.ILGeneration|4.3.0;
System.Reflection.Emit.Lightweight|4.3.0;
System.Reflection.Extensions|4.3.0;
System.Reflection.Primitives|4.3.0;
System.Reflection.TypeExtensions|4.3.0;
System.Resources.ResourceManager|4.3.0;
System.Runtime|4.3.0;
System.Runtime.Extensions|4.3.0;
System.Runtime.Handles|4.3.0;
System.Runtime.InteropServices|4.3.0;
System.Runtime.InteropServices.RuntimeInformation|4.3.0;
System.Runtime.Loader|4.3.0;
System.Runtime.Numerics|4.3.0;
System.Runtime.Serialization.Formatters|4.3.0;
System.Runtime.Serialization.Json|4.3.0;
System.Runtime.Serialization.Primitives|4.3.0;
System.Security.AccessControl|4.4.0;
System.Security.Claims|4.3.0;
System.Security.Cryptography.Algorithms|4.3.0;
System.Security.Cryptography.Csp|4.3.0;
System.Security.Cryptography.Encoding|4.3.0;
System.Security.Cryptography.Primitives|4.3.0;
System.Security.Cryptography.X509Certificates|4.3.0;
System.Security.Cryptography.Xml|4.4.0;
System.Security.Principal|4.3.0;
System.Security.Principal.Windows|4.4.0;
System.Text.Encoding|4.3.0;
System.Text.Encoding.Extensions|4.3.0;
System.Text.RegularExpressions|4.3.0;
System.Threading|4.3.0;
System.Threading.Overlapped|4.3.0;
System.Threading.Tasks|4.3.0;
System.Threading.Tasks.Extensions|4.3.0;
System.Threading.Tasks.Parallel|4.3.0;
System.Threading.Thread|4.3.0;
System.Threading.ThreadPool|4.3.0;
System.Threading.Timer|4.3.0;
System.ValueTuple|4.3.0;
System.Xml.ReaderWriter|4.3.0;
System.Xml.XDocument|4.3.0;
System.Xml.XmlDocument|4.3.0;
System.Xml.XmlSerializer|4.3.0;
System.Xml.XPath|4.3.0;
System.Xml.XPath.XDocument|4.3.0;
Microsoft.NETCore.App.Ref
OverriddenPackages = Microsoft.CSharp|4.4.0Microsoft.Win32.Primitives|4.3.0Microsoft.Win32.Registry|4.4.0runtime.debian.8-x64.runtime.native.System|4.3.0runtime.debian.8-x64.runtime.native.System.IO.Compression|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Http|4.3.0runtime.debian.8-x64.runtime.native.System.Net.Security|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.23-x64.runtime.native.System|4.3.0runtime.fedora.23-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.23-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.fedora.24-x64.runtime.native.System|4.3.0runtime.fedora.24-x64.runtime.native.System.IO.Compression|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Http|4.3.0runtime.fedora.24-x64.runtime.native.System.Net.Security|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.13.2-x64.runtime.native.System|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.opensuse.42.1-x64.runtime.native.System|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.IO.Compression|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Http|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Net.Security|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.osx.10.10-x64.runtime.native.System|4.3.0runtime.osx.10.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Http|4.3.0runtime.osx.10.10-x64.runtime.native.System.Net.Security|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple|4.3.0runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.rhel.7-x64.runtime.native.System|4.3.0runtime.rhel.7-x64.runtime.native.System.IO.Compression|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Http|4.3.0runtime.rhel.7-x64.runtime.native.System.Net.Security|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.IO.Compression|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Http|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Net.Security|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography|4.3.0runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl|4.3.0System.AppContext|4.3.0System.Buffers|4.4.0System.Collections|4.3.0System.Collections.Concurrent|4.3.0System.Collections.Immutable|1.4.0System.Collections.NonGeneric|4.3.0System.Collections.Specialized|4.3.0System.ComponentModel|4.3.0System.ComponentModel.EventBasedAsync|4.3.0System.ComponentModel.Primitives|4.3.0System.ComponentModel.TypeConverter|4.3.0System.Console|4.3.0System.Data.Common|4.3.0System.Diagnostics.Contracts|4.3.0System.Diagnostics.Debug|4.3.0System.Diagnostics.DiagnosticSource|4.4.0System.Diagnostics.FileVersionInfo|4.3.0System.Diagnostics.Process|4.3.0System.Diagnostics.StackTrace|4.3.0System.Diagnostics.TextWriterTraceListener|4.3.0System.Diagnostics.Tools|4.3.0System.Diagnostics.TraceSource|4.3.0System.Diagnostics.Tracing|4.3.0System.Dynamic.Runtime|4.3.0System.Globalization|4.3.0System.Globalization.Calendars|4.3.0System.Globalization.Extensions|4.3.0System.IO|4.3.0System.IO.Compression|4.3.0System.IO.Compression.ZipFile|4.3.0System.IO.FileSystem|4.3.0System.IO.FileSystem.AccessControl|4.4.0System.IO.FileSystem.DriveInfo|4.3.0System.IO.FileSystem.Primitives|4.3.0System.IO.FileSystem.Watcher|4.3.0System.IO.IsolatedStorage|4.3.0System.IO.MemoryMappedFiles|4.3.0System.IO.Pipes|4.3.0System.IO.UnmanagedMemoryStream|4.3.0System.Linq|4.3.0System.Linq.Expressions|4.3.0System.Linq.Queryable|4.3.0System.Net.Http|4.3.0System.Net.NameResolution|4.3.0System.Net.Primitives|4.3.0System.Net.Requests|4.3.0System.Net.Security|4.3.0System.Net.Sockets|4.3.0System.Net.WebHeaderCollection|4.3.0System.ObjectModel|4.3.0System.Private.DataContractSerialization|4.3.0System.Reflection|4.3.0System.Reflection.Emit|4.3.0System.Reflection.Emit.ILGeneration|4.3.0System.Reflection.Emit.Lightweight|4.3.0System.Reflection.Extensions|4.3.0System.Reflection.Metadata|1.5.0System.Reflection.Primitives|4.3.0System.Reflection.TypeExtensions|4.3.0System.Resources.ResourceManager|4.3.0System.Runtime|4.3.0System.Runtime.Extensions|4.3.0System.Runtime.Handles|4.3.0System.Runtime.InteropServices|4.3.0System.Runtime.InteropServices.RuntimeInformation|4.3.0System.Runtime.Loader|4.3.0System.Runtime.Numerics|4.3.0System.Runtime.Serialization.Formatters|4.3.0System.Runtime.Serialization.Json|4.3.0System.Runtime.Serialization.Primitives|4.3.0System.Security.AccessControl|4.4.0System.Security.Claims|4.3.0System.Security.Cryptography.Algorithms|4.3.0System.Security.Cryptography.Cng|4.4.0System.Security.Cryptography.Csp|4.3.0System.Security.Cryptography.Encoding|4.3.0System.Security.Cryptography.OpenSsl|4.4.0System.Security.Cryptography.Primitives|4.3.0System.Security.Cryptography.X509Certificates|4.3.0System.Security.Cryptography.Xml|4.4.0System.Security.Principal|4.3.0System.Security.Principal.Windows|4.4.0System.Text.Encoding|4.3.0System.Text.Encoding.Extensions|4.3.0System.Text.RegularExpressions|4.3.0System.Threading|4.3.0System.Threading.Overlapped|4.3.0System.Threading.Tasks|4.3.0System.Threading.Tasks.Extensions|4.3.0System.Threading.Tasks.Parallel|4.3.0System.Threading.Thread|4.3.0System.Threading.ThreadPool|4.3.0System.Threading.Timer|4.3.0System.ValueTuple|4.3.0System.Xml.ReaderWriter|4.3.0System.Xml.XDocument|4.3.0System.Xml.XmlDocument|4.3.0System.Xml.XmlSerializer|4.3.0System.Xml.XPath|4.3.0System.Xml.XPath.XDocument|4.3.0

So the problem is that the PackageOverrides.txt file defined by the shared framework is not being used by the SDK /cc @dsplaisted@dagood

Will keep looking for root cause of that and a workaround.

[ericstj]

Ok, it looks like the new info is read, but is garbled. The PackageConflictOverrides returned from the ResolveTargetingPackAssets task has all package IDs running together. This means that all the content in PackageOverrides.txt is effectively ignored.

[ericstj]

Opened the bug here: dotnet/sdk#10904

To workaround you can run a target that fixes it, stay tuned and will share that.

[ericstj]

Ok, I tried to workaround the PackageOverride reading bug, but then encountered another issue, so reactivated dotnet/runtime#33450

So I took another tactic at fixing this: just avoiding the reference to the 1.x era packages. I found these were coming in transitively from a number of packages (Moq, xunit, etc) which reference NETStandard.Library 1.6.0. This is issue dotnet/sdk#3044. I'll workaround it by updating that transitive reference.

[JunTaoLuo]

FYI responding to an earlier conversation, there's no ingestion of dotnet/runtime packages (e.g. primitives) in this PR. I've paused that work until all packages are brought up in dotnet/runtime.

Edit: I see everyone has already figured this out, sorry for the spam

[dotnet-maestro]

Auto-Merge Status

This pull request will be merged because the following merge policies have succeeded.

• ✔️ Standard Merge Policies Succeeded - Successful checks: extensions-ci, extensions-ci (Build Ubuntu 16.04 Release), extensions-ci (Build Ubuntu 16.04 Debug), extensions-ci (Build Test: Linux Source Build), extensions-ci (Build OSX Release), extensions-ci (Build OSX Debug), extensions-ci (Build Windows Release), extensions-ci (Build Windows Debug), extensions-ci (Build Code check)
• ✔️ Standard Merge Policies Succeeded - No reviews have requested changes.

[safern]

Thanks @ericstj

[JunTaoLuo]

🎉