Fix Android crypto asserts

[vcsjones]

This fixes three asserts that were started occurring in the native Android cryptographic primitives.

• One shot hashing now tolerates empty/null input.
• Hashing and HMAC will now no-op if the append is empty.
• RSA encryption now tolerates empty/null input.

/cc @AaronRobinsonMSFT.

Closes#61783

[ghost]

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @GrabYourPitchforks
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This fixes three asserts that were started occurring in the native Android cryptographic primitives.

• One shot hashing now tolerates empty/null input.
• Hashing and HMAC will now no-op if the append is empty.
• RSA encryption now tolerates empty/null input.

/cc @AaronRobinsonMSFT.

Author:	vcsjones
Assignees:	-
Labels:	area-System.Security, community-contribution
Milestone:	-

[ghost]

Tagging subscribers to 'arch-android': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This fixes three asserts that were started occurring in the native Android cryptographic primitives.

• One shot hashing now tolerates empty/null input.
• Hashing and HMAC will now no-op if the append is empty.
• RSA encryption now tolerates empty/null input.

/cc @AaronRobinsonMSFT.

Author:	vcsjones
Assignees:	-
Labels:	area-System.Security, os-android, community-contribution
Milestone:	-

[MaximLipnin]

@vcsjones thanks for your help, let me run some additional CI lanes to verify the fix

[MaximLipnin]

/azp run runtime-manual

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[vcsjones]

Note, I saw a number of failures in System.Security.Cryptography on Android, most of them because it seems to be trying to load openssl or run RC2 tests. However none of the tests are tripping an assert now.

[steveisok]

Note, I saw a number of failures in System.Security.Cryptography on Android, most of them because it seems to be trying to load openssl or run RC2 tests. However none of the tests are tripping an assert now.

Why would it pick the wrong DefaultECDsaProvider ? The logs indicate trying the .Unix one when it should be picking .Android.

[vcsjones]

Why would it pick the wrong DefaultECDsaProvider ? The logs indicate trying the .Unix one when it should be picking .Android.

I'm guessing this happened when .Algorithms was consolidated. The OpenSSL initializer in the cctor is firing for Android. I'm guessing that should not be happening.

11-19 16:31:47.506 3525 3548 I DOTNET : at System.Security.Cryptography.Tests.ECKeyFileTests`1[[System.Security.Cryptography.ECDsa, System.Security.Cryptography, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]]..cctor() in /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/EC/ECKeyFileTests.cs:line 30
11-19 16:31:47.506 3525 3548 I DOTNET : ----- Inner Stack Trace -----
11-19 16:31:47.506 3525 3548 I DOTNET : at Interop.Crypto..cctor() in /_/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.Initialization.cs:line 18
11-19 16:31:47.506 3525 3548 I DOTNET : ----- Inner Stack Trace -----
11-19 16:31:47.506 3525 3548 I DOTNET : at Interop.CryptoInitializer..cctor() in /_/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.Initialization.cs:line 34
11-19 16:31:47.506 3525 3548 I DOTNET : Execution time: 0
11-19 16:31:47.507 3525 3548 I DOTNET : [FAIL] System.Security.Cryptography.EcDsa.Tests.ECDsaKeyFileTests.ReadWriteSect163k1Key1ExplicitPkcs8
11-19 16:31:47.507 3525 3548 I DOTNET : [FAIL] System.Security.Cryptography.EcDsa.Tests.ECDsaKeyFileTests.ReadWriteSect163k1Key1ExplicitPkcs8 Test name: System.Security.Cryptography.EcDsa.Tests.ECDsaKeyFileTests.ReadWriteSect163k1Key1ExplicitPkcs8
11-19 16:31:47.507 3525 3548 I DOTNET : Assembly: [System.Security.Cryptography.Tests, Version=7.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
11-19 16:31:47.507 3525 3548 I DOTNET : Exception messages: System.TypeInitializationException : The type initializer for 'System.Security.Cryptography.Tests.ECKeyFileTests`1' threw an exception.
11-19 16:31:47.507 3525 3548 I DOTNET : ---- System.TypeInitializationException : The type initializer for 'Crypto' threw an exception.
11-19 16:31:47.507 3525 3548 I DOTNET : -------- System.TypeInitializationException : The type initializer for 'CryptoInitializer' threw an exception.
11-19 16:31:47.507 3525 3548 I DOTNET : ------------ System.DllNotFoundException : libSystem.Security.Cryptography.Native.OpenSsl Exception stack traces: at System.Security.Cryptography.EcDsa.Tests.ECDsaKeyFileTests..ctor()
11-19 16:31:47.507 3525 3548 I DOTNET : at System.Reflection.RuntimeConstructorInfo.InternalInvoke(Object obj, Span`1 parameters, Boolean wrapExceptions)
11-19 16:31:47.507 3525 3548 I DOTNET : ----- Inner Stack Trace -----

Let me see if I can fix that up. But I would not hold this PR for that. This PR is at least some progress (assuming I did it correctly 😄)

[bartonjs]

runtime/src/libraries/System.Security.Cryptography/tests/System.Security.Cryptography.Tests.csproj

Lines 276 to 291 in 6d9edd4

	<ItemGroupCondition="('$(TargetsUnix)' == 'true' or '$(TargetsBrowser)' == 'true') and '$(UseAndroidCrypto)' != 'true'">
	<CompileInclude="DefaultECDsaProvider.Unix.cs" />
	<CompileInclude="DefaultECDiffieHellmanProvider.Unix.cs" />
	<CompileInclude="$(CommonPath)Interop\Unix\Interop.Libraries.cs"
	Link="Common\Interop\Unix\Interop.Libraries.cs" />
	<CompileInclude="$(CommonPath)Interop\Unix\System.Security.Cryptography.Native\Interop.Initialization.cs"
	Link="Common\Interop\Unix\System.Security.Cryptography.Native\Interop.Initialization.cs" />
	</ItemGroup>
	<ItemGroupCondition="'$(UseAndroidCrypto)' == 'true'">
	<CompileInclude="$(CommonPath)Interop\Android\Interop.Libraries.cs"
	Link="Common\Interop\Android\Interop.Libraries.cs" />
	<CompileInclude="$(CommonPath)Interop\Android\System.Security.Cryptography.Native.Android\Interop.Initialization.cs"
	Link="Common\Interop\Android\System.Security.Cryptography.Native.Android\Interop.Initialization.cs" />
	<CompileInclude="DefaultECDsaProvider.Android.cs" />
	<CompileInclude="DefaultECDiffieHellmanProvider.Android.cs" />
	</ItemGroup>

Looks good, ja? Well, the properties section doesn't defined UseAndroidCrypto. My bad.

[vcsjones]

That fixed the crypto initialization issues. Tests are still failing because it's trying to run the RC2 tests. I will fix that one next.

[vcsjones]

And now the RC2 tests are fixed, so this is green locally for me. I would appreciate if someone could start the relevant pipelines in CI.

(Previously, the RC2 one-shot tests were being excluded by csproj. They aren't anymore, so they are marked conditional as the other RC2 tests are).

[steveisok]

/azp run runtime-manual

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[vcsjones]

@bartonjs what do y'all think about having Android tests run automatically if anything changes in src/libraries/System.Security.Cryptography/**/*? Seems like changes in S.S.Cryptography have a higher chance of breaking Android since these are, at the core, a massive p/invoke abstraction.

[bartonjs]

what do y'all think about having Android tests run automatically if anything changes in src/libraries/System.Security.Cryptography/**/*?

SGTM, (even if it increases the CI duration for crypto-affecting PRs) but I don't know how any of that works. What do you think, @steveisok? (Clearly it didn't occur to me that Android tests weren't run when I made the consolidation)

[steveisok]

what do y'all think about having Android tests run automatically if anything changes in src/libraries/System.Security.Cryptography/**/*?

SGTM, (even if it increases the CI duration for crypto-affecting PRs) but I don't know how any of that works. What do you think, @steveisok? (Clearly it didn't occur to me that Android tests weren't run when I made the consolidation)

We're going to try and get there as part of #61017. By default, only System.Runtime tests will execute on PR's for mobile legs. I think detecting the libraries you change and only run them is a good middle ground.

[vcsjones]

Clearly it didn't occur to me that Android tests weren't run when I made the consolidation

Same, I've broken the Android build a couple of times now: #59812, #59818, #55699, etc.

I think detecting the libraries you change and only run them is a good middle ground.

Even if we run all Android tests for changes in S.S.Cryptography, I think that is an improvement over the current situation. Ideally yes, only run tests for changed projects, but the nature of S.S.Cryptography is "giant abstraction of p/invoke" so there is a fairly high chance that changes here have an impact on Android's CI.

[steveisok]

Even if we run all Android tests for changes in S.S.Cryptography, I think that is an improvement over the current situation. Ideally yes, only run tests for changed projects, but the nature of S.S.Cryptography is "giant abstraction of p/invoke" so there is a fairly high chance that changes here have an impact on Android's CI.

Good point. Running them all the time again is something to consider.

BTW - the tests look good. The emulators are red because they sometimes fail S.S.Cryptography tests due to running out of memory. Many attempts were made to figure out why, none have solved the problem thus far.

[bartonjs]

@steveisok I think you're saying it's good to merge even with the failures that are presented; but I'm not sure enough that I'm willing to push the button.

[20:40:58] info: Running instrumentation class net.dot.MonoRunner took 28.2490973 seconds
[20:40:58] dbug: Exit code: 0
Standard Output:
INSTRUMENTATION_RESULT: shortMsg=Process crashed.
INSTRUMENTATION_CODE: 0
[20:40:58] info: Short Message: Process crashed.
[20:40:58] fail: No value for 'return-code' provided in instrumentation result. This may indicate a crashed test (see log)
[20:41:01] dbug: Executing command: '/datadisks/disk1/work/A62008FE/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21561.1/runtimes/any/native/adb/linux/adb -s emulator-5554 logcat -d '
[20:41:01] info: Wrote current ADB log to /datadisks/disk1/work/A62008FE/w/B7220A4A/uploads/adb-logcat-net.dot.System.Security.Cryptography.Tests-net.dot.MonoRunner.log
[20:41:04] dbug: Executing command: '/datadisks/disk1/work/A62008FE/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21561.1/runtimes/any/native/adb/linux/adb -s emulator-5554 bugreport /datadisks/disk1/work/A62008FE/w/B7220A4A/uploads/adb-bugreport-net.dot.System.Security.Cryptography.Tests.zip'
[20:42:03] info: Wrote ADB bugreport to /datadisks/disk1/work/A62008FE/w/B7220A4A/uploads/adb-bugreport-net.dot.System.Security.Cryptography.Tests.zip
[20:42:03] fail: Non-success instrumentation exit code: 82, expected: 0
[20:42:03] info: Attempting to remove apk 'net.dot.System.Security.Cryptography.Tests': [20:42:03] dbug: Executing command: '/datadisks/disk1/work/A62008FE/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21561.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.System.Security.Cryptography.Tests'
[20:42:03] info: Successfully uninstalled net.dot.System.Security.Cryptography.Tests.
[20:42:03] dbug: Saving diagnostics data to '/datadisks/disk1/work/A62008FE/w/B7220A4A/e/diagnostics.json'
XHarness exit code: 1 (TESTS_FAILED)

doesn't look happy to me... but if that's the OOM failure you mentioned, then OK.

(As of this message, at least, all checks have completed. The Browser/wasm failure looks like some piece of infra failed after some tests, so it's only the runtime-manual checks that I'm not sure about)

[steveisok]

In the adb logs, you'll see a bunch of these before the test activity crashes:

11-19 20:38:01.647 1735 1735 E lowmemorykiller: Kill 'com.android.dialer' (2973), uid 10101, oom_adj 250 to free 64092kB
11-19 20:38:01.647 1735 1735 I lowmemorykiller: Reclaimed 64092kB, cache(128148kB) and free(134692kB)-reserved(80816kB) below min(129024kB) for oom_adj 250
11-19 20:38:01.647 1735 1735 I lowmemorykiller: Suppressed 24 failed kill reports
11-19 20:38:01.664 1681 1681 I Zygote : Process 2973 exited due to signal 9 (Killed)

The failures in the other legs are known issues. I think this is good to go. Especially given the crypto tests pass on android devices (arm/arm64 legs).

[MaximLipnin]

Thanks for looking into this. JFR - There are other OOM-related issue, e.g. #55691. so it's not specific for this PR. Since the initial issue seems to be addressed, it would be good to merge this change.

[MaximLipnin]

Thanks for looking into this. JFR - There are other OOM-related issue, e.g. #55691. so it's not specific for this PR. Since the initial issue seems to be addressed, it would be good to merge this change.

Merging it as it will help to clean up the failures on android lanes.