[wasm] rewrite crypto-worker to typescript by pavelsavara · Pull Request #73073 · dotnet/runtime

pavelsavara · 2022-07-29T17:46:04Z

rewrite to TS and add type of InitCryptoMessageData
in the crypto-worker we forward the logs to WS based on config.diagnosticTracing
returned some console writes in hope they will prevented the deadlock.

Moved some changes from this PR to #73468 and #73073

ghost · 2022-07-29T17:46:31Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

split logging code into separate file logging.ts
moved readSymbolMapFile() to startup.ts
in the crypto-worker we forward the logs to WS only on Debug build of runtime. We should not do it in production.
fixed bug in setup_proxy_console() which didn't really copy the original log and error functions of the console and caused recursion on error.

Author:	pavelsavara
Assignees:	pavelsavara
Labels:	`arch-wasm`, `area-System.Runtime.InteropServices.JavaScript`
Milestone:	7.0.0

pavelsavara · 2022-08-01T10:20:28Z

/azp run runtime-wasm

azure-pipelines · 2022-08-01T10:20:38Z

Azure Pipelines successfully started running 1 pipeline(s).

pavelsavara · 2022-08-01T11:04:17Z

/azp run runtime-wasm

azure-pipelines · 2022-08-01T11:04:29Z

Azure Pipelines successfully started running 1 pipeline(s).

lambdageek

thanks for converting the crypto worker to TS!

pavelsavara · 2022-08-01T16:28:37Z

I wrote and then spoke to @javiercn this morning about how to locate the url of the dotnet-crypto-worker.js during Blazor startup. How they could influence the location of the file.
We agreed it would be ideal if they could give us the pre-resolved URL as config.asset.

I added new resource type js-module-crypto and js-module-threads, for which Blazor could configure it's asset resolvedUrl.
I also changed the detection of blazor startup sequence to not be disabled by presence of config, but only by config.assets having some assembly in it.

@radical I know you guys also spoke about it.
@lambdageek MT will also need asset for it's dotnet.worker.js I also included it in this PR too.

pavelsavara · 2022-08-01T16:28:49Z

/azp run runtime-wasm

azure-pipelines · 2022-08-01T16:28:58Z

Azure Pipelines successfully started running 1 pipeline(s).

pavelsavara · 2022-08-01T16:29:49Z

Why is this changing from slice? I think slice reads better.

because slice makes copy, the constructor make a view. Unless I made some mistake.

eerhardt · 2022-08-01T16:34:10Z

because slice makes copy, the constructor make a view. Unless I made some mistake.

Ah, that's a difference between JS and .NET. Looks like what we want is subarray: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/subarray

lambdageek · 2022-08-01T17:23:53Z

@lambdageek MT will also need asset for it's dotnet.worker.js I also included it in this PR too.

I'm not familiar with how the assets stuff works... Emscripten just does this:

allocateUnusedWorker: function(){if(!Module["locateFile"]){PThread.unusedWorkers.push(newWorker(newURL("dotnet.worker.js",import.meta.url)));return;}varpthreadMainJs=locateFile("dotnet.worker.js");PThread.unusedWorkers.push(newWorker(pthreadMainJs));},

but I don't see any changes to locateFile in this PR

Update I suppose we could replace PThread.allocateUnusedWorker with one that goes through the asset mechanism

pavelsavara · 2022-08-01T17:37:42Z

Update I suppose we could replace PThread.allocateUnusedWorker with one that goes through the asset mechanism

Right, different PR

pavelsavara · 2022-08-01T19:42:40Z

/azp run runtime-wasm

azure-pipelines · 2022-08-01T19:42:55Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2022-08-05T11:20:48Z

Azure Pipelines successfully started running 1 pipeline(s).

pavelsavara · 2022-08-05T13:14:44Z

/azp run runtime-wasm

azure-pipelines · 2022-08-05T13:14:58Z

Azure Pipelines successfully started running 1 pipeline(s).

eerhardt · 2022-08-05T15:53:50Z

have you seen the subtlecrypto tests taking more than 15mins on CI?

No - locally on my codespaces VM they take around 70-90 seconds.

pavelsavara · 2022-08-05T16:01:18Z

have you seen the subtlecrypto tests taking more than 15mins on CI?
No - locally on my codespaces VM they take around 70-90 seconds.

On my windows, when it works the it's 2 minutes.
Total: 2038, Errors: 0, Failed: 0, Skipped: 1, Time: 126.22934s

But I saw it deadlock multiple times, now I'm trying to reproduce it on main, with just the logging fixed here #73468

eerhardt · 2022-08-05T17:12:30Z

now I'm trying to reproduce it on main, with just the logging fixed here

Yeah, one concern I have with this PR is that you are trying to change too much at once. It is super hard to know what went wrong when making multiple changes in a single PR.

pavelsavara · 2022-08-05T17:23:35Z

I was able to reproduce the deadlock on this branch locally on my windows
It takes many iterations of

dotnet.exe build /p:TargetOS=Browser /p:TargetArchitecture=wasm /p:Configuration=Debug /t:Test src\libraries\System.Security.Cryptography\tests /p:Scenario=WasmTestOnBrowser /p:UseSubtleCryptoForTests=true /p:WasmTestAppArgs="-class System.Security.Cryptography.Encryption.Aes.Tests.AesContractTests"

pavelsavara · 2022-08-05T18:29:10Z

It is super hard to know what went wrong when making multiple changes in a single PR.

Moved some changes from this PR to #73468 and #73073 and left only rewrite to TS here.

pavelsavara · 2022-08-05T18:33:59Z

/azp run runtime-wasm

azure-pipelines · 2022-08-05T18:34:08Z

Azure Pipelines successfully started running 1 pipeline(s).

lot happened since

eerhardt

LGTM - thanks for splitting this up!

pavelsavara · 2022-08-05T18:45:11Z

@eerhardt are you guys able to have look at the deadlock ? do you need more info?

eerhardt · 2022-08-05T18:47:39Z

I haven't been able to look at the deadlock. Does it happen in main, or only with your change?

pavelsavara · 2022-08-05T18:50:04Z

Probably on main as well, but the fix to logging will help you debugging it without XHarness. Otherwise your chrome will crash with OOM.

eerhardt · 2022-08-05T18:51:21Z

Probably on main as well

I haven't seen a test failure report in a CI, and the tests have been running for over a month.

…ccess and preventing the deadlock. This is RC1 stop-gap.

pavelsavara · 2022-08-05T19:54:00Z

Returned some console writes (which I commented out 10 days ago) in hope they prevented the deadlock. I think they were serializing the access to shared buffer. Perhaps we could find better solution later.

pavelsavara · 2022-08-05T20:04:33Z

/azp run runtime-wasm

azure-pipelines · 2022-08-05T20:04:47Z

Azure Pipelines successfully started running 1 pipeline(s).

pavelsavara added arch-wasm WebAssembly architecture area-System.Runtime.InteropServices.JavaScript labels Jul 29, 2022

pavelsavara added this to the 7.0.0 milestone Jul 29, 2022

pavelsavara requested review from eerhardt, radical and simonrozsival July 29, 2022 17:46

pavelsavara requested a review from lewing as a code owner July 29, 2022 17:46

pavelsavara self-assigned this Jul 29, 2022

eerhardt reviewed Jul 29, 2022
View reviewed changes

Comment threadsrc/mono/wasm/runtime/logging.ts Outdated
Comment threadsrc/mono/wasm/runtime/workers/dotnet-crypto-worker.js Outdated
Comment threadsrc/mono/wasm/runtime/logging.ts Outdated
Comment threadsrc/mono/wasm/runtime/logging.ts Outdated

radical reviewed Jul 30, 2022
View reviewed changes

Comment threadsrc/mono/wasm/runtime/workers/dotnet-crypto-worker.js Outdated

pavelsavara requested review from eerhardt and radical August 1, 2022 10:13

eerhardt reviewed Aug 1, 2022
View reviewed changes

Comment threadsrc/mono/wasm/runtime/workers/dotnet-crypto-worker.ts Outdated
Comment threadsrc/mono/wasm/runtime/workers/dotnet-crypto-worker.ts Outdated

lambdageek reviewed Aug 1, 2022
View reviewed changes

Comment threadsrc/mono/wasm/runtime/workers/dotnet-crypto-worker.ts Outdated
Comment threadsrc/mono/wasm/runtime/workers/dotnet-crypto-worker.ts Outdated

lambdageek reviewed Aug 1, 2022
View reviewed changes

Comment threadsrc/mono/wasm/runtime/logging.ts Outdated

lambdageek mentioned this pull request Aug 1, 2022
[wasm-mt] Publish dotnet.worker.js#68509
Closed

This was referenced Aug 5, 2022
Infra improvements for Helix #68176
Closed
GC/API/GC/GetGCMemoryInfo/GetGCMemoryInfo.sh test failing intermittently on CoreCLR Linux ARM32 #73247
Closed

move changes to other PRs and rebase on main
1013b63

pavelsavara force-pushed the fix_crypto_crash branch from 0830d25 to 1013b63Compare August 5, 2022 18:26

pavelsavara changed the title ~~[wasm] asset loading for subtle crypto + logging~~[wasm] rewrite crypto-worker to typescriptAug 5, 2022

pavelsavara requested a review from eerhardt August 5, 2022 18:38

pavelsavara mentioned this pull request Aug 5, 2022
[wasm] asset loading for workers #73484
Merged

eerhardt approved these changes Aug 5, 2022
View reviewed changes

returning some noisy console logs in hope that they are serializing a…
29c3e50
…ccess and preventing the deadlock. This is RC1 stop-gap.

pavelsavara merged commit 7ade2b4 into dotnet:mainAug 6, 2022

ghost locked as resolved and limited conversation to collaborators Sep 5, 2022

pavelsavara deleted the fix_crypto_crash branch November 29, 2022 16:22

Uh oh!

Conversation

pavelsavara commented Jul 29, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Jul 29, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pavelsavara commented Aug 1, 2022

Uh oh!

azure-pipelinesBot commented Aug 1, 2022

Uh oh!

pavelsavara commented Aug 1, 2022

Uh oh!

azure-pipelinesBot commented Aug 1, 2022

Uh oh!

Uh oh!

Uh oh!

lambdageek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pavelsavara commented Aug 1, 2022

Uh oh!

pavelsavara commented Aug 1, 2022

Uh oh!

azure-pipelinesBot commented Aug 1, 2022

Uh oh!

pavelsavara commented Aug 1, 2022

Uh oh!

eerhardt commented Aug 1, 2022

Uh oh!

lambdageek commented Aug 1, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pavelsavara commented Aug 1, 2022

Uh oh!

pavelsavara commented Aug 1, 2022

Uh oh!

azure-pipelinesBot commented Aug 1, 2022

Uh oh!

azure-pipelinesBot commented Aug 5, 2022

Uh oh!

pavelsavara commented Aug 5, 2022

Uh oh!

azure-pipelinesBot commented Aug 5, 2022

Uh oh!

eerhardt commented Aug 5, 2022

Uh oh!

pavelsavara commented Aug 5, 2022

Uh oh!

eerhardt commented Aug 5, 2022

Uh oh!

pavelsavara commented Aug 5, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pavelsavara commented Aug 5, 2022

Uh oh!

pavelsavara commented Aug 5, 2022

Uh oh!

azure-pipelinesBot commented Aug 5, 2022

Uh oh!

eerhardt left a comment

Choose a reason for hiding this comment

Uh oh!

pavelsavara commented Aug 5, 2022

Uh oh!

eerhardt commented Aug 5, 2022

Uh oh!

pavelsavara commented Aug 5, 2022

Uh oh!

eerhardt commented Aug 5, 2022

Uh oh!

pavelsavara commented Aug 5, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

pavelsavara commented Jul 29, 2022•
edited
Loading

lambdageek commented Aug 1, 2022•
edited
Loading

pavelsavara commented Aug 5, 2022•
edited
Loading

pavelsavara commented Aug 5, 2022•
edited
Loading