[wasm] Add support for using custom native libraries by radical · Pull Request #55797 · dotnet/runtime

radical · 2021-07-16T04:50:37Z

Native libraries can be referenced via @(NativeReference) item, eg:

<NativeFileReference Include="NativeLib.o" />

JS files can be added via @(Content), with a Kind metadata, eg:

<Content Include="foo.js" Kind="pre-js" />

Native libraries can be referenced via `@(NativeReference)` item, eg: `<NativeReference Include="NativeLib.o" />` - JS files can be added via `@(Content)`, with a `Kind` metadata, eg: `<Content Include="foo.js" Kind="pre-js" />`

ghost · 2021-07-16T04:50:43Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Native libraries can be referenced via @(NativeReference) item, eg:

<NativeReference Include="NativeLib.o" />

JS files can be added via @(Content), with a Kind metadata, eg:

<Content Include="foo.js" Kind="pre-js" />

Author:	radical
Assignees:	-
Labels:	`arch-wasm`, `area-Build-mono`
Milestone:	-

lewing · 2021-07-16T13:47:48Z

failure is


Error message
System.ComponentModel.Win32Exception : Text file busy
Stack trace
at System.Diagnostics.Process.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) in System.Diagnostics.Process.dll:token 0x6000122+0xad
at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) in System.Diagnostics.Process.dll:token 0x6000121+0x1aa
at System.Diagnostics.Process.Start() in System.Diagnostics.Process.dll:token 0x60000fa+0xab
at Microsoft.DotNet.Cli.Build.Framework.Command.Start() in /_/src/installer/tests/TestUtils/Command.cs:line 199
at Microsoft.DotNet.Cli.Build.Framework.Command.Execute(Boolean fExpectedToFail) in /_/src/installer/tests/TestUtils/Command.cs:line 239
at Microsoft.DotNet.Cli.Build.Framework.Command.Execute() in /_/src/installer/tests/TestUtils/Command.cs:line 171
at AppHost.Bundle.Tests.NetCoreApp3CompatModeTests.Bundle_Is_Extracted() in /_/src/installer/tests/Microsoft.NET.HostModel.Tests/AppHost.Bundle.Tests/NetCoreApp3CompatModeTests.cs:line 33

https://dev.azure.com/dnceng/public/_build/results?buildId=1241841&view=ms.vss-test-web.build-test-results-tab&runId=36929180&resultId=100004&paneView=debug

.. and update the tests to use that.

lewing · 2021-07-19T01:19:01Z

/azp run runtime,runtime-staging

azure-pipelines · 2021-07-19T01:19:38Z

Azure Pipelines successfully started running 2 pipeline(s).

(cherry picked from commit d574b03)

…raries (#56013) * [wasm] Add support for using custom native libraries (#55797) (cherry picked from commit d574b03) * [wasm] Use compile rsp instead of link, for compiling native files (#55848) .. and fix logging that broke recently. `tasks/Common/Utils.cs`: TaskLoggingHelper Utils.Logger is a static field, which must be set by task else any methods in Utils, eg. RunProcess, silently fail to log any messages. Also, this would be a problem when building multiple projects in parallel, since the logger is a task-specific one. Instead, we pass logger as an arg to all the methods. (cherry picked from commit 3301e9d) * Link with EmccCompileOptimizationFlag==-Oz by default in release (#55939) (cherry picked from commit 04072ff) * [wasm] Fix regression in compiling .bc -> .o files (#56063) * [wasm] Add back --emit-llvm that got removed mistakenly, in an earlier commit .. found thanks to Jerome Laban. * [wasm] Set EmccCompile's messages to MessageImportance.Low by default. .. and to MessageImportance.Normal if `$(EmccVerbose)==true`. * [wasm] Quote filenames passed to emcc compile command line * Add more blazorwasm tests - for debug/release, aot/relinking * Bump sdk for workload testing to 6.0.100-rc.1.21370.2 * [wasm] Fix regression in compiling bitcode -> .o The `-emit-llvm` arg has been incorrectly added, and removed from the args used for compiling .bc->.o . This commit fixes it, and adds a crude test for it, so we don't regress again. * Fix build (cherry picked from commit 1d8ad03) * [wasm] Bump sdk for workload testing to 6.0.100-preview.7.21372.19 Co-authored-by: Larry Ewing <lewing@microsoft.com>

TL;dr `publish` fails on any blazorwasm project with VS17 A recent commit[1] moved initializing `$(_WasmIntermediateOutputPath)` from a target, to project level `PropertyGroup`. It is set as: `<_WasmIntermediateOutputPath>$([MSBuild]::NormalizeDirectory($(IntermediateOutputPath), 'wasm'))</_WasmIntermediateOutputPath>` The `NormalizeDirectory` call converts this to a full path, presumably using the current directory. Because we are setting `$(_WasmIntermediateOutputPath)` at the project level, it gets evaluated during the evaluation phase. And the current directory doesn't seem to be set to the project directory at that point in VS. So, `$(_WasmIntermediateOutputPath)` gets a wrong path like: `C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm`. And then when actually publishing, it fails to create this directory with: `Unable to create directory "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\". Access to the path 'C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\' is denied.` Fix: Set the property in `_InitializeCommonProperties` *target*, at which point the current directory is correctly set. Note: - This doesn't seem to be reproducible outside VS - It happens only on `publish`, because that's when the wasm targets come into play. -- 1. ``` commit d574b03 Author: Ankit Jain <radical@gmail.com> Date: Mon Jul 19 01:02:01 2021 -0400 [wasm] Add support for using custom native libraries (#55797) ```

TL;dr `publish` fails on any blazorwasm project with VS17 A recent commit[1] moved initializing `$(_WasmIntermediateOutputPath)` from a target, to project level `PropertyGroup`. It is set as: `<_WasmIntermediateOutputPath>$([MSBuild]::NormalizeDirectory($(IntermediateOutputPath), 'wasm'))</_WasmIntermediateOutputPath>` The `NormalizeDirectory` call converts this to a full path, presumably using the current directory. Because we are setting `$(_WasmIntermediateOutputPath)` at the project level, it gets evaluated during the evaluation phase. And the current directory doesn't seem to be set to the project directory at that point in VS. So, `$(_WasmIntermediateOutputPath)` gets a wrong path like: `C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm`. And then when actually publishing, it fails to create this directory with: `Unable to create directory "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\". Access to the path 'C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\' is denied.` Fix: Set the property in `_InitializeCommonProperties` *target*, at which point the current directory is correctly set. Note: - This doesn't seem to be reproducible outside VS - It happens only on `publish`, because that's when the wasm targets come into play. -- 1. ``` commit d574b03 Author: Ankit Jain <radical@gmail.com> Date: Mon Jul 19 01:02:01 2021 -0400 [wasm] Add support for using custom native libraries (#55797) ``` Co-authored-by: Ankit Jain <radical@gmail.com>

TL;dr `publish` fails on any blazorwasm project with VS17 A recent commit[1] moved initializing `$(_WasmIntermediateOutputPath)` from a target, to project level `PropertyGroup`. It is set as: `<_WasmIntermediateOutputPath>$([MSBuild]::NormalizeDirectory($(IntermediateOutputPath), 'wasm'))</_WasmIntermediateOutputPath>` The `NormalizeDirectory` call converts this to a full path, presumably using the current directory. Because we are setting `$(_WasmIntermediateOutputPath)` at the project level, it gets evaluated during the evaluation phase. And the current directory doesn't seem to be set to the project directory at that point in VS. So, `$(_WasmIntermediateOutputPath)` gets a wrong path like: `C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm`. And then when actually publishing, it fails to create this directory with: `Unable to create directory "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\". Access to the path 'C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\' is denied.` Fix: Set the property in `_InitializeCommonProperties` *target*, at which point the current directory is correctly set. Note: - This doesn't seem to be reproducible outside VS - It happens only on `publish`, because that's when the wasm targets come into play. -- 1. ``` commit d574b03 Author: Ankit Jain <radical@gmail.com> Date: Mon Jul 19 01:02:01 2021 -0400 [wasm] Add support for using custom native libraries (dotnet#55797) ```

TL;dr `publish` fails on any blazorwasm project with VS17 A recent commit[1] moved initializing `$(_WasmIntermediateOutputPath)` from a target, to project level `PropertyGroup`. It is set as: `<_WasmIntermediateOutputPath>$([MSBuild]::NormalizeDirectory($(IntermediateOutputPath), 'wasm'))</_WasmIntermediateOutputPath>` The `NormalizeDirectory` call converts this to a full path, presumably using the current directory. Because we are setting `$(_WasmIntermediateOutputPath)` at the project level, it gets evaluated during the evaluation phase. And the current directory doesn't seem to be set to the project directory at that point in VS. So, `$(_WasmIntermediateOutputPath)` gets a wrong path like: `C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm`. And then when actually publishing, it fails to create this directory with: `Unable to create directory "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\". Access to the path 'C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\' is denied.` Fix: Set the property in `_InitializeCommonProperties` *target*, at which point the current directory is correctly set. Note: - This doesn't seem to be reproducible outside VS - It happens only on `publish`, because that's when the wasm targets come into play. -- 1. ``` commit d574b03 Author: Ankit Jain <radical@gmail.com> Date: Mon Jul 19 01:02:01 2021 -0400 [wasm] Add support for using custom native libraries (#55797) ```

[wasm] Add support for using custom native libraries
d68b40b
Native libraries can be referenced via `@(NativeReference)` item, eg: `<NativeReference Include="NativeLib.o" />` - JS files can be added via `@(Content)`, with a `Kind` metadata, eg: `<Content Include="foo.js" Kind="pre-js" />`

radical added arch-wasm WebAssembly architecture area-Build-mono labels Jul 16, 2021

radical requested review from lewing and vargaz July 16, 2021 04:50

radical requested a review from marek-safar as a code owner July 16, 2021 04:50

radical added 4 commits July 16, 2021 00:56

Add license header for new source files
47a4d0b

Use @(NativeFileReference) instead of @(NativeReference)
cfcaa6f

Add missing NativeLibraryTests.cs, and enable aot builds
1adbf31

Add missing NativeLib.o
7973d6f

lewing requested review from akoeplinger and radekdoulik July 16, 2021 18:47

radical force-pushed the wasm-native-libs branch from 30f7ae6 to 1ebae57Compare July 16, 2021 21:31

PInvokeTableGenerator: handle native libs with hyphen in their name
0ea5099
.. and update the tests to use that.

radical force-pushed the wasm-native-libs branch from 1ebae57 to 0ea5099Compare July 16, 2021 21:53

lewing approved these changes Jul 19, 2021
View reviewed changes

radical merged commit d574b03 into dotnet:mainJul 19, 2021

radical deleted the wasm-native-libs branch July 19, 2021 05:02

ManickaP mentioned this pull request Jul 20, 2021
[QUIC] Remove AppContext switch from S.N.Quic #56027
Merged

radical added a commit to radical/runtime that referenced this pull request Jul 22, 2021
[wasm] Add support for using custom native libraries (dotnet#55797)
b0c2e73
(cherry picked from commit d574b03)

radical mentioned this pull request Jul 22, 2021
[release/6.0-preview7] [wasm] Add support for using custom native libraries #56013
Merged

karelz mentioned this pull request Jul 28, 2021
MsQuic tests hang / long running #56487
Closed

ghost locked as resolved and limited conversation to collaborators Aug 18, 2021

Navigation Menu

Uh oh!

[wasm] Add support for using custom native libraries - #55797

[wasm] Add support for using custom native libraries#55797
radical merged 6 commits into
dotnet:mainfrom
radical:wasm-native-libs

radical commented Jul 16, 2021•
edited
Loading

Uh oh!

ghost commented Jul 16, 2021

Uh oh!

lewing commented Jul 16, 2021

Uh oh!

lewing commented Jul 19, 2021

Uh oh!

azure-pipelinesBot commented Jul 19, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

radical commented Jul 16, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Jul 16, 2021

Uh oh!

lewing commented Jul 16, 2021

Uh oh!

lewing commented Jul 19, 2021

Uh oh!

azure-pipelinesBot commented Jul 19, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

radical commented Jul 16, 2021•
edited
Loading