[wasi+browser] bundle timezones into .wasm by pavelsavara · Pull Request #82250 · dotnet/runtime

pavelsavara · 2023-02-16T18:22:43Z

Motivation

the VFS is slow in the browser. I expect to save 50ms of startup time
this will bring de-duplication of files with same hash and bit smaller overall download size.
it will enable memory snapshot of the initialized runtime
necessary for WASI with <WasmSingleFileBundle>

Changes

TimeZoneInfo will use this new embedded TZ database for browser and wasi OS
- unless TZDIR env variable is set
- if TZDIR is set it would fallback to unix-like file system approach (as before).
added mono_wasm_get_bundled_file and mono_wasm_add_bundled_file
mono_wasm_add_bundled_file is called with file names from generated mono_wasm_register_bundle_timezones
added SystemNative_GetTimeZoneData which calls new mono_wasm_get_bundled_file
in of <WasmSingleFileBundle> we also generate mono_wasm_register_bundle_assemblies for WASI
removed dotnet.timezones.blat from runtime pack
added wasm-bundled-timezones.a to workloads
fixed TARGET_BROWSER detection for native/libs
do not set TZ env variable to UTC as that's not valid timezone anyway. dotnet will fallback to UTC.
create usr/share on browser VFS because it's mapped to SpecialFolder.CommonApplicationData which would not exist otherwise.
rewrote EmitWasmBundleObjectFile as new EmitWasmBundleFiles
- it can bundle any file not just assemblies
- it can generate code to to wasi clang input stream or to files for emcc
- it skips files which are already converted to
new Target GenerateTimezonesArchive in both wasi.proj and wasm.proj
- find all files to bundle, get hash for them
- pass the list to EmitWasmBundleFiles Task
  - [for wasi] object files will be produced
  - [for emcc] compile source files to obj file after this
- remove old source and object files
updated Target _GenerateAssemblyObjectFiles to use new EmitWasmBundleFiles task

future work

this will need SDK change for blazor
next PR make BlazorEnableTimeZoneSupport work again,
- when workload is available.
- via something like Add feature-switch for invariant TimeZone support #37973
make file bundling more incremental
similar bundling could be used for WASI and ICU data files

ghost · 2023-02-16T18:22:54Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

work in progress

Author:	pavelsavara
Assignees:	pavelsavara
Labels:	`arch-wasm`, `os-wasi`
Milestone:	8.0.0

pavelsavara · 2023-02-22T20:15:40Z

I will need to deal with BlazorEnableTimeZoneSupport
See SDK and #37973

…nix.NonAndroid.cs Co-authored-by: Ankit Jain <radical@gmail.com>

…fferent asset types

pavelsavara · 2023-02-28T11:04:50Z

@radical do you have further feedback ?

.

Co-authored-by: Ankit Jain <radical@gmail.com>

radical

LGTM 👍

radical · 2023-03-01T08:19:29Z

+ <RunWithEmSdkEnv Command="$(EmccCmd) -xc -c @$(_WasmTimezonesSourcesRsp)"
+ WorkingDirectory="$(WasmObjDir)"
+ EmSdkPath="$(EMSDK_PATH)"
+ ConsoleToMsBuild="true"
+ IgnoreStandardErrorWarningFormat="true">
+ </RunWithEmSdkEnv>


For a follow up PR, consider using EmccCompile task which compiles in parallel.

pavelsavara · 2023-03-01T11:54:36Z

/azp run runtime-wasm

azure-pipelines · 2023-03-01T11:54:53Z

Azure Pipelines successfully started running 1 pipeline(s).

ilonatommy

Looks good, I especially like the description ✨

pavelsavara · 2023-03-01T15:08:51Z

CI failures are unrelated.

pavelsavara · 2023-03-03T13:47:09Z

Original size of dotnet.timezones.blat was 336270 bytes.
The size of new data embedded in the dotnet.wasm is 221991 bytes.

Some of the startup speed improvement is also caused by it.

pavelsavara added arch-wasm WebAssembly architecture os-wasi Related to WASI variant of arch-wasm labels Feb 16, 2023

pavelsavara added this to the 8.0.0 milestone Feb 16, 2023

pavelsavara self-assigned this Feb 16, 2023

ghost added the area-Build-mono label Feb 16, 2023

This was referenced Feb 16, 2023
OSX infra issue - prereq check for 'pkg-config' missing #82240
Closed
Roslyn source generator crash on mono/linux/arm64 #81123
Closed
SupportedLinuxPlatforms_IsSupportedIsTrue fails on distros without quic #82077
Closed

jakobbotsch mentioned this pull request Feb 17, 2023
[QUIC] Missing libmsquic on tested platforms #81901
Closed
4 tasks

pavelsavara force-pushed the wasi_bundle_timezones branch 3 times, most recently from 18e709c to a45ecbfCompare February 21, 2023 17:00

pavelsavara changed the title ~~[wasi] bundle timezones into .wasm~~[wasi+browser] bundle timezones into .wasmFeb 21, 2023

pavelsavara commented Feb 21, 2023
View reviewed changes

Comment threadsrc/native/libs/System.Native/entrypoints.c Outdated

build-analysisBot mentioned this pull request Feb 21, 2023
Tracking issue for CI build timeouts #76454
Closed

pavelsavara force-pushed the wasi_bundle_timezones branch from 9c2f6d6 to 3d4cfb7Compare February 22, 2023 10:34

wip
89d4f74

pavelsavara force-pushed the wasi_bundle_timezones branch from 37a1350 to 89d4f74Compare February 22, 2023 17:47

Merge branch 'main' into wasi_bundle_timezones
c0e9a8b

pavelsavara requested a review from mkhamoyan February 23, 2023 15:42

pavelsavara marked this pull request as ready for review February 23, 2023 15:52

pavelsavara requested review from kg, lambdageek, lewing, marek-safar, radical and vargaz as code owners February 23, 2023 15:52

fix
42db50f

pavelsavaraand others added 5 commits February 27, 2023 09:25

Update src/libraries/System.Private.CoreLib/src/System/TimeZoneInfo.U…
ac2521d
…nix.NonAndroid.cs Co-authored-by: Ankit Jain <radical@gmail.com>

feedback
8cbf72f

added /usr/share/zoneinfo/ prefix - namespace so that we don't mix di…
01d3c60
…fferent asset types

Merge branch 'main' into wasi_bundle_timezones
9d3e75c

fix
26d5f31

pavelsavara requested a review from radical February 27, 2023 15:50

pavelsavara mentioned this pull request Feb 28, 2023
[wasi][icu] Add ICU sharding for wasi #82614
Open

radical reviewed Mar 1, 2023
View reviewed changes

Comment threadsrc/libraries/System.Private.CoreLib/src/System/TimeZoneInfo.Unix.NonAndroid.cs
Comment threadsrc/libraries/System.Private.CoreLib/src/System/TimeZoneInfo.Unix.NonAndroid.cs
Comment threadsrc/tasks/WasmAppBuilder/EmitWasmBundleBase.cs Outdated

pavelsavaraand others added 2 commits March 1, 2023 08:33

Update src/tasks/WasmAppBuilder/EmitWasmBundleBase.cs
c8af4b8
Co-authored-by: Ankit Jain <radical@gmail.com>

feedback
e674fd7

radical approved these changes Mar 1, 2023
View reviewed changes

radical reviewed Mar 1, 2023
View reviewed changes

maraf approved these changes Mar 1, 2023
View reviewed changes

ilonatommy approved these changes Mar 1, 2023
View reviewed changes

Comment threadsrc/mono/mono/utils/mono-dl-wasm.c

This was referenced Mar 1, 2023
Test failure: System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests/CreateChain_Hybrid #25979
Closed
System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice failure #65448
Closed

pavelsavara merged commit 340508f into dotnet:mainMar 1, 2023

This was referenced Mar 1, 2023
System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice failure #82837
Open
Some System.Buffers.Binary.Tests.ReverseEndiannessUnitTests tests broken on main#82868
Closed

mkhamoyan mentioned this pull request Mar 2, 2023
[wasi][icu] bundle icu files into .wasm #82895
Closed

pavelsavara mentioned this pull request Mar 3, 2023
Add feature-switch for invariant TimeZone support #37973
Closed

ilonatommy mentioned this pull request Mar 28, 2023
Update off-by-default analyzers to account for rename dotnet/sdk#31460
Merged

ghost locked as resolved and limited conversation to collaborators Apr 2, 2023

pavelsavara deleted the wasi_bundle_timezones branch September 2, 2024 15:29

Navigation Menu

Uh oh!

[wasi+browser] bundle timezones into .wasm - #82250

[wasi+browser] bundle timezones into .wasm#82250
pavelsavara merged 26 commits into
dotnet:mainfrom
pavelsavara:wasi_bundle_timezones

pavelsavara commented Feb 16, 2023•
edited
Loading

Uh oh!

ghost commented Feb 16, 2023

Uh oh!

Uh oh!

pavelsavara commented Feb 22, 2023•
edited
Loading

Uh oh!

pavelsavara commented Feb 28, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

radical left a comment

Uh oh!

radicalMar 1, 2023

Uh oh!

pavelsavara commented Mar 1, 2023

Uh oh!

azure-pipelinesBot commented Mar 1, 2023

Uh oh!

ilonatommy left a comment

Uh oh!

Uh oh!

pavelsavara commented Mar 1, 2023

Uh oh!

pavelsavara commented Mar 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

pavelsavara commented Feb 16, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

future work

Uh oh!

ghost commented Feb 16, 2023

Uh oh!

Uh oh!

pavelsavara commented Feb 22, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pavelsavara commented Feb 28, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

radical left a comment

Choose a reason for hiding this comment

Uh oh!

radicalMar 1, 2023

Choose a reason for hiding this comment

Uh oh!

pavelsavara commented Mar 1, 2023

Uh oh!

azure-pipelinesBot commented Mar 1, 2023

Uh oh!

ilonatommy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pavelsavara commented Mar 1, 2023

Uh oh!

pavelsavara commented Mar 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pavelsavara commented Feb 16, 2023•
edited
Loading

pavelsavara commented Feb 22, 2023•
edited
Loading