Uh oh!
There was an error while loading. Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork 5.6k
[browser][non-icu] HybridGlobalization change case#84019
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uh oh!
There was an error while loading. Please reload this page.
Changes from all commits
e9b6ebb1fb1e22c8e80ee738c702f397168d4de1267a1b55c30c7039f6f9fcfa8ef06076abac8376d12dceee24a9afe4164349310ab8fb1089cbb73de57dec9066e111d1564552935e3c082aea1f8d4f91aea9a8ef626b4File filter
Filter by extension
Conversations
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Hybrid Globalization | ||
| Description, purpose and instruction how to use. | ||
| ## Behavioral differences | ||
| Hybrid mode does not use ICU data for some functions connected with globalization but relies on functions native to the platform. Because native APIs do not fully cover all the functionalities we currently support and because ICU data can be excluded from the ICU datafile only in batches defined by ICU filters, not all functions will work the same way or not all will be supported. To see what to expect after switching on `HybridGlobalization`, read the following paragraphs. | ||
| ### WASM | ||
| For WebAssembly, both on Browser and WASI, we are using Web API instead of some ICU data. | ||
| **Case change** | ||
| Affected public APIs: | ||
| - TextInfo.ToLower, | ||
| - TextInfo.ToUpper, | ||
| - TextInfo.ToTitleCase. | ||
| Case change with invariant culture uses `toUpperCase` / `toLoweCase` functions that do not guarantee a full match with the original invariant culture. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,5 @@ | ||
| Microsoft Visual Studio Solution File, Format Version 12.00 | ||
| # | ||
| Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "System.Private.CoreLib", "..\..\coreclr\System.Private.CoreLib\System.Private.CoreLib.csproj", "{E269F8BB-F629-4C96-B9B2-03A00D8B1BFB}" | ||
| EndProject | ||
| Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "TestUtilities.Unicode", "..\Common\tests\TestUtilities.Unicode\TestUtilities.Unicode.csproj", "{79613DED-481D-44EF-BB89-7AC6BD53026B}" | ||
| @@ -33,6 +34,10 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "ref", "ref", "{A93AFF96-DB2 | ||
| EndProject | ||
| Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "gen", "gen", "{0378EF1C-9838-4AD0-867D-506FB02F8BBB}" | ||
| EndProject | ||
| Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Hybrid.IOS.Tests", "tests\Hybrid\Hybrid.IOS.Tests.csproj", "{16D9996B-A4E1-440B-8D74-C9ED3715158D}" | ||
| EndProject | ||
| Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Hybrid.WASM.Tests", "tests\Hybrid\Hybrid.WASM.Tests.csproj", "{CAA35471-75A3-41A8-B09D-0CC9822A8E3B}" | ||
| EndProject | ||
ilonatommy marked this conversation as resolved.
Uh oh!There was an error while loading. Please reload this page. | ||
| Global | ||
| GlobalSection(SolutionConfigurationPlatforms) = preSolution | ||
| Debug|Any CPU = Debug|Any CPU | ||
| @@ -244,6 +249,42 @@ Global | ||
| {41F80FEC-8515-455F-AC3E-D88B6CAAF8DA}.Checked|Any CPU.ActiveCfg = Debug|Any CPU | ||
| {41F80FEC-8515-455F-AC3E-D88B6CAAF8DA}.Checked|x64.ActiveCfg = Debug|Any CPU | ||
| {41F80FEC-8515-455F-AC3E-D88B6CAAF8DA}.Checked|x86.ActiveCfg = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Debug|Any CPU.Build.0 = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Debug|x64.ActiveCfg = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Debug|x64.Build.0 = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Debug|x86.ActiveCfg = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Debug|x86.Build.0 = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Release|Any CPU.ActiveCfg = Release|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Release|Any CPU.Build.0 = Release|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Release|x64.ActiveCfg = Release|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Release|x64.Build.0 = Release|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Release|x86.ActiveCfg = Release|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Release|x86.Build.0 = Release|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Checked|Any CPU.ActiveCfg = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Checked|Any CPU.Build.0 = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Checked|x64.ActiveCfg = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Checked|x64.Build.0 = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Checked|x86.ActiveCfg = Debug|Any CPU | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D}.Checked|x86.Build.0 = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Debug|Any CPU.Build.0 = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Debug|x64.ActiveCfg = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Debug|x64.Build.0 = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Debug|x86.ActiveCfg = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Debug|x86.Build.0 = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Release|Any CPU.ActiveCfg = Release|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Release|Any CPU.Build.0 = Release|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Release|x64.ActiveCfg = Release|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Release|x64.Build.0 = Release|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Release|x86.ActiveCfg = Release|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Release|x86.Build.0 = Release|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Checked|Any CPU.ActiveCfg = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Checked|Any CPU.Build.0 = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Checked|x64.ActiveCfg = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Checked|x64.Build.0 = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Checked|x86.ActiveCfg = Debug|Any CPU | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B}.Checked|x86.Build.0 = Debug|Any CPU | ||
| EndGlobalSection | ||
| GlobalSection(SolutionProperties) = preSolution | ||
| HideSolutionNode = FALSE | ||
| @@ -262,6 +303,8 @@ Global | ||
| {74CAB3C9-1AE1-467E-B139-35E7113F4660} = {0378EF1C-9838-4AD0-867D-506FB02F8BBB} | ||
| {12E788BB-7E58-4780-B52E-DB5A91A49DFF} = {0378EF1C-9838-4AD0-867D-506FB02F8BBB} | ||
| {F4A35959-8F1B-4CA9-B672-3ACFBDD54174} = {0378EF1C-9838-4AD0-867D-506FB02F8BBB} | ||
| {16D9996B-A4E1-440B-8D74-C9ED3715158D} = {C223E72F-FD21-43C3-AC7A-62BCF4A5C379} | ||
| {CAA35471-75A3-41A8-B09D-0CC9822A8E3B} = {C223E72F-FD21-43C3-AC7A-62BCF4A5C379} | ||
| EndGlobalSection | ||
ilonatommy marked this conversation as resolved.
Uh oh!There was an error while loading. Please reload this page. | ||
| GlobalSection(ExtensibilityGlobals) = postSolution | ||
| SolutionGuid = {33E0B3D0-C6E1-4B75-A025-AE012AD424F7} | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -7,4 +7,4 @@ | ||
| <ItemGroup> | ||
| <Compile Include="HybridMode.cs" /> | ||
| </ItemGroup> | ||
| </Project> | ||
| </Project> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
| <PropertyGroup> | ||
| <TargetFramework>$(NetCoreAppCurrent)</TargetFramework> | ||
| <TestRuntime>true</TestRuntime> | ||
| <HybridGlobalization>true</HybridGlobalization> | ||
| </PropertyGroup> | ||
| <ItemGroup> | ||
| <Compile Include="..\System\Globalization\TextInfoTests.cs" /> | ||
| </ItemGroup> | ||
| </Project> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -13,7 +13,7 @@ internal static partial class GlobalizationMode | ||
| private static partial class Settings | ||
| { | ||
| internal static bool Invariant { get; } = AppContextConfigHelper.GetBooleanConfig("System.Globalization.Invariant", "DOTNET_SYSTEM_GLOBALIZATION_INVARIANT"); | ||
| #if TARGET_OSX || TARGET_MACCATALYST || TARGET_IOS || TARGET_TVOS | ||
ilonatommy marked this conversation as resolved.
Uh oh!There was an error while loading. Please reload this page. | ||
| #if TARGET_OSX || TARGET_MACCATALYST || TARGET_IOS || TARGET_TVOS || TARGET_BROWSER || TARGET_WASI | ||
Member There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. no WASI ? | ||
| internal static bool Hybrid { get; } = AppContextConfigHelper.GetBooleanConfig("System.Globalization.Hybrid", "DOTNET_SYSTEM_GLOBALIZATION_HYBRID"); | ||
| #endif | ||
| internal static bool PredefinedCulturesOnly { get; } = AppContextConfigHelper.GetBooleanConfig("System.Globalization.PredefinedCulturesOnly", "DOTNET_SYSTEM_GLOBALIZATION_PREDEFINED_CULTURES_ONLY", GlobalizationMode.Invariant); | ||
| @@ -23,7 +23,7 @@ private static partial class Settings | ||
| // This allows for the whole Settings nested class to be trimmed when Invariant=true, and allows for the Settings | ||
| // static cctor (on Unix) to be preserved when Invariant=false. | ||
| internal static bool Invariant => Settings.Invariant; | ||
| #if TARGET_OSX || TARGET_MACCATALYST || TARGET_IOS || TARGET_TVOS | ||
| #if TARGET_OSX || TARGET_MACCATALYST || TARGET_IOS || TARGET_TVOS || TARGET_BROWSER || TARGET_WASI | ||
Member There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. no WASI ? | ||
| internal static bool Hybrid => Settings.Hybrid; | ||
| #endif | ||
| internal static bool PredefinedCulturesOnly => Settings.PredefinedCulturesOnly; | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
| using System; | ||
| using System.Diagnostics; | ||
| using System.Runtime.CompilerServices; | ||
| namespace System.Globalization | ||
| { | ||
| internal static unsafe class TextInfoInterop | ||
| { | ||
| [MethodImplAttribute(MethodImplOptions.InternalCall)] | ||
| internal static extern unsafe void ChangeCaseInvariantJS(out string exceptionMessage, char* src, int srcLen, char* dstBuffer, int dstBufferCapacity, bool bToUpper); | ||
| [MethodImplAttribute(MethodImplOptions.InternalCall)] | ||
| internal static extern unsafe void ChangeCaseJS(out string exceptionMessage, in string culture, char* src, int srcLen, char* dstBuffer, int dstBufferCapacity, bool bToUpper); | ||
| } | ||
| public partial class TextInfo | ||
| { | ||
| internal unsafe void JsChangeCase(char* src, int srcLen, char* dstBuffer, int dstBufferCapacity, bool toUpper) | ||
| { | ||
| Debug.Assert(!GlobalizationMode.Invariant); | ||
| Debug.Assert(!GlobalizationMode.UseNls); | ||
ilonatommy marked this conversation as resolved.
Uh oh!There was an error while loading. Please reload this page. | ||
| Debug.Assert(GlobalizationMode.Hybrid); | ||
| string exceptionMessage; | ||
| if (HasEmptyCultureName) | ||
| { | ||
| TextInfoInterop.ChangeCaseInvariantJS(out exceptionMessage, src, srcLen, dstBuffer, dstBufferCapacity, toUpper); | ||
| } | ||
| else | ||
| { | ||
| TextInfoInterop.ChangeCaseJS(out exceptionMessage, _cultureName, src, srcLen, dstBuffer, dstBufferCapacity, toUpper); | ||
| } | ||
| if (!string.IsNullOrEmpty(exceptionMessage)) | ||
| throw new Exception(exceptionMessage); | ||
| } | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -28,6 +28,7 @@ | ||
| - $(WasmProfilers) - Profilers to use | ||
| - $(AOTProfilePath) - profile data file to be used for profile-guided optimization | ||
| - $(InvariantGlobalization) - Whenever to disable ICU. Defaults to false. | ||
| - $(HybridGlobalization) - Whenever to enable reduced ICU + native platform functions. Defaults to false and can be set only for InvariantGlobalization=false, WasmIncludeFullIcuData=false and empty WasmIcuDataFileName. | ||
| - $(WasmResolveAssembliesBeforeBuild) - Resolve the assembly dependencies. Defaults to false | ||
| - $(WasmAssemblySearchPaths) - used for resolving assembly dependencies | ||
| @@ -71,7 +72,7 @@ | ||
| - $(WasmEnableExceptionHandling) - Enable support for the WASM Exception Handling feature. | ||
| - $(WasmEnableSIMD) - Enable support for the WASM SIMD feature. | ||
| - $(WasmEnableWebcil) - Enable conversion of assembly .dlls to .webcil | ||
| - $(WasmIncludeFullIcuData) - Loads full ICU data (icudt.dat). Defaults to false. Only applicable when InvariantGlobalization=false. | ||
| - $(WasmIncludeFullIcuData) - Loads full ICU data (icudt.dat). Defaults to false. Only applicable when InvariantGlobalization=false. | ||
| - $(WasmIcuDataFileName) - Name/path of ICU globalization file loaded to app. Only when InvariantGloblization=false and WasmIncludeFullIcuData=false. | ||
| - $(WasmAllowUndefinedSymbols) - Controls whether undefined symbols are allowed or not, | ||
| if true, appends 'allow-undefined' and sets 'ERROR_ON_UNDEFINED_SYMBOLS=0' as arguments for wasm-ld, | ||
| @@ -326,7 +327,9 @@ | ||
| </Target> | ||
| <Target Name="_GetWasmGenerateAppBundleDependencies"> | ||
| <Warning Condition="'$(InvariantGlobalization)' == 'true' and '$(HybridGlobalization)' == 'true'" Text="%24(HybridGlobalization) has no effect when %24(InvariantGlobalization) is set to true." /> | ||
| <PropertyGroup> | ||
| <HybridGlobalization Condition="'$(InvariantGlobalization)' == 'true'">false</HybridGlobalization> | ||
ilonatommy marked this conversation as resolved.
Uh oh!There was an error while loading. Please reload this page. | ||
| <_HasDotnetWasm Condition="'%(WasmNativeAsset.FileName)%(WasmNativeAsset.Extension)' == 'dotnet.wasm'">true</_HasDotnetWasm> | ||
| <_HasDotnetJsWorker Condition="'%(WasmNativeAsset.FileName)%(WasmNativeAsset.Extension)' == 'dotnet.worker.js'">true</_HasDotnetJsWorker> | ||
| <_HasDotnetJsSymbols Condition="'%(WasmNativeAsset.FileName)%(WasmNativeAsset.Extension)' == 'dotnet.js.symbols'">true</_HasDotnetJsSymbols> | ||
| @@ -335,6 +338,11 @@ | ||
| <_WasmIcuDataFileName Condition="'$(WasmIcuDataFileName)' != '' and !Exists('$(WasmIcuDataFileName)')">$(MicrosoftNetCoreAppRuntimePackRidNativeDir)$(WasmIcuDataFileName)</_WasmIcuDataFileName> | ||
| </PropertyGroup> | ||
| <PropertyGroup Condition="'$(HybridGlobalization)' == 'true' and '$(WasmIcuDataFileName)' == ''"> | ||
| <!-- to be renamed to icudt_wasm.dat when the contents of the file get defined and it gets added to repo --> | ||
| <_WasmIcuDataFileName>$(MicrosoftNetCoreAppRuntimePackRidNativeDir)icudt.dat</_WasmIcuDataFileName> | ||
| </PropertyGroup> | ||
| <ItemGroup> | ||
| <!-- If dotnet.{wasm,js} weren't added already (eg. AOT can add them), then add the default ones --> | ||
| <WasmNativeAsset Include="$(MicrosoftNetCoreAppRuntimePackRidNativeDir)dotnet.wasm" Condition="'$(_HasDotnetWasm)' != 'true'" /> | ||
| @@ -383,6 +391,7 @@ | ||
| RuntimeArgsForHost="@(WasmMonoRuntimeArgs)" | ||
| DefaultHostConfig="$(DefaultWasmHostConfig)" | ||
| InvariantGlobalization="$(InvariantGlobalization)" | ||
| HybridGlobalization="$(HybridGlobalization)" | ||
| SatelliteAssemblies="@(_WasmSatelliteAssemblies)" | ||
| FilesToIncludeInFileSystem="@(WasmFilesToIncludeInFileSystem)" | ||
| IcuDataFileNames="@(WasmIcuDataFileNames)" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -42,6 +42,10 @@ extern void mono_wasm_typed_array_from_ref (int ptr, int begin, int end, int byt | ||
| extern void* mono_wasm_invoke_js_blazor (MonoString **exceptionMessage, void *callInfo, void* arg0, void* arg1, void* arg2); | ||
| #endif /* ENABLE_LEGACY_JS_INTEROP */ | ||
| // HybridGlobalization | ||
| extern void mono_wasm_change_case_invariant(MonoString **exceptionMessage, const uint16_t* src, int32_t srcLength, uint16_t* dst, int32_t dstLength, mono_bool bToUpper); | ||
ilonatommy marked this conversation as resolved.
Uh oh!There was an error while loading. Please reload this page. | ||
| extern void mono_wasm_change_case(MonoString **exceptionMessage, MonoString **culture, const uint16_t* src, int32_t srcLength, uint16_t* dst, int32_t dstLength, mono_bool bToUpper); | ||
| void bindings_initialize_internals (void) | ||
| { | ||
| mono_add_internal_call ("System.Runtime.InteropServices.JavaScript.JSSynchronizationContext::ScheduleBackgroundJob", mono_threads_schedule_background_job); | ||
| @@ -69,4 +73,6 @@ void bindings_initialize_internals (void) | ||
| // Blazor specific custom routines - see dotnet_support.js for backing code | ||
| mono_add_internal_call ("WebAssembly.JSInterop.InternalCalls::InvokeJS", mono_wasm_invoke_js_blazor); | ||
| #endif /* ENABLE_LEGACY_JS_INTEROP */ | ||
| mono_add_internal_call ("System.Globalization.TextInfoInterop::ChangeCaseInvariantJS", mono_wasm_change_case_invariant); | ||
| mono_add_internal_call ("System.Globalization.TextInfoInterop::ChangeCaseJS", mono_wasm_change_case); | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no Web API in WASI. It probably means that the whole
Hybrid Globalizationcan't be implemented there.