[browser][non-icu] HybridGlobalization SortKey

[ilonatommy]

Implements a chunk of web-api based globalization. Is a part of HybridGlobalization feature and contributes to #79989.

Old, icu-based private API: GlobalizationNative_GetSortKey

New, non-icu private API: none, Web API does not expose SortKey. We throw PlatformNotSupportedException.

Affected public API (see: tests in CompareInfoTests.cs):

• CompareInfo.GetSortKey
• CompareInfo.GetSortKeyLength
• CompareInfo.GetHashCode

All changes in behavior are listed in docs\design\features\hybrid-globalization.md

This PR also refactors CompareInfo tests - it moves common properties/fields to CompareInfoTestsBase.

cc @SamMonoRT

[ghost]

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Implements a chunk of web-api based globalization. Is a part of HybridGlobalization feature and contributes to #79989.

Old, icu-based private API: GlobalizationNative_GetSortKey

New, non-icu private API: none, Web API does not expose SortKey. We throw PlatformNotSupportedException.

Affected public API (see: tests in CompareInfoTests.cs):

• CompareInfo.GetSortKey
• CompareInfo.GetSortKeyLength
• CompareInfo.GetHashCode

All changes in behavior are listed in docs\design\features\hybrid-globalization.md

This PR also refactors CompareInfo tests - it moves common properties/fields to CompareInfoTestsBase.

cc @SamMonoRT

Author:	ilonatommy
Assignees:	ilonatommy
Labels:	arch-wasm, area-System.Globalization
Milestone:	-

[ghost]

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Implements a chunk of web-api based globalization. Is a part of HybridGlobalization feature and contributes to #79989.

Old, icu-based private API: GlobalizationNative_GetSortKey

New, non-icu private API: none, Web API does not expose SortKey. We throw PlatformNotSupportedException.

Affected public API (see: tests in CompareInfoTests.cs):

• CompareInfo.GetSortKey
• CompareInfo.GetSortKeyLength
• CompareInfo.GetHashCode

All changes in behavior are listed in docs\design\features\hybrid-globalization.md

This PR also refactors CompareInfo tests - it moves common properties/fields to CompareInfoTestsBase.

cc @SamMonoRT

Author:	ilonatommy
Assignees:	ilonatommy
Labels:	arch-wasm, area-System.Globalization
Milestone:	-

[pavelsavara]

What are the size wins ?
What are the use-cases which would be affected by this change ?

[ilonatommy]

What are the size wins ? What are the use-cases which would be affected by this change ?

Size win:
lack of SortKey is a side effect of removing collations data from ICU. Overall size decrease on full WASM ICU file (icudt.dat) ~40% (329770 bytes -> 202330 bytes on brotli compressed).

Use-cases (only of changes from this PR, not the whole collation):

• CompareInfo.GetSortKey
• CompareInfo.GetSortKeyLength
• CompareInfo.GetHashCode

[pavelsavara]

(329770 bytes -> 202330 bytes on brotli compressed).

Thanks

Use-cases (only of changes from this PR, not the whole collation):

* CompareInfo.GetSortKey
* CompareInfo.GetSortKeyLength
* CompareInfo.GetHashCode

I mean what are the use-cases where those APIs are useful ?

This is what I tried https://github.com/search?q=CompareInfo.GetSortKey+lang%3AC%23+&type=code&p=3
I see mostly database oriented libraries.

Would it be better if we provide bitwise/invariant sortkey instead of PNSE ? Just an idea.

[ilonatommy]

I mean what are the use-cases where those APIs are useful ?

This is what I tried https://github.com/search?q=CompareInfo.GetSortKey+lang%3AC%23+&type=code&p=3 I see mostly database oriented libraries.

Sorry, I did not get it. It seems you are right, mostly DBs.

Would it be better if we provide bitwise/invariant sortkey instead of PNSE ? Just an idea.

It is not a bad idea but I think if we make it a rule for all locales, it could be confusing. User that calls e.g. new CultureInfo("de-DE").CompareInfo.GetSortKey() expects locale-sensitive sort key.
If they need invariant SortKey, they can switch on InvariantGlobalization and when they need locale-sensitive, functional SortKey, they should not use HybridGlobalization.
I can think about a use-case when your solution would be perfectly useful:

• the user needs to cut down on runtime size (HybridGlobalization on)
• they use sort key for locale-insensitive case, e.g.: CultureInfo.InvariantCulture.CompareInfo.GetSortKey()
• but do not want to switch on InvariantGlobalization becuse they need locale-sensitive operations somewhere else.

I will stash this idea in the tracking issue in case there is a demand on the above case.

[pavelsavara]

LGMT

What is our plan for Android/iOS where maybe Hybrid could support these methods ?

[pavelsavara]

Should this be in resource file and localized ?

[ilonatommy]

Good idea, I can see all exception text in PrivateCoreLib are stored like this, so you are right.

[ilonatommy]

What is our plan for Android/iOS where maybe Hybrid could support these methods ?

Android might have an invariant equivalent, it's dedicated for LDAP sorting: https://docs.oracle.com/javase/7/docs/api/javax/naming/ldap/SortKey.html. I do not see any culture-related SortKey.
For iOS I cannot find an equivalent, @mkhamoyan do you know about any?
Even if Hybrid SortKey will be supported on both of them, then PNSE will be left only for WASM.