Use utf conversions from minipal

[LakshanF]

Leverage the UTF conversion functionality needed for EventPipe from minipal.

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Leverage the UTF conversion functionality needed for EventPipe from minipal.

Author:	LakshanF
Assignees:	LakshanF
Labels:	area-NativeAOT-coreclr
Milestone:	8.0.0

[am11]

👍

[lewing]

The test failures here are known

[elinor-fung]

Why do we need the +1 here? Adding the null terminator is handled explicitly below?

[am11]

This is an existing pattern:

runtime/src/mono/mono/eglib/giconv.c

Lines 340 to 343 in 8f8a007

	if (len<0)
	len= (glong)strlen(str) +1;
	glongret= (glong)minipal_get_length_utf8_to_utf16 (str, len, flags);

Reason is, minipal get_length API needs the source length with null-terminator accounted for, so it returns the correct destination count. Handling it within minipal was requiring some additional conditions which we decided to leave out for the caller to handle.

[elinor-fung]

Does the length in ep_rt_utf16_to_utf8_string need +1 too, then?

[jkotas]

This is an existing pattern:

It has the same issue of allocating one extra byte than necessary, and it is further complicated by returning items_written.

We do not need to replicate the issue here.

[am11]

@jkotas, not sure which issue you are referring to. The easiest way to understand what goes wrong without +1 is to remove it from coreclr version (it's on the callsite of PAL API) and run PAL tests. fb613a5 did the same thing.

[jkotas]

@jkotas, not sure which issue you are referring to.

Allocating buffer that is one character longer than necessary.

For example, if ep_rt_utf8_to_utf16le_string is called with a string that is 1 ascii character + zero terminator, the current code allocates 3 * sizeof(CHAR16_T). It is one CHAR16_T more than what is actually needed.

[LakshanF]

It seems like the pattern with the extra +1 that @am11 mentions is needed to preserve the current behavior. For example, this test fails without +1.

[jkotas]

Are you saying that the event pipe code expects the strings returned by this method to have double zero terminators? It is very atypical contract. If it is really the case, these should be a long comment explaining it.

[am11]

Allocating buffer that is one character longer than necessary.

PAL, COM and reflection tests are very sensitive about "fill the exact slot or fail" cases. So if that was the case, they all will fail. IOW, that +1 effect gets cancelled out how the code in utf8.c works.

[elinor-fung]

nit: rename to str_utf8 and move the declaration down to where it is first assigned.