[API Proposal] Expose missing BinaryPrimitives APIs

[tannergooding]

System.Buffers.Binary.BinaryPrimitives exposes many APIs but there are few notable ones missing for core number types. We should extend the type to ensure that it supports the basic operations for all basic number types.

namespace System.Buffers.Binary;

public static partial class BinaryPrimitives
{
 // Read*BigEndian and Read*LittleEndian

 public static nint ReadIntPtrBigEndian(ReadOnlySpan<byte> source);
 public static nint ReadIntPtrLittleEndian(ReadOnlySpan<byte> source);

 public static Int128 ReadInt128BigEndian(ReadOnlySpan<byte> source);
 public static Int128 ReadInt128LittleEndian(ReadOnlySpan<byte> source);

 public static nuint ReadUIntPtrBigEndian(ReadOnlySpan<byte> source);
 public static nuint ReadUIntPtrLittleEndian(ReadOnlySpan<byte> source);

 public static UInt128 ReadUInt128BigEndian(ReadOnlySpan<byte> source);
 public static UInt128 ReadUInt128LittleEndian(ReadOnlySpan<byte> source);

 // ReverseEndianness

 public static nint ReverseEndianness(nint value);
 public static Int128 ReverseEndianness(Int128 value);
 public static nuint ReverseEndianness(nuint value);
 public static UInt128 ReverseEndianness(UInt128 value);

 // TryRead*BigEndian and TryRead*LittleEndian

 public static bool TryReadIntPtrBigEndian(ReadOnlySpan<byte> source, out nint value);
 public static bool TryReadIntPtrLittleEndian(ReadOnlySpan<byte> source, out nint value);

 public static bool TryReadInt128BigEndian(ReadOnlySpan<byte> source, out Int128 value);
 public static bool TryReadInt128LittleEndian(ReadOnlySpan<byte> source, out Int128 value);

 public static bool TryReadUIntPtrBigEndian(ReadOnlySpan<byte> source, out nuint value);
 public static bool TryReadUIntPtrLittleEndian(ReadOnlySpan<byte> source, out nuint value);

 public static bool TryReadUInt128BigEndian(ReadOnlySpan<byte> source, out UInt128 value);
 public static bool TryReadUInt128LittleEndian(ReadOnlySpan<byte> source, out UInt128 value);

 // TryWrite*BigEndian and TryWrite*LittleEndian

 public static bool TryWriteIntPtrBigEndian(ReadOnlySpan<byte> destination, nint value);
 public static bool TryWriteIntPtrLittleEndian(ReadOnlySpan<byte> destination, nint value);

 public static bool TryWriteInt128BigEndian(ReadOnlySpan<byte> destination, Int128 value);
 public static bool TryWriteInt128LittleEndian(ReadOnlySpan<byte> destination, Int128 value);

 public static bool TryWriteUIntPtrBigEndian(ReadOnlySpan<byte> destination, nuint value);
 public static bool TryWriteUIntPtrLittleEndian(ReadOnlySpan<byte> destination, nuint value);

 public static bool TryWriteUInt128BigEndian(ReadOnlySpan<byte> destination, UInt128 value);
 public static bool TryWriteUInt128LittleEndian(ReadOnlySpan<byte> destination, UInt128 value);

 // Write*BigEndian and Write*LittleEndian

 public static void WriteIntPtrBigEndian(ReadOnlySpan<byte> destination, nint value);
 public static void WriteIntPtrLittleEndian(ReadOnlySpan<byte> destination, nint value);

 public static void WriteInt128BigEndian(ReadOnlySpan<byte> destination, Int128 value);
 public static void WriteInt128LittleEndian(ReadOnlySpan<byte> destination, Int128 value);

 public static void WriteUIntPtrBigEndian(ReadOnlySpan<byte> destination, nuint value);
 public static void WriteUIntPtrLittleEndian(ReadOnlySpan<byte> destination, nuint value);

 public static void WriteUInt128BigEndian(ReadOnlySpan<byte> destination, UInt128 value);
 public static void WriteUInt128LittleEndian(ReadOnlySpan<byte> destination, UInt128 value);
}

Additional Notes

Also missing are some APIs around char, Half, float, and double. However, there are concerns around char not being a "core" numeric type and around possible normalization issues for the various floating-point types, so they aren't included.

Tagging subscribers to this area: @dotnet/area-system-buffers
See info in area-owners.md if you want to be subscribed.

Issue Details

---

System.Buffers.Binary.BinaryPrimitives exposes many APIs but there are few notable ones missing for core number types. We should extend the type to ensure that it supports the basic operations for all basic number types.

namespace System.Buffers.Binary;

public static partial class BinaryPrimitives
{
 // Read*BigEndian and Read*LittleEndian

 public static char ReadCharBigEndian(ReadOnlySpan<byte> source);
 public static char ReadCharLittleEndian(ReadOnlySpan<byte> source);

 public static nint ReadIntPtrBigEndian(ReadOnlySpan<byte> source);
 public static nint ReadIntPtrLittleEndian(ReadOnlySpan<byte> source);

 public static Int128 ReadInt128BigEndian(ReadOnlySpan<byte> source);
 public static Int128 ReadInt128LittleEndian(ReadOnlySpan<byte> source);

 public static nuint ReadUIntPtrBigEndian(ReadOnlySpan<byte> source);
 public static nuint ReadUIntPtrLittleEndian(ReadOnlySpan<byte> source);

 public static UInt128 ReadUInt128BigEndian(ReadOnlySpan<byte> source);
 public static UInt128 ReadUInt128LittleEndian(ReadOnlySpan<byte> source);

 // ReverseEndianness

 public static char ReverseEndianness(char value);
 public static double ReverseEndianness(double value);
 public static Half ReverseEndianness(Half value);
 public static nint ReverseEndianness(nint value);
 public static Int128 ReverseEndianness(Int128 value);
 public static float ReverseEndianness(float value);
 public static nuint ReverseEndianness(nuint value);
 public static UInt128 ReverseEndianness(UInt128 value);

 // TryRead*BigEndian and TryRead*LittleEndian

 public static bool TryReadCharBigEndian(ReadOnlySpan<byte> source, out char value);
 public static bool TryReadCharLittleEndian(ReadOnlySpan<byte> source, out char value);

 public static bool TryReadIntPtrBigEndian(ReadOnlySpan<byte> source, out nint value);
 public static bool TryReadIntPtrLittleEndian(ReadOnlySpan<byte> source, out nint value);

 public static bool TryReadInt128BigEndian(ReadOnlySpan<byte> source, out Int128 value);
 public static bool TryReadInt128LittleEndian(ReadOnlySpan<byte> source, out Int128 value);

 public static bool TryReadUIntPtrBigEndian(ReadOnlySpan<byte> source, out nuint value);
 public static bool TryReadUIntPtrLittleEndian(ReadOnlySpan<byte> source, out nuint value);

 public static bool TryReadUInt128BigEndian(ReadOnlySpan<byte> source, out UInt128 value);
 public static bool TryReadUInt128LittleEndian(ReadOnlySpan<byte> source, out UInt128 value);

 // TryWrite*BigEndian and TryWrite*LittleEndian

 public static bool TryWriteCharBigEndian(ReadOnlySpan<byte> destination, char value);
 public static bool TryWriteCharLittleEndian(ReadOnlySpan<byte> destination, char value);

 public static bool TryWriteIntPtrBigEndian(ReadOnlySpan<byte> destination, nint value);
 public static bool TryWriteIntPtrLittleEndian(ReadOnlySpan<byte> destination, nint value);

 public static bool TryWriteInt128BigEndian(ReadOnlySpan<byte> destination, Int128 value);
 public static bool TryWriteInt128LittleEndian(ReadOnlySpan<byte> destination, Int128 value);

 public static bool TryWriteUIntPtrBigEndian(ReadOnlySpan<byte> destination, nuint value);
 public static bool TryWriteUIntPtrLittleEndian(ReadOnlySpan<byte> destination, nuint value);

 public static bool TryWriteUInt128BigEndian(ReadOnlySpan<byte> destination, UInt128 value);
 public static bool TryWriteUInt128LittleEndian(ReadOnlySpan<byte> destination, UInt128 value);

 // Write*BigEndian and Write*LittleEndian

 public static void WriteCharBigEndian(ReadOnlySpan<byte> destination, char value);
 public static void WriteCharLittleEndian(ReadOnlySpan<byte> destination, char value);

 public static void WriteIntPtrBigEndian(ReadOnlySpan<byte> destination, nint value);
 public static void WriteIntPtrLittleEndian(ReadOnlySpan<byte> destination, nint value);

 public static void WriteInt128BigEndian(ReadOnlySpan<byte> destination, Int128 value);
 public static void WriteInt128LittleEndian(ReadOnlySpan<byte> destination, Int128 value);

 public static void WriteUIntPtrBigEndian(ReadOnlySpan<byte> destination, nuint value);
 public static void WriteUIntPtrLittleEndian(ReadOnlySpan<byte> destination, nuint value);

 public static void WriteUInt128BigEndian(ReadOnlySpan<byte> destination, UInt128 value);
 public static void WriteUInt128LittleEndian(ReadOnlySpan<byte> destination, UInt128 value);
}

Author:	tannergooding
Assignees:	-
Labels:	api-suggestion, area-System.Buffers
Milestone:	Future

[tannergooding]

CC. @dakersnar, @GrabYourPitchforks

Any concerns here or do you think this is fine to mark api-ready-for-review?

[tannergooding]

-- Noting that there is still the "downside" that BinaryPrimitives isn't very discoverable and its where most of the Endianness APIs are, except for IsLittleEndian which is on System.BitConverter.

These are "split" due to an old layering scenario around .NET Standard and ensuring the APIs were available "downlevel". It might be worth exposing an static bool IsLittleEndian { get; } (and static bool IsBigEndian { get; }) API on BinaryPrimitives for discoverability.

[dakersnar]

What does IsLittleEndian get called on? The type, the source data, or the machine's architecture?

[GrabYourPitchforks]

char isn't really a core numeric type, so it's weird to me to put endianness-related APIs over it. I'd much rather people cast it to ushort if they wanted to do anything that involves bit twiddling, then they can call the other existing APIs.

That said, it's only a weak preference. There's nothing wrong with it per se, more of a "if you need this functionality you're probably modeling your data structures using incorrect primitives" feeling.

[tannergooding]

char isn't really a core numeric type, so it's weird to me to put endianness-related APIs over it

It's a case that does have endianness considerations in that there is explicitly both UTF-16 LE and UTF-16 BE so this will be "useful" for manual lexing/parsing scenarios. Also notably, char->ushort is an implicit conversion but ushort->char is not, so ushort bits = BinaryPrimitives.ReverseEndianness('c'); works, but not char bits = BinaryPrimitives.ReverseEndianness('c');, exposing the API would "fix" that.

What does IsLittleEndian get called on? The type, the source data, or the machine's architecture?

It refers to the endianness of the host architecture. You use ReadInt32LittleEndian when you know the data is in little-endian format, the API will read an int32 and on big endian platforms will then ReverseEndianness before returning the result. If you are doing your own endianness fixups or handling, you would query IsLittleEndian directly. IsBigEndian is then a "readability" API since it is much clearer then !IsLittleEndian.

[Clockwork-Muse]

.... a reverse-endianed float/double isn't (likely) to be a valid value, so whether the output gets coerced to something or not is a bit of a mystery.
I could have sworn we just had an issue about it around here...?

[alexrp]

I could have sworn we just had an issue about it around here...?

I don't know about "just", but there was some discussion on the subject in this issue: #2365 (comment)

[tannergooding]

.... a reverse-endianed float/double isn't (likely) to be a valid value

Right, but we also have ReadSingleLittle/BigEndian and friends, so its a "missing" API. The alternative is requiring users to Read, Bitcast, Reverse, Bitcast (or Bitcast, Reverse, Bitcast, Write), which isn't very friendly considering the other APIs we're already providing (and that we can do a "better" thing).

[tannergooding]

Notably, the perf issue is basically a non-concern on modern hardware and is "normal" to do with more complex SIMD algorithms and floating-point manipulation.

I'm not hard set on adding them however, but they should be raised to API review alongside the other APIs since they are the only ones that aren't here.

[Clockwork-Muse]

Sure, but what I was getting at is the possibility the runtime or even the hardware might end up coercing them to a valid NaN value, which means you couldn't roundtrip with it, and if you'd been given the "wrong" one to begin with it likely wouldn't be a valid value you could reverse either, in such a scenario.

[jkotas]

Reverse-endianes on float/double was discussed in the original set and explicitly rejected for these reasons.