Allow ref struct to contain ref fields

[Sergio0694]

The current plan is to add ref fields as first class construct to the type system. See proposal at dotnet/csharplang#3936 for more details.

---

Original proposal

Overview

Make the System.ByReference<T> type (here) public instead of internal.

Rationale

It's already possible to achieve the same behavior of ByReference<T>, namely to store a ref T to some variable as a field in a ref struct, by leveraging the Span<T> and MemoryMarshal types, like so:

public readonly ref struct FakeByReference<T>
{
 private readonly Span<T> span;

 public ByReference(ref T value)
 {
 span = MemoryMarshal.CreateSpan(ref value, 1);
 }

 public ref T Value => ref MemoryMarshal.GetReference(span);
}

This can either be used as is, as a field in some other ref struct type, or the same technique (storing a 1-length Span<T> in place of the missing ByReference<T> type) can be used in other ref struct types to reproduce this same behavior: storing a managed reference as a field in a ref struct type.

Given that this is already possible with existing APIs, as shown above, but it's less performant than the original ByReference<T>, more verbose and more error prone, why not just make the original ByReference<T> type public so everyone can use it with no need to reinvent the wheel?

It'd be a neat API to have, and at basically no cost, since it's already in the .NET Core BCL anyway. 😄

[jkotas]

We would prefer this to be exposed as ref fields in C# language: dotnet/csharplang#1147 . This would be even faster, leaner, and less error prone than ByReference<T>.

[Sergio0694]

Hi @jkotas - thanks for linking that issue, I had missed that! 😊

Is there an ETA for that though? By the looks of it, and the fact it has been opened for over two years already, it looks like it still has a way to go, plus all the additional difficulties caused by the fact that that would require at least some Roslyn updates to support that, if not some CLR changes too.

My rationale here was that making the ByReference<T> type public wouldn't allow new scenarios not achievable today, but simply make them more convenient for devs that were interested in using them (with the assumption that those devs would already be aware of all the various implications of using unsafe APIs in that case). Even if a ref T field feature was to be implemented one day, wouldn't this still be a nice and easy to implement stopgap for the time being? Again, given that the same exact feature can still be achieved today anyway, just less efficiently and with more room for mistakes (and the additional verbosity of having to reimplement that fake ByReference<T> type).

[jkotas]

We are here for the long haul and want to do things properly. We try hard to avoid exposing temporary workarounds that we are stuck forever even once the more appropriate solution is available.

[Sergio0694]

Sure thing, that's fair enough 😄
Also, just to be clear, it wasn't my intention to suggest this change as a quick and sloppy solution instead of doing things properly, as I said my rationale was more of an "iterative" improvement for a scenario that's technically supported already, but with an even worse implementation. But I can see how it'd be preferable to jump straight to the optimal and built-in solution, absolutely 🚀

Is there any approximate ETA on that proposal? I see the issue has been open for over two years already and it's not currently assigned to any milestone, not even X.0 or X.X. Is that a feature that's still being evaluated and not yet decided on, or is it something the various teams have already decided to implement, just not with an exact timeframe for it?

Thanks again for your time! 😊

[jkotas]

It did not make it into the list of C# features we are looking at for .NET 5.

[DaZombieKiller]

Wouldn't exposing this type make it possible to create ref fields in languages other than C#?
To me that sounds like something worth considering.

[jkotas]

This type as it exists today is not safe construct. It is very easy to create dangling pointers with it, without guardrails in the language. If we were to expose it, we would likely mark it with the special ObsoleteAttribute to block its use in languages that are not aware of it (e.g. similar to how ref structs are blocked).

[GrabYourPitchforks]

On a related note, Jan, you and I previously talked about a syntax that would allow operating with gcrefs as naturally as operating with pointers. Things that would allow natural syntax like the + operator without bouncing through Unsafe.Add. Would something like that ever make sense to add as a language feature, or as a ByRef-like type, or as something else?

[Sergio0694]

Hey @jkotas - I have a follow up question on this, specifically regarding the general criteria being used to expose APIs or add explicit support for "unsafe" operations. You mentioned how with the ByReference<T> type is't possible to create dangling pointers, which is certainly true. My question though is about the seemingly arbitrary decision not to make the type public, despite the BCL having a ton of potentially dangerous APIs already. Consider this snippet:

public static Span<T> GetFaultySpan<T>()
{
 T value = default;
 return MemoryMarshal.CreateSpan(ref value, 1);
}

This compiles fine, but results in a dangling pointer just like the one you mentioned. It's not perfectly clear to me why is this API public, but ByReference<T> is not. Don't get me wrong, this API is super useful (and I love the fact that it's available!), but I don't get why the same rationale doesn't allow to ByReference<T> as well: "here's a potentially dangerous but useful tool, use at your own risk!".

Or, just like the BCL already includes dedicated code to properly handle devs doing stuff that's technically not valid (eg. Memory<T> wrapping strings as mutable), then why not just use the same rationale here as well, and provide a tool that enables devs to do exactly the same thing they can already do with MemoryMarshal.CreateSpan, but just in a more efficient and less error prone way?

I just can't stress enough how useful the ByReference<T> type would be, and with the native support for ref T fields not making the cut for .NET 5 I'd say it'll be at least another year (if not more) until that eventually makes it to a public release? As you said, you could also mark it as [Obsolete] just like Span<T>, to avoid issues with other .NET languages.

Just though I'd share my thoughts on this, thank you in advance for your time! 😊

P.S. Just so I'm perfectly clear, I'm not criticizing your work in any way, the .NET team is doing an incredible job and I really love where the whole platform is going! Just trying to make my case here as to why I think making ByReference<T> public could be a win-win for everyone. 😄

[tannergooding]

@jaredpar had some reasons on why this wasn't safe from the language perspective and would be extremely likely to cause bugs without proper language support for tracking the lifetime.

[jkotas]

This compiles fine, but results in a dangling pointer just like the one you mentioned

Methods in System.Runtime.InteropServices are unsafe. They can produce dangling pointers, corrupt memory, etc. You have to know what you are doing when using them and they should be avoided where possible.

extremely likely to cause bugs without proper language support for tracking the lifetime.

Correct. We would like ref fields to have proper language that guarantees its safety.

[Sergio0694]

Ah, yeah that's a fair point, having ByReference<T> under System.* would definitely be a problem - I hadn't noticed that before and I had just assumed it was under System.Runtime.* somewhere like Unsafe or MemoryMarshal. I guess it could very well be moved there since right now it's internal, but I can see how that would require more work/refactoring than just making it public 👍

And yeah I agree that just having language-level support for ref fields would be even better, absolutely no dispute there. I was just hoping for a better workaround for the time being, seeing as that feature at least for now seems to be pretty far from completion, is all.

Thank you both again for your time, I really appreciate the conversation here!