Simplify range validation in Memory.Span - #118585

Closed
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck
Closed

Simplify range validation in Memory.Span#118585
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck

Conversation

@xtqqczze

@xtqqczzextqqczze commented Aug 11, 2025

Copy link
Copy Markdown
Contributor
  • desiredStartIndex cannot be negative as high order bit of _index has been removed.
  • desiredLength cannot be negative as all public constructors validate _length is not negative.
  • lengthOfUnderlyingSpan cannot be negative as _object.Length as the length property of an array, string, or span cannot be negative.

@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Aug 11, 2025
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this + overflows the check is passed, isn't ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of thing where the logic is getting even more dense and hard to comprehend

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Otherwise, this needs a significant number of comments and asserts covering those invariants and why the various overflow considerations are "safe" before it could be accepted.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Related issue: #118587

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

We already assume the invariant _length >= 0 holds

publicMemory<T>Slice(intstart)
{
if((uint)start>(uint)_length)
{
ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.start);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Yes, which is necessary/important. The team has a firm stance that we can't rely on users doing safe threading and that we need to do the "right things".

It's the same general reason we have both sets of lengths checks when dealing with List<T> (CC. @GrabYourPitchforks)

We already assume the invariant _length >= 0 holds

It's still something where adding an assert is beneficial as it explicitly documents the expectation.

But, I'd generally rather we have the JIT doing these types of optimizations. The managed code should prefer being readable/understandable first. If something is truly perf critical, then making it less readable with added comments/asserts is ok. However, the JIT automatically recognizing the critical pattern and doing the right thing is even better, as then the code stays readable and other paths likely benefit as well.

@xtqqczze

Copy link
Copy Markdown
ContributorAuthor

@MihuBot

@MihaZupan

Copy link
Copy Markdown
Member

Changes here overlap with #115275

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Memory.cs
Comment on lines +332 to +333
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This at least needs a comment explaining why it's safe (that it can't overflow) and should likely do (uint)(desiredStartIndex + desiredLength) instead of (uint)desiredStartIndex + (uint)desiredLength

Additional numbers/info showing the wins would also be beneficial.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Unsigned overflow cannot occur here because of the following invariants:// desiredStartIndex <= int.MaxValue; as ReadOnlyMemory<T>.RemoveFlagsBitMask == int.MaxValue// desiredLength >= 0; as it is assigned from the _length field which is non-negative by construction// lengthOfUnderlyingSpan >= 0; as it is assigned from the object's Length property which has a non-negative invariant

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable. I might put // * desiredStartIndex and similarly for the other 2 listed invariants to help show its part of a list, but I think that fits the need here and covers future readers.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised from your comment in the other issue that _length >= 0 may not hold if the struct is torn. Since the condition in question is specifically checking for a torn structure that would be issue without changes to the Slice method.

Ah right, Span<T> is stack only (well non-heap, but anything off the managed heap or stack would have to be a Span<T>* and so unsafe) and the memory model makes it UB to read from/write to the stack of another thread.

Where-as you can have class C { Memory<T> _field; } where one thread reads (objA, index: 5, length: 20) and another thread writes (objB, index: 5, length: 5) in which case a race could exist in Slice(start: 10) such as we get (objB, index: 5, length: -5) because the compare (start > _length) thought it was length: 20 but then the constructed Memory<T> ends up with objB and a length: 5 so we do 5 - 10.

Originally posted by @tannergooding in #119708

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that'd be a risk and might mean this is something that can't be done.

You would need to show if the same thing could be violated via the existing logic and/or if the field was hoisted into a local.

@tannergooding

Copy link
Copy Markdown
Member

This is still pending resolution of the comments above and how it preserves the behavior described in the comment

// If the Memory or ReadOnlyMemory instance is torn, this property getter has undefined behavior.
// We try to detect this condition and throw an exception, but it's possible that a torn struct might
// appear to us to be valid, and we'll return an undesired span. Such a span is always guaranteed at
// least to be in-bounds when compared with the original Memory instance, so using the span won't
// AV the process.

If this was a valid transform, the Span<T>.Slice API would likely also be appropriate to update and for similar reasons.

@xtqqczze
xtqqczze marked this pull request as draft November 6, 2025 19:32
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jan 6, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Memorycommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@xtqqczze@MihaZupan@tannergooding@EgorBo@Clockwork-Muse
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Simplify range validation in Memory.Span - #118585

Closed
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck
Closed

Simplify range validation in Memory.Span#118585
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck

Conversation

@xtqqczze

@xtqqczzextqqczze commented Aug 11, 2025

Copy link
Copy Markdown
Contributor
  • desiredStartIndex cannot be negative as high order bit of _index has been removed.
  • desiredLength cannot be negative as all public constructors validate _length is not negative.
  • lengthOfUnderlyingSpan cannot be negative as _object.Length as the length property of an array, string, or span cannot be negative.

@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Aug 11, 2025
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this + overflows the check is passed, isn't ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of thing where the logic is getting even more dense and hard to comprehend

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Otherwise, this needs a significant number of comments and asserts covering those invariants and why the various overflow considerations are "safe" before it could be accepted.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Related issue: #118587

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

We already assume the invariant _length >= 0 holds

publicMemory<T>Slice(intstart)
{
if((uint)start>(uint)_length)
{
ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.start);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Yes, which is necessary/important. The team has a firm stance that we can't rely on users doing safe threading and that we need to do the "right things".

It's the same general reason we have both sets of lengths checks when dealing with List<T> (CC. @GrabYourPitchforks)

We already assume the invariant _length >= 0 holds

It's still something where adding an assert is beneficial as it explicitly documents the expectation.

But, I'd generally rather we have the JIT doing these types of optimizations. The managed code should prefer being readable/understandable first. If something is truly perf critical, then making it less readable with added comments/asserts is ok. However, the JIT automatically recognizing the critical pattern and doing the right thing is even better, as then the code stays readable and other paths likely benefit as well.

@xtqqczze

Copy link
Copy Markdown
ContributorAuthor

@MihuBot

@MihaZupan

Copy link
Copy Markdown
Member

Changes here overlap with #115275

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Memory.cs
Comment on lines +332 to +333
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This at least needs a comment explaining why it's safe (that it can't overflow) and should likely do (uint)(desiredStartIndex + desiredLength) instead of (uint)desiredStartIndex + (uint)desiredLength

Additional numbers/info showing the wins would also be beneficial.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Unsigned overflow cannot occur here because of the following invariants:// desiredStartIndex <= int.MaxValue; as ReadOnlyMemory<T>.RemoveFlagsBitMask == int.MaxValue// desiredLength >= 0; as it is assigned from the _length field which is non-negative by construction// lengthOfUnderlyingSpan >= 0; as it is assigned from the object's Length property which has a non-negative invariant

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable. I might put // * desiredStartIndex and similarly for the other 2 listed invariants to help show its part of a list, but I think that fits the need here and covers future readers.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised from your comment in the other issue that _length >= 0 may not hold if the struct is torn. Since the condition in question is specifically checking for a torn structure that would be issue without changes to the Slice method.

Ah right, Span<T> is stack only (well non-heap, but anything off the managed heap or stack would have to be a Span<T>* and so unsafe) and the memory model makes it UB to read from/write to the stack of another thread.

Where-as you can have class C { Memory<T> _field; } where one thread reads (objA, index: 5, length: 20) and another thread writes (objB, index: 5, length: 5) in which case a race could exist in Slice(start: 10) such as we get (objB, index: 5, length: -5) because the compare (start > _length) thought it was length: 20 but then the constructed Memory<T> ends up with objB and a length: 5 so we do 5 - 10.

Originally posted by @tannergooding in #119708

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that'd be a risk and might mean this is something that can't be done.

You would need to show if the same thing could be violated via the existing logic and/or if the field was hoisted into a local.

@tannergooding

Copy link
Copy Markdown
Member

This is still pending resolution of the comments above and how it preserves the behavior described in the comment

// If the Memory or ReadOnlyMemory instance is torn, this property getter has undefined behavior.
// We try to detect this condition and throw an exception, but it's possible that a torn struct might
// appear to us to be valid, and we'll return an undesired span. Such a span is always guaranteed at
// least to be in-bounds when compared with the original Memory instance, so using the span won't
// AV the process.

If this was a valid transform, the Span<T>.Slice API would likely also be appropriate to update and for similar reasons.

@xtqqczze
xtqqczze marked this pull request as draft November 6, 2025 19:32
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jan 6, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Memorycommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@xtqqczze@MihaZupan@tannergooding@EgorBo@Clockwork-Muse
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Simplify range validation in Memory.Span - #118585

Closed
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck
Closed

Simplify range validation in Memory.Span#118585
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck

Conversation

@xtqqczze

@xtqqczzextqqczze commented Aug 11, 2025

Copy link
Copy Markdown
Contributor
  • desiredStartIndex cannot be negative as high order bit of _index has been removed.
  • desiredLength cannot be negative as all public constructors validate _length is not negative.
  • lengthOfUnderlyingSpan cannot be negative as _object.Length as the length property of an array, string, or span cannot be negative.

@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Aug 11, 2025
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this + overflows the check is passed, isn't ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of thing where the logic is getting even more dense and hard to comprehend

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Otherwise, this needs a significant number of comments and asserts covering those invariants and why the various overflow considerations are "safe" before it could be accepted.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Related issue: #118587

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

We already assume the invariant _length >= 0 holds

publicMemory<T>Slice(intstart)
{
if((uint)start>(uint)_length)
{
ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.start);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Yes, which is necessary/important. The team has a firm stance that we can't rely on users doing safe threading and that we need to do the "right things".

It's the same general reason we have both sets of lengths checks when dealing with List<T> (CC. @GrabYourPitchforks)

We already assume the invariant _length >= 0 holds

It's still something where adding an assert is beneficial as it explicitly documents the expectation.

But, I'd generally rather we have the JIT doing these types of optimizations. The managed code should prefer being readable/understandable first. If something is truly perf critical, then making it less readable with added comments/asserts is ok. However, the JIT automatically recognizing the critical pattern and doing the right thing is even better, as then the code stays readable and other paths likely benefit as well.

@xtqqczze

Copy link
Copy Markdown
ContributorAuthor

@MihuBot

@MihaZupan

Copy link
Copy Markdown
Member

Changes here overlap with #115275

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Memory.cs
Comment on lines +332 to +333
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This at least needs a comment explaining why it's safe (that it can't overflow) and should likely do (uint)(desiredStartIndex + desiredLength) instead of (uint)desiredStartIndex + (uint)desiredLength

Additional numbers/info showing the wins would also be beneficial.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Unsigned overflow cannot occur here because of the following invariants:// desiredStartIndex <= int.MaxValue; as ReadOnlyMemory<T>.RemoveFlagsBitMask == int.MaxValue// desiredLength >= 0; as it is assigned from the _length field which is non-negative by construction// lengthOfUnderlyingSpan >= 0; as it is assigned from the object's Length property which has a non-negative invariant

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable. I might put // * desiredStartIndex and similarly for the other 2 listed invariants to help show its part of a list, but I think that fits the need here and covers future readers.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised from your comment in the other issue that _length >= 0 may not hold if the struct is torn. Since the condition in question is specifically checking for a torn structure that would be issue without changes to the Slice method.

Ah right, Span<T> is stack only (well non-heap, but anything off the managed heap or stack would have to be a Span<T>* and so unsafe) and the memory model makes it UB to read from/write to the stack of another thread.

Where-as you can have class C { Memory<T> _field; } where one thread reads (objA, index: 5, length: 20) and another thread writes (objB, index: 5, length: 5) in which case a race could exist in Slice(start: 10) such as we get (objB, index: 5, length: -5) because the compare (start > _length) thought it was length: 20 but then the constructed Memory<T> ends up with objB and a length: 5 so we do 5 - 10.

Originally posted by @tannergooding in #119708

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that'd be a risk and might mean this is something that can't be done.

You would need to show if the same thing could be violated via the existing logic and/or if the field was hoisted into a local.

@tannergooding

Copy link
Copy Markdown
Member

This is still pending resolution of the comments above and how it preserves the behavior described in the comment

// If the Memory or ReadOnlyMemory instance is torn, this property getter has undefined behavior.
// We try to detect this condition and throw an exception, but it's possible that a torn struct might
// appear to us to be valid, and we'll return an undesired span. Such a span is always guaranteed at
// least to be in-bounds when compared with the original Memory instance, so using the span won't
// AV the process.

If this was a valid transform, the Span<T>.Slice API would likely also be appropriate to update and for similar reasons.

@xtqqczze
xtqqczze marked this pull request as draft November 6, 2025 19:32
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jan 6, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Memorycommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@xtqqczze@MihaZupan@tannergooding@EgorBo@Clockwork-Muse
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Simplify range validation in Memory.Span - #118585

Closed
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck
Closed

Simplify range validation in Memory.Span#118585
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck

Conversation

@xtqqczze

@xtqqczzextqqczze commented Aug 11, 2025

Copy link
Copy Markdown
Contributor
  • desiredStartIndex cannot be negative as high order bit of _index has been removed.
  • desiredLength cannot be negative as all public constructors validate _length is not negative.
  • lengthOfUnderlyingSpan cannot be negative as _object.Length as the length property of an array, string, or span cannot be negative.

@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Aug 11, 2025
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this + overflows the check is passed, isn't ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of thing where the logic is getting even more dense and hard to comprehend

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Otherwise, this needs a significant number of comments and asserts covering those invariants and why the various overflow considerations are "safe" before it could be accepted.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Related issue: #118587

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

We already assume the invariant _length >= 0 holds

publicMemory<T>Slice(intstart)
{
if((uint)start>(uint)_length)
{
ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.start);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Yes, which is necessary/important. The team has a firm stance that we can't rely on users doing safe threading and that we need to do the "right things".

It's the same general reason we have both sets of lengths checks when dealing with List<T> (CC. @GrabYourPitchforks)

We already assume the invariant _length >= 0 holds

It's still something where adding an assert is beneficial as it explicitly documents the expectation.

But, I'd generally rather we have the JIT doing these types of optimizations. The managed code should prefer being readable/understandable first. If something is truly perf critical, then making it less readable with added comments/asserts is ok. However, the JIT automatically recognizing the critical pattern and doing the right thing is even better, as then the code stays readable and other paths likely benefit as well.

@xtqqczze

Copy link
Copy Markdown
ContributorAuthor

@MihuBot

@MihaZupan

Copy link
Copy Markdown
Member

Changes here overlap with #115275

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Memory.cs
Comment on lines +332 to +333
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This at least needs a comment explaining why it's safe (that it can't overflow) and should likely do (uint)(desiredStartIndex + desiredLength) instead of (uint)desiredStartIndex + (uint)desiredLength

Additional numbers/info showing the wins would also be beneficial.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Unsigned overflow cannot occur here because of the following invariants:// desiredStartIndex <= int.MaxValue; as ReadOnlyMemory<T>.RemoveFlagsBitMask == int.MaxValue// desiredLength >= 0; as it is assigned from the _length field which is non-negative by construction// lengthOfUnderlyingSpan >= 0; as it is assigned from the object's Length property which has a non-negative invariant

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable. I might put // * desiredStartIndex and similarly for the other 2 listed invariants to help show its part of a list, but I think that fits the need here and covers future readers.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised from your comment in the other issue that _length >= 0 may not hold if the struct is torn. Since the condition in question is specifically checking for a torn structure that would be issue without changes to the Slice method.

Ah right, Span<T> is stack only (well non-heap, but anything off the managed heap or stack would have to be a Span<T>* and so unsafe) and the memory model makes it UB to read from/write to the stack of another thread.

Where-as you can have class C { Memory<T> _field; } where one thread reads (objA, index: 5, length: 20) and another thread writes (objB, index: 5, length: 5) in which case a race could exist in Slice(start: 10) such as we get (objB, index: 5, length: -5) because the compare (start > _length) thought it was length: 20 but then the constructed Memory<T> ends up with objB and a length: 5 so we do 5 - 10.

Originally posted by @tannergooding in #119708

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that'd be a risk and might mean this is something that can't be done.

You would need to show if the same thing could be violated via the existing logic and/or if the field was hoisted into a local.

@tannergooding

Copy link
Copy Markdown
Member

This is still pending resolution of the comments above and how it preserves the behavior described in the comment

// If the Memory or ReadOnlyMemory instance is torn, this property getter has undefined behavior.
// We try to detect this condition and throw an exception, but it's possible that a torn struct might
// appear to us to be valid, and we'll return an undesired span. Such a span is always guaranteed at
// least to be in-bounds when compared with the original Memory instance, so using the span won't
// AV the process.

If this was a valid transform, the Span<T>.Slice API would likely also be appropriate to update and for similar reasons.

@xtqqczze
xtqqczze marked this pull request as draft November 6, 2025 19:32
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jan 6, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Memorycommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@xtqqczze@MihaZupan@tannergooding@EgorBo@Clockwork-Muse
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Simplify range validation in Memory.Span - #118585

Closed
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck
Closed

Simplify range validation in Memory.Span#118585
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck

Conversation

@xtqqczze

@xtqqczzextqqczze commented Aug 11, 2025

Copy link
Copy Markdown
Contributor
  • desiredStartIndex cannot be negative as high order bit of _index has been removed.
  • desiredLength cannot be negative as all public constructors validate _length is not negative.
  • lengthOfUnderlyingSpan cannot be negative as _object.Length as the length property of an array, string, or span cannot be negative.

@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Aug 11, 2025
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this + overflows the check is passed, isn't ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of thing where the logic is getting even more dense and hard to comprehend

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Otherwise, this needs a significant number of comments and asserts covering those invariants and why the various overflow considerations are "safe" before it could be accepted.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Related issue: #118587

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

We already assume the invariant _length >= 0 holds

publicMemory<T>Slice(intstart)
{
if((uint)start>(uint)_length)
{
ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.start);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Yes, which is necessary/important. The team has a firm stance that we can't rely on users doing safe threading and that we need to do the "right things".

It's the same general reason we have both sets of lengths checks when dealing with List<T> (CC. @GrabYourPitchforks)

We already assume the invariant _length >= 0 holds

It's still something where adding an assert is beneficial as it explicitly documents the expectation.

But, I'd generally rather we have the JIT doing these types of optimizations. The managed code should prefer being readable/understandable first. If something is truly perf critical, then making it less readable with added comments/asserts is ok. However, the JIT automatically recognizing the critical pattern and doing the right thing is even better, as then the code stays readable and other paths likely benefit as well.

@xtqqczze

Copy link
Copy Markdown
ContributorAuthor

@MihuBot

@MihaZupan

Copy link
Copy Markdown
Member

Changes here overlap with #115275

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Memory.cs
Comment on lines +332 to +333
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This at least needs a comment explaining why it's safe (that it can't overflow) and should likely do (uint)(desiredStartIndex + desiredLength) instead of (uint)desiredStartIndex + (uint)desiredLength

Additional numbers/info showing the wins would also be beneficial.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Unsigned overflow cannot occur here because of the following invariants:// desiredStartIndex <= int.MaxValue; as ReadOnlyMemory<T>.RemoveFlagsBitMask == int.MaxValue// desiredLength >= 0; as it is assigned from the _length field which is non-negative by construction// lengthOfUnderlyingSpan >= 0; as it is assigned from the object's Length property which has a non-negative invariant

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable. I might put // * desiredStartIndex and similarly for the other 2 listed invariants to help show its part of a list, but I think that fits the need here and covers future readers.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised from your comment in the other issue that _length >= 0 may not hold if the struct is torn. Since the condition in question is specifically checking for a torn structure that would be issue without changes to the Slice method.

Ah right, Span<T> is stack only (well non-heap, but anything off the managed heap or stack would have to be a Span<T>* and so unsafe) and the memory model makes it UB to read from/write to the stack of another thread.

Where-as you can have class C { Memory<T> _field; } where one thread reads (objA, index: 5, length: 20) and another thread writes (objB, index: 5, length: 5) in which case a race could exist in Slice(start: 10) such as we get (objB, index: 5, length: -5) because the compare (start > _length) thought it was length: 20 but then the constructed Memory<T> ends up with objB and a length: 5 so we do 5 - 10.

Originally posted by @tannergooding in #119708

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that'd be a risk and might mean this is something that can't be done.

You would need to show if the same thing could be violated via the existing logic and/or if the field was hoisted into a local.

@tannergooding

Copy link
Copy Markdown
Member

This is still pending resolution of the comments above and how it preserves the behavior described in the comment

// If the Memory or ReadOnlyMemory instance is torn, this property getter has undefined behavior.
// We try to detect this condition and throw an exception, but it's possible that a torn struct might
// appear to us to be valid, and we'll return an undesired span. Such a span is always guaranteed at
// least to be in-bounds when compared with the original Memory instance, so using the span won't
// AV the process.

If this was a valid transform, the Span<T>.Slice API would likely also be appropriate to update and for similar reasons.

@xtqqczze
xtqqczze marked this pull request as draft November 6, 2025 19:32
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jan 6, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Memorycommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@xtqqczze@MihaZupan@tannergooding@EgorBo@Clockwork-Muse
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Simplify range validation in Memory.Span - #118585

Closed
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck
Closed

Simplify range validation in Memory.Span#118585
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck

Conversation

@xtqqczze

@xtqqczzextqqczze commented Aug 11, 2025

Copy link
Copy Markdown
Contributor
  • desiredStartIndex cannot be negative as high order bit of _index has been removed.
  • desiredLength cannot be negative as all public constructors validate _length is not negative.
  • lengthOfUnderlyingSpan cannot be negative as _object.Length as the length property of an array, string, or span cannot be negative.

@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Aug 11, 2025
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this + overflows the check is passed, isn't ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of thing where the logic is getting even more dense and hard to comprehend

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Otherwise, this needs a significant number of comments and asserts covering those invariants and why the various overflow considerations are "safe" before it could be accepted.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Related issue: #118587

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

We already assume the invariant _length >= 0 holds

publicMemory<T>Slice(intstart)
{
if((uint)start>(uint)_length)
{
ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.start);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Yes, which is necessary/important. The team has a firm stance that we can't rely on users doing safe threading and that we need to do the "right things".

It's the same general reason we have both sets of lengths checks when dealing with List<T> (CC. @GrabYourPitchforks)

We already assume the invariant _length >= 0 holds

It's still something where adding an assert is beneficial as it explicitly documents the expectation.

But, I'd generally rather we have the JIT doing these types of optimizations. The managed code should prefer being readable/understandable first. If something is truly perf critical, then making it less readable with added comments/asserts is ok. However, the JIT automatically recognizing the critical pattern and doing the right thing is even better, as then the code stays readable and other paths likely benefit as well.

@xtqqczze

Copy link
Copy Markdown
ContributorAuthor

@MihuBot

@MihaZupan

Copy link
Copy Markdown
Member

Changes here overlap with #115275

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Memory.cs
Comment on lines +332 to +333
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This at least needs a comment explaining why it's safe (that it can't overflow) and should likely do (uint)(desiredStartIndex + desiredLength) instead of (uint)desiredStartIndex + (uint)desiredLength

Additional numbers/info showing the wins would also be beneficial.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Unsigned overflow cannot occur here because of the following invariants:// desiredStartIndex <= int.MaxValue; as ReadOnlyMemory<T>.RemoveFlagsBitMask == int.MaxValue// desiredLength >= 0; as it is assigned from the _length field which is non-negative by construction// lengthOfUnderlyingSpan >= 0; as it is assigned from the object's Length property which has a non-negative invariant

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable. I might put // * desiredStartIndex and similarly for the other 2 listed invariants to help show its part of a list, but I think that fits the need here and covers future readers.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised from your comment in the other issue that _length >= 0 may not hold if the struct is torn. Since the condition in question is specifically checking for a torn structure that would be issue without changes to the Slice method.

Ah right, Span<T> is stack only (well non-heap, but anything off the managed heap or stack would have to be a Span<T>* and so unsafe) and the memory model makes it UB to read from/write to the stack of another thread.

Where-as you can have class C { Memory<T> _field; } where one thread reads (objA, index: 5, length: 20) and another thread writes (objB, index: 5, length: 5) in which case a race could exist in Slice(start: 10) such as we get (objB, index: 5, length: -5) because the compare (start > _length) thought it was length: 20 but then the constructed Memory<T> ends up with objB and a length: 5 so we do 5 - 10.

Originally posted by @tannergooding in #119708

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that'd be a risk and might mean this is something that can't be done.

You would need to show if the same thing could be violated via the existing logic and/or if the field was hoisted into a local.

@tannergooding

Copy link
Copy Markdown
Member

This is still pending resolution of the comments above and how it preserves the behavior described in the comment

// If the Memory or ReadOnlyMemory instance is torn, this property getter has undefined behavior.
// We try to detect this condition and throw an exception, but it's possible that a torn struct might
// appear to us to be valid, and we'll return an undesired span. Such a span is always guaranteed at
// least to be in-bounds when compared with the original Memory instance, so using the span won't
// AV the process.

If this was a valid transform, the Span<T>.Slice API would likely also be appropriate to update and for similar reasons.

@xtqqczze
xtqqczze marked this pull request as draft November 6, 2025 19:32
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jan 6, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Memorycommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@xtqqczze@MihaZupan@tannergooding@EgorBo@Clockwork-Muse
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Simplify range validation in Memory.Span - #118585

Closed
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck
Closed

Simplify range validation in Memory.Span#118585
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck

Conversation

@xtqqczze

@xtqqczzextqqczze commented Aug 11, 2025

Copy link
Copy Markdown
Contributor
  • desiredStartIndex cannot be negative as high order bit of _index has been removed.
  • desiredLength cannot be negative as all public constructors validate _length is not negative.
  • lengthOfUnderlyingSpan cannot be negative as _object.Length as the length property of an array, string, or span cannot be negative.

@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Aug 11, 2025
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this + overflows the check is passed, isn't ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of thing where the logic is getting even more dense and hard to comprehend

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Otherwise, this needs a significant number of comments and asserts covering those invariants and why the various overflow considerations are "safe" before it could be accepted.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Related issue: #118587

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

We already assume the invariant _length >= 0 holds

publicMemory<T>Slice(intstart)
{
if((uint)start>(uint)_length)
{
ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.start);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Yes, which is necessary/important. The team has a firm stance that we can't rely on users doing safe threading and that we need to do the "right things".

It's the same general reason we have both sets of lengths checks when dealing with List<T> (CC. @GrabYourPitchforks)

We already assume the invariant _length >= 0 holds

It's still something where adding an assert is beneficial as it explicitly documents the expectation.

But, I'd generally rather we have the JIT doing these types of optimizations. The managed code should prefer being readable/understandable first. If something is truly perf critical, then making it less readable with added comments/asserts is ok. However, the JIT automatically recognizing the critical pattern and doing the right thing is even better, as then the code stays readable and other paths likely benefit as well.

@xtqqczze

Copy link
Copy Markdown
ContributorAuthor

@MihuBot

@MihaZupan

Copy link
Copy Markdown
Member

Changes here overlap with #115275

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Memory.cs
Comment on lines +332 to +333
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This at least needs a comment explaining why it's safe (that it can't overflow) and should likely do (uint)(desiredStartIndex + desiredLength) instead of (uint)desiredStartIndex + (uint)desiredLength

Additional numbers/info showing the wins would also be beneficial.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Unsigned overflow cannot occur here because of the following invariants:// desiredStartIndex <= int.MaxValue; as ReadOnlyMemory<T>.RemoveFlagsBitMask == int.MaxValue// desiredLength >= 0; as it is assigned from the _length field which is non-negative by construction// lengthOfUnderlyingSpan >= 0; as it is assigned from the object's Length property which has a non-negative invariant

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable. I might put // * desiredStartIndex and similarly for the other 2 listed invariants to help show its part of a list, but I think that fits the need here and covers future readers.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised from your comment in the other issue that _length >= 0 may not hold if the struct is torn. Since the condition in question is specifically checking for a torn structure that would be issue without changes to the Slice method.

Ah right, Span<T> is stack only (well non-heap, but anything off the managed heap or stack would have to be a Span<T>* and so unsafe) and the memory model makes it UB to read from/write to the stack of another thread.

Where-as you can have class C { Memory<T> _field; } where one thread reads (objA, index: 5, length: 20) and another thread writes (objB, index: 5, length: 5) in which case a race could exist in Slice(start: 10) such as we get (objB, index: 5, length: -5) because the compare (start > _length) thought it was length: 20 but then the constructed Memory<T> ends up with objB and a length: 5 so we do 5 - 10.

Originally posted by @tannergooding in #119708

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that'd be a risk and might mean this is something that can't be done.

You would need to show if the same thing could be violated via the existing logic and/or if the field was hoisted into a local.

@tannergooding

Copy link
Copy Markdown
Member

This is still pending resolution of the comments above and how it preserves the behavior described in the comment

// If the Memory or ReadOnlyMemory instance is torn, this property getter has undefined behavior.
// We try to detect this condition and throw an exception, but it's possible that a torn struct might
// appear to us to be valid, and we'll return an undesired span. Such a span is always guaranteed at
// least to be in-bounds when compared with the original Memory instance, so using the span won't
// AV the process.

If this was a valid transform, the Span<T>.Slice API would likely also be appropriate to update and for similar reasons.

@xtqqczze
xtqqczze marked this pull request as draft November 6, 2025 19:32
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jan 6, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Memorycommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@xtqqczze@MihaZupan@tannergooding@EgorBo@Clockwork-Muse
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Simplify range validation in Memory.Span - #118585

Closed
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck
Closed

Simplify range validation in Memory.Span#118585
xtqqczze wants to merge 1 commit into
dotnet:mainfrom
xtqqczze:simplify-memorygetspan-rangecheck

Conversation

@xtqqczze

@xtqqczzextqqczze commented Aug 11, 2025

Copy link
Copy Markdown
Contributor
  • desiredStartIndex cannot be negative as high order bit of _index has been removed.
  • desiredLength cannot be negative as all public constructors validate _length is not negative.
  • lengthOfUnderlyingSpan cannot be negative as _object.Length as the length property of an array, string, or span cannot be negative.

@dotnet-policy-servicedotnet-policy-serviceBot added the community-contribution Indicates that the PR has been added by a community member label Aug 11, 2025
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this + overflows the check is passed, isn't ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the kind of thing where the logic is getting even more dense and hard to comprehend

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Otherwise, this needs a significant number of comments and asserts covering those invariants and why the various overflow considerations are "safe" before it could be accepted.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would much rather we do such transforms in the JIT when we know that a given invariant is held, so that the high level managed code can remain easy to understand.

Related issue: #118587

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

desiredLength should never be negative, this could do with an assert

We already assume the invariant _length >= 0 holds

publicMemory<T>Slice(intstart)
{
if((uint)start>(uint)_length)
{
ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.start);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here is only to prevent undefined behaviour if the struct is torn, if it wasn't for this it could be removed completely.

Yes, which is necessary/important. The team has a firm stance that we can't rely on users doing safe threading and that we need to do the "right things".

It's the same general reason we have both sets of lengths checks when dealing with List<T> (CC. @GrabYourPitchforks)

We already assume the invariant _length >= 0 holds

It's still something where adding an assert is beneficial as it explicitly documents the expectation.

But, I'd generally rather we have the JIT doing these types of optimizations. The managed code should prefer being readable/understandable first. If something is truly perf critical, then making it less readable with added comments/asserts is ok. However, the JIT automatically recognizing the critical pattern and doing the right thing is even better, as then the code stays readable and other paths likely benefit as well.

@xtqqczze

Copy link
Copy Markdown
ContributorAuthor

@MihuBot

@MihaZupan

Copy link
Copy Markdown
Member

Changes here overlap with #115275

#else
if ((uint)desiredStartIndex > (uint)lengthOfUnderlyingSpan || (uint)desiredLength > (uint)lengthOfUnderlyingSpan - (uint)desiredStartIndex)
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - it's why the 32-bit target check is written the way it was below (which is a common overflow-avoiding pattern).

We can't "simplify" this, though - the 64 and 32 bit targets are different for a reason; they're taking advantage of different bit widths (or not able to) on different platforms.

Comment threadsrc/libraries/System.Private.CoreLib/src/System/Memory.cs
Comment on lines +332 to +333
Debug.Assert((int)desiredStartIndex >= 0 && lengthOfUnderlyingSpan >= 0);
if ((uint)desiredStartIndex + (uint)desiredLength > (uint)lengthOfUnderlyingSpan)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This at least needs a comment explaining why it's safe (that it can't overflow) and should likely do (uint)(desiredStartIndex + desiredLength) instead of (uint)desiredStartIndex + (uint)desiredLength

Additional numbers/info showing the wins would also be beneficial.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Unsigned overflow cannot occur here because of the following invariants:// desiredStartIndex <= int.MaxValue; as ReadOnlyMemory<T>.RemoveFlagsBitMask == int.MaxValue// desiredLength >= 0; as it is assigned from the _length field which is non-negative by construction// lengthOfUnderlyingSpan >= 0; as it is assigned from the object's Length property which has a non-negative invariant

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable. I might put // * desiredStartIndex and similarly for the other 2 listed invariants to help show its part of a list, but I think that fits the need here and covers future readers.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised from your comment in the other issue that _length >= 0 may not hold if the struct is torn. Since the condition in question is specifically checking for a torn structure that would be issue without changes to the Slice method.

Ah right, Span<T> is stack only (well non-heap, but anything off the managed heap or stack would have to be a Span<T>* and so unsafe) and the memory model makes it UB to read from/write to the stack of another thread.

Where-as you can have class C { Memory<T> _field; } where one thread reads (objA, index: 5, length: 20) and another thread writes (objB, index: 5, length: 5) in which case a race could exist in Slice(start: 10) such as we get (objB, index: 5, length: -5) because the compare (start > _length) thought it was length: 20 but then the constructed Memory<T> ends up with objB and a length: 5 so we do 5 - 10.

Originally posted by @tannergooding in #119708

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that'd be a risk and might mean this is something that can't be done.

You would need to show if the same thing could be violated via the existing logic and/or if the field was hoisted into a local.

@tannergooding

Copy link
Copy Markdown
Member

This is still pending resolution of the comments above and how it preserves the behavior described in the comment

// If the Memory or ReadOnlyMemory instance is torn, this property getter has undefined behavior.
// We try to detect this condition and throw an exception, but it's possible that a torn struct might
// appear to us to be valid, and we'll return an undesired span. Such a span is always guaranteed at
// least to be in-bounds when compared with the original Memory instance, so using the span won't
// AV the process.

If this was a valid transform, the Span<T>.Slice API would likely also be appropriate to update and for similar reasons.

@xtqqczze
xtqqczze marked this pull request as draft November 6, 2025 19:32
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jan 6, 2026
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Memorycommunity-contributionIndicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@xtqqczze@MihaZupan@tannergooding@EgorBo@Clockwork-Muse