Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. - #103244

Merged
michaelgsharp merged 7 commits into
dotnet:mainfrom
michaelgsharp:tensor-struct
Jun 20, 2024
Merged

Changed TensorSpan and ReadOnlyTensorSpan layout for better performance.#103244
michaelgsharp merged 7 commits into
dotnet:mainfrom
michaelgsharp:tensor-struct

Conversation

@michaelgsharp

@michaelgsharpmichaelgsharp commented Jun 10, 2024

Copy link
Copy Markdown
Contributor

Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. This included making a new span to hold all the shared data between them and allowing for allocation free behavior for dimensions less than 6.

@ghost

ghost commented Jun 10, 2024

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

1 similar comment
@ghost

ghost commented Jun 10, 2024

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@dotnet-policy-service

dotnet-policy-serviceBot commented Jun 10, 2024

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors
See info in area-owners.md if you want to be subscribed.

This was referenced Jun 10, 2024
Closed
Closed
Closed
@michaelgsharpmichaelgsharp mentioned this pull request Jun 12, 2024
Closed
@ericstj

ericstj commented Jun 19, 2024

Copy link
Copy Markdown
Member

@tannergooding did you have any more feedback on this one?

eiriktsarpalis
eiriktsarpalis reviewed Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs

namespace System.Numerics.Tensors
{
internal readonly struct TensorShape

@eiriktsarpaliseiriktsarpalisJun 19, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be a ref struct?

@tannergoodingtannergoodingJun 19, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to be able to use the same struct as part of Tensor<T> so we can avoid additional allocations for common cases there as well.

eiriktsarpalis
eiriktsarpalis reviewed Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs
{
internal readonly nint[]? _metadata; // 8 bytes

internal readonly nint _memoryLength; // 8 bytes

@eiriktsarpaliseiriktsarpalisJun 19, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the layout and size concerns of this change, does the new type require an explicit StructLayoutAttribute annotation? Are we ok with using the default layout? Is there an expectation that the fields will be consumed by unmanaged code.

@tannergoodingtannergoodingJun 19, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).

@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).

Does Mono behave the same? I think it might respect Sequential for managed types.

@tannergoodingtannergoodingJun 19, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.

That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).

This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.

@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.

That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).

This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.

Yeah but I've meant that Auto could maybe still be beneficial for Mono.

@tannergoodingtannergoodingJun 19, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono should likely consider adjusting their layout algorithm to match the RyuJIT algorithm in that scenario.

There are thousands of structs across the BCL and we are not going to go and annotate every single one of them to be explicitly Auto because they contain some managed field.

tannergooding
tannergooding reviewed Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan.cs

_flattenedLength = linearLength;
_memoryLength = array.Length;
_shape = new TensorShape(array.Length - start, lengths, strides);

@tannergoodingtannergoodingJun 19, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if some of the validation logic could be in a helper on TensorShape

-- Not needed to fix in this PR, just a possible simplification opportunity

tannergooding
tannergooding approved these changes Jun 19, 2024
View reviewed changes

@tannergoodingtannergooding left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave some nits, but nothing that should block this being merged

This was referenced Jun 20, 2024
Open
Closed
Closed
Closed
Closed
@michaelgsharp

michaelgsharp commented Jun 20, 2024

Copy link
Copy Markdown
ContributorAuthor

/ba-g suppressing unrelated test failure to merge ahead of snap.

@michaelgsharp
michaelgsharp merged commit 4cfc93b into dotnet:mainJun 20, 2024
rzikm pushed a commit to rzikm/dotnet-runtime that referenced this pull request Jun 24, 2024
@michaelgsharp@rzikm
Changed TensorSpan and ReadOnlyTensorSpan layout for better performan…
50458c6
…ce. (dotnet#103244)
* TensorShape
* fixed TensorShape issue
* removed extra inline buffer type. Fixed tests. Added large dimension testing
* removed typo
* adding unsaved files
* changes to use const for stack alloc comparison
* changes from pr comments
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Jul 21, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@stephentoubstephentoubstephentoub left review comments
@eiriktsarpaliseiriktsarpaliseiriktsarpalis left review comments
@tannergoodingtannergoodingtannergooding approved these changes
@ericstjericstjAwaiting requested review from ericstj
+1 more reviewer
@MichalPetrykaMichalPetrykaMichalPetryka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area-System.Numerics.Tensors

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@michaelgsharp@ericstj@stephentoub@eiriktsarpalis@tannergooding@MichalPetryka@teo-tsirpanis
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. - #103244
Merged
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Jun 20, 2024
Merged
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance.#103244
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. This included making a new span to hold all the shared data between them and allowing for allocation free behavior for dimensions less than 6.
@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.


1 similar comment

@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@dotnet-policy-service
dotnet-policy-serviceBot
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.

This was referenced Jun 10, 2024

Closed


Closed


Closed

@michaelgsharpmichaelgsharp
mentioned this pull request
Jun 12, 2024

Closed

@ericstj
ericstj
commented
Jun 19, 2024

Copy link


Copy Markdown

Member
@tannergooding did you have any more feedback on this one?
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs

namespace System.Numerics.Tensors
{
 internal readonly struct TensorShape
@eiriktsarpaliseiriktsarpalisJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be a ref struct?
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to be able to use the same struct as part of Tensor<T> so we can avoid additional allocations for common cases there as well.
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs
 {
 internal readonly nint[]? _metadata; // 8 bytes

 internal readonly nint _memoryLength; // 8 bytes
@eiriktsarpaliseiriktsarpalisJun 19, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the layout and size concerns of this change, does the new type require an explicit StructLayoutAttribute annotation? Are we ok with using the default layout? Is there an expectation that the fields will be consumed by unmanaged code.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
Does Mono behave the same? I think it might respect Sequential for managed types.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
Yeah but I've meant that Auto could maybe still be beneficial for Mono.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono should likely consider adjusting their layout algorithm to match the RyuJIT algorithm in that scenario.
There are thousands of structs across the BCL and we are not going to go and annotate every single one of them to be explicitly Auto because they contain some managed field.
tannergooding
tannergooding
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan.cs

 _flattenedLength = linearLength;
 _memoryLength = array.Length;
 _shape = new TensorShape(array.Length - start, lengths, strides);
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if some of the validation logic could be in a helper on TensorShape
-- Not needed to fix in this PR, just a possible simplification opportunity
tannergooding
tannergooding
approved these changes
Jun 19, 2024
View reviewed changes
@tannergoodingtannergooding
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave some nits, but nothing that should block this being merged

This was referenced Jun 20, 2024

Open


Closed


Closed


Closed


Closed

@michaelgsharp
michaelgsharp
commented
Jun 20, 2024

Copy link


Copy Markdown

ContributorAuthor
/ba-g suppressing unrelated test failure to merge ahead of snap.
@michaelgsharp
michaelgsharp
merged commit 4cfc93b
into
dotnet:mainJun 20, 2024
rzikm pushed a commit
to rzikm/dotnet-runtime
that referenced
this pull request
Jun 24, 2024
@michaelgsharp@rzikm
Changed TensorSpan and ReadOnlyTensorSpan layout for better performan…
50458c6
…ce. (dotnet#103244)
* TensorShape
* fixed TensorShape issue
* removed extra inline buffer type. Fixed tests. Added large dimension testing
* removed typo
* adding unsaved files
* changes to use const for stack alloc comparison
* changes from pr comments
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Jul 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@stephentoubstephentoubstephentoub left review comments
@eiriktsarpaliseiriktsarpaliseiriktsarpalis left review comments
@tannergoodingtannergoodingtannergooding approved these changes
@ericstjericstjAwaiting requested review from ericstj

+1 more reviewer

@MichalPetrykaMichalPetrykaMichalPetryka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area-System.Numerics.Tensors

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@michaelgsharp@ericstj@stephentoub@eiriktsarpalis@tannergooding@MichalPetryka@teo-tsirpanis

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. - #103244
Merged
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Jun 20, 2024
Merged
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance.#103244
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. This included making a new span to hold all the shared data between them and allowing for allocation free behavior for dimensions less than 6.
@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.


1 similar comment

@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@dotnet-policy-service
dotnet-policy-serviceBot
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.

This was referenced Jun 10, 2024

Closed


Closed


Closed

@michaelgsharpmichaelgsharp
mentioned this pull request
Jun 12, 2024

Closed

@ericstj
ericstj
commented
Jun 19, 2024

Copy link


Copy Markdown

Member
@tannergooding did you have any more feedback on this one?
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs

namespace System.Numerics.Tensors
{
 internal readonly struct TensorShape
@eiriktsarpaliseiriktsarpalisJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be a ref struct?
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to be able to use the same struct as part of Tensor<T> so we can avoid additional allocations for common cases there as well.
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs
 {
 internal readonly nint[]? _metadata; // 8 bytes

 internal readonly nint _memoryLength; // 8 bytes
@eiriktsarpaliseiriktsarpalisJun 19, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the layout and size concerns of this change, does the new type require an explicit StructLayoutAttribute annotation? Are we ok with using the default layout? Is there an expectation that the fields will be consumed by unmanaged code.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
Does Mono behave the same? I think it might respect Sequential for managed types.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
Yeah but I've meant that Auto could maybe still be beneficial for Mono.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono should likely consider adjusting their layout algorithm to match the RyuJIT algorithm in that scenario.
There are thousands of structs across the BCL and we are not going to go and annotate every single one of them to be explicitly Auto because they contain some managed field.
tannergooding
tannergooding
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan.cs

 _flattenedLength = linearLength;
 _memoryLength = array.Length;
 _shape = new TensorShape(array.Length - start, lengths, strides);
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if some of the validation logic could be in a helper on TensorShape
-- Not needed to fix in this PR, just a possible simplification opportunity
tannergooding
tannergooding
approved these changes
Jun 19, 2024
View reviewed changes
@tannergoodingtannergooding
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave some nits, but nothing that should block this being merged

This was referenced Jun 20, 2024

Open


Closed


Closed


Closed


Closed

@michaelgsharp
michaelgsharp
commented
Jun 20, 2024

Copy link


Copy Markdown

ContributorAuthor
/ba-g suppressing unrelated test failure to merge ahead of snap.
@michaelgsharp
michaelgsharp
merged commit 4cfc93b
into
dotnet:mainJun 20, 2024
rzikm pushed a commit
to rzikm/dotnet-runtime
that referenced
this pull request
Jun 24, 2024
@michaelgsharp@rzikm
Changed TensorSpan and ReadOnlyTensorSpan layout for better performan…
50458c6
…ce. (dotnet#103244)
* TensorShape
* fixed TensorShape issue
* removed extra inline buffer type. Fixed tests. Added large dimension testing
* removed typo
* adding unsaved files
* changes to use const for stack alloc comparison
* changes from pr comments
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Jul 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@stephentoubstephentoubstephentoub left review comments
@eiriktsarpaliseiriktsarpaliseiriktsarpalis left review comments
@tannergoodingtannergoodingtannergooding approved these changes
@ericstjericstjAwaiting requested review from ericstj

+1 more reviewer

@MichalPetrykaMichalPetrykaMichalPetryka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area-System.Numerics.Tensors

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@michaelgsharp@ericstj@stephentoub@eiriktsarpalis@tannergooding@MichalPetryka@teo-tsirpanis

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. - #103244
Merged
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Jun 20, 2024
Merged
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance.#103244
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. This included making a new span to hold all the shared data between them and allowing for allocation free behavior for dimensions less than 6.
@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.


1 similar comment

@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@dotnet-policy-service
dotnet-policy-serviceBot
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.

This was referenced Jun 10, 2024

Closed


Closed


Closed

@michaelgsharpmichaelgsharp
mentioned this pull request
Jun 12, 2024

Closed

@ericstj
ericstj
commented
Jun 19, 2024

Copy link


Copy Markdown

Member
@tannergooding did you have any more feedback on this one?
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs

namespace System.Numerics.Tensors
{
 internal readonly struct TensorShape
@eiriktsarpaliseiriktsarpalisJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be a ref struct?
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to be able to use the same struct as part of Tensor<T> so we can avoid additional allocations for common cases there as well.
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs
 {
 internal readonly nint[]? _metadata; // 8 bytes

 internal readonly nint _memoryLength; // 8 bytes
@eiriktsarpaliseiriktsarpalisJun 19, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the layout and size concerns of this change, does the new type require an explicit StructLayoutAttribute annotation? Are we ok with using the default layout? Is there an expectation that the fields will be consumed by unmanaged code.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
Does Mono behave the same? I think it might respect Sequential for managed types.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
Yeah but I've meant that Auto could maybe still be beneficial for Mono.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono should likely consider adjusting their layout algorithm to match the RyuJIT algorithm in that scenario.
There are thousands of structs across the BCL and we are not going to go and annotate every single one of them to be explicitly Auto because they contain some managed field.
tannergooding
tannergooding
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan.cs

 _flattenedLength = linearLength;
 _memoryLength = array.Length;
 _shape = new TensorShape(array.Length - start, lengths, strides);
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if some of the validation logic could be in a helper on TensorShape
-- Not needed to fix in this PR, just a possible simplification opportunity
tannergooding
tannergooding
approved these changes
Jun 19, 2024
View reviewed changes
@tannergoodingtannergooding
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave some nits, but nothing that should block this being merged

This was referenced Jun 20, 2024

Open


Closed


Closed


Closed


Closed

@michaelgsharp
michaelgsharp
commented
Jun 20, 2024

Copy link


Copy Markdown

ContributorAuthor
/ba-g suppressing unrelated test failure to merge ahead of snap.
@michaelgsharp
michaelgsharp
merged commit 4cfc93b
into
dotnet:mainJun 20, 2024
rzikm pushed a commit
to rzikm/dotnet-runtime
that referenced
this pull request
Jun 24, 2024
@michaelgsharp@rzikm
Changed TensorSpan and ReadOnlyTensorSpan layout for better performan…
50458c6
…ce. (dotnet#103244)
* TensorShape
* fixed TensorShape issue
* removed extra inline buffer type. Fixed tests. Added large dimension testing
* removed typo
* adding unsaved files
* changes to use const for stack alloc comparison
* changes from pr comments
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Jul 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@stephentoubstephentoubstephentoub left review comments
@eiriktsarpaliseiriktsarpaliseiriktsarpalis left review comments
@tannergoodingtannergoodingtannergooding approved these changes
@ericstjericstjAwaiting requested review from ericstj

+1 more reviewer

@MichalPetrykaMichalPetrykaMichalPetryka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area-System.Numerics.Tensors

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@michaelgsharp@ericstj@stephentoub@eiriktsarpalis@tannergooding@MichalPetryka@teo-tsirpanis

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. - #103244
Merged
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Jun 20, 2024
Merged
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance.#103244
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. This included making a new span to hold all the shared data between them and allowing for allocation free behavior for dimensions less than 6.
@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.


1 similar comment

@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@dotnet-policy-service
dotnet-policy-serviceBot
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.

This was referenced Jun 10, 2024

Closed


Closed


Closed

@michaelgsharpmichaelgsharp
mentioned this pull request
Jun 12, 2024

Closed

@ericstj
ericstj
commented
Jun 19, 2024

Copy link


Copy Markdown

Member
@tannergooding did you have any more feedback on this one?
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs

namespace System.Numerics.Tensors
{
 internal readonly struct TensorShape
@eiriktsarpaliseiriktsarpalisJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be a ref struct?
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to be able to use the same struct as part of Tensor<T> so we can avoid additional allocations for common cases there as well.
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs
 {
 internal readonly nint[]? _metadata; // 8 bytes

 internal readonly nint _memoryLength; // 8 bytes
@eiriktsarpaliseiriktsarpalisJun 19, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the layout and size concerns of this change, does the new type require an explicit StructLayoutAttribute annotation? Are we ok with using the default layout? Is there an expectation that the fields will be consumed by unmanaged code.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
Does Mono behave the same? I think it might respect Sequential for managed types.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
Yeah but I've meant that Auto could maybe still be beneficial for Mono.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono should likely consider adjusting their layout algorithm to match the RyuJIT algorithm in that scenario.
There are thousands of structs across the BCL and we are not going to go and annotate every single one of them to be explicitly Auto because they contain some managed field.
tannergooding
tannergooding
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan.cs

 _flattenedLength = linearLength;
 _memoryLength = array.Length;
 _shape = new TensorShape(array.Length - start, lengths, strides);
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if some of the validation logic could be in a helper on TensorShape
-- Not needed to fix in this PR, just a possible simplification opportunity
tannergooding
tannergooding
approved these changes
Jun 19, 2024
View reviewed changes
@tannergoodingtannergooding
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave some nits, but nothing that should block this being merged

This was referenced Jun 20, 2024

Open


Closed


Closed


Closed


Closed

@michaelgsharp
michaelgsharp
commented
Jun 20, 2024

Copy link


Copy Markdown

ContributorAuthor
/ba-g suppressing unrelated test failure to merge ahead of snap.
@michaelgsharp
michaelgsharp
merged commit 4cfc93b
into
dotnet:mainJun 20, 2024
rzikm pushed a commit
to rzikm/dotnet-runtime
that referenced
this pull request
Jun 24, 2024
@michaelgsharp@rzikm
Changed TensorSpan and ReadOnlyTensorSpan layout for better performan…
50458c6
…ce. (dotnet#103244)
* TensorShape
* fixed TensorShape issue
* removed extra inline buffer type. Fixed tests. Added large dimension testing
* removed typo
* adding unsaved files
* changes to use const for stack alloc comparison
* changes from pr comments
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Jul 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@stephentoubstephentoubstephentoub left review comments
@eiriktsarpaliseiriktsarpaliseiriktsarpalis left review comments
@tannergoodingtannergoodingtannergooding approved these changes
@ericstjericstjAwaiting requested review from ericstj

+1 more reviewer

@MichalPetrykaMichalPetrykaMichalPetryka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area-System.Numerics.Tensors

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@michaelgsharp@ericstj@stephentoub@eiriktsarpalis@tannergooding@MichalPetryka@teo-tsirpanis

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. - #103244
Merged
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Jun 20, 2024
Merged
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance.#103244
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. This included making a new span to hold all the shared data between them and allowing for allocation free behavior for dimensions less than 6.
@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.


1 similar comment

@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@dotnet-policy-service
dotnet-policy-serviceBot
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.

This was referenced Jun 10, 2024

Closed


Closed


Closed

@michaelgsharpmichaelgsharp
mentioned this pull request
Jun 12, 2024

Closed

@ericstj
ericstj
commented
Jun 19, 2024

Copy link


Copy Markdown

Member
@tannergooding did you have any more feedback on this one?
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs

namespace System.Numerics.Tensors
{
 internal readonly struct TensorShape
@eiriktsarpaliseiriktsarpalisJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be a ref struct?
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to be able to use the same struct as part of Tensor<T> so we can avoid additional allocations for common cases there as well.
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs
 {
 internal readonly nint[]? _metadata; // 8 bytes

 internal readonly nint _memoryLength; // 8 bytes
@eiriktsarpaliseiriktsarpalisJun 19, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the layout and size concerns of this change, does the new type require an explicit StructLayoutAttribute annotation? Are we ok with using the default layout? Is there an expectation that the fields will be consumed by unmanaged code.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
Does Mono behave the same? I think it might respect Sequential for managed types.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
Yeah but I've meant that Auto could maybe still be beneficial for Mono.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono should likely consider adjusting their layout algorithm to match the RyuJIT algorithm in that scenario.
There are thousands of structs across the BCL and we are not going to go and annotate every single one of them to be explicitly Auto because they contain some managed field.
tannergooding
tannergooding
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan.cs

 _flattenedLength = linearLength;
 _memoryLength = array.Length;
 _shape = new TensorShape(array.Length - start, lengths, strides);
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if some of the validation logic could be in a helper on TensorShape
-- Not needed to fix in this PR, just a possible simplification opportunity
tannergooding
tannergooding
approved these changes
Jun 19, 2024
View reviewed changes
@tannergoodingtannergooding
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave some nits, but nothing that should block this being merged

This was referenced Jun 20, 2024

Open


Closed


Closed


Closed


Closed

@michaelgsharp
michaelgsharp
commented
Jun 20, 2024

Copy link


Copy Markdown

ContributorAuthor
/ba-g suppressing unrelated test failure to merge ahead of snap.
@michaelgsharp
michaelgsharp
merged commit 4cfc93b
into
dotnet:mainJun 20, 2024
rzikm pushed a commit
to rzikm/dotnet-runtime
that referenced
this pull request
Jun 24, 2024
@michaelgsharp@rzikm
Changed TensorSpan and ReadOnlyTensorSpan layout for better performan…
50458c6
…ce. (dotnet#103244)
* TensorShape
* fixed TensorShape issue
* removed extra inline buffer type. Fixed tests. Added large dimension testing
* removed typo
* adding unsaved files
* changes to use const for stack alloc comparison
* changes from pr comments
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Jul 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@stephentoubstephentoubstephentoub left review comments
@eiriktsarpaliseiriktsarpaliseiriktsarpalis left review comments
@tannergoodingtannergoodingtannergooding approved these changes
@ericstjericstjAwaiting requested review from ericstj

+1 more reviewer

@MichalPetrykaMichalPetrykaMichalPetryka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area-System.Numerics.Tensors

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@michaelgsharp@ericstj@stephentoub@eiriktsarpalis@tannergooding@MichalPetryka@teo-tsirpanis

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. - #103244
Merged
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Jun 20, 2024
Merged
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance.#103244
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. This included making a new span to hold all the shared data between them and allowing for allocation free behavior for dimensions less than 6.
@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.


1 similar comment

@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@dotnet-policy-service
dotnet-policy-serviceBot
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.

This was referenced Jun 10, 2024

Closed


Closed


Closed

@michaelgsharpmichaelgsharp
mentioned this pull request
Jun 12, 2024

Closed

@ericstj
ericstj
commented
Jun 19, 2024

Copy link


Copy Markdown

Member
@tannergooding did you have any more feedback on this one?
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs

namespace System.Numerics.Tensors
{
 internal readonly struct TensorShape
@eiriktsarpaliseiriktsarpalisJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be a ref struct?
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to be able to use the same struct as part of Tensor<T> so we can avoid additional allocations for common cases there as well.
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs
 {
 internal readonly nint[]? _metadata; // 8 bytes

 internal readonly nint _memoryLength; // 8 bytes
@eiriktsarpaliseiriktsarpalisJun 19, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the layout and size concerns of this change, does the new type require an explicit StructLayoutAttribute annotation? Are we ok with using the default layout? Is there an expectation that the fields will be consumed by unmanaged code.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
Does Mono behave the same? I think it might respect Sequential for managed types.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
Yeah but I've meant that Auto could maybe still be beneficial for Mono.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono should likely consider adjusting their layout algorithm to match the RyuJIT algorithm in that scenario.
There are thousands of structs across the BCL and we are not going to go and annotate every single one of them to be explicitly Auto because they contain some managed field.
tannergooding
tannergooding
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan.cs

 _flattenedLength = linearLength;
 _memoryLength = array.Length;
 _shape = new TensorShape(array.Length - start, lengths, strides);
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if some of the validation logic could be in a helper on TensorShape
-- Not needed to fix in this PR, just a possible simplification opportunity
tannergooding
tannergooding
approved these changes
Jun 19, 2024
View reviewed changes
@tannergoodingtannergooding
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave some nits, but nothing that should block this being merged

This was referenced Jun 20, 2024

Open


Closed


Closed


Closed


Closed

@michaelgsharp
michaelgsharp
commented
Jun 20, 2024

Copy link


Copy Markdown

ContributorAuthor
/ba-g suppressing unrelated test failure to merge ahead of snap.
@michaelgsharp
michaelgsharp
merged commit 4cfc93b
into
dotnet:mainJun 20, 2024
rzikm pushed a commit
to rzikm/dotnet-runtime
that referenced
this pull request
Jun 24, 2024
@michaelgsharp@rzikm
Changed TensorSpan and ReadOnlyTensorSpan layout for better performan…
50458c6
…ce. (dotnet#103244)
* TensorShape
* fixed TensorShape issue
* removed extra inline buffer type. Fixed tests. Added large dimension testing
* removed typo
* adding unsaved files
* changes to use const for stack alloc comparison
* changes from pr comments
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Jul 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@stephentoubstephentoubstephentoub left review comments
@eiriktsarpaliseiriktsarpaliseiriktsarpalis left review comments
@tannergoodingtannergoodingtannergooding approved these changes
@ericstjericstjAwaiting requested review from ericstj

+1 more reviewer

@MichalPetrykaMichalPetrykaMichalPetryka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area-System.Numerics.Tensors

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@michaelgsharp@ericstj@stephentoub@eiriktsarpalis@tannergooding@MichalPetryka@teo-tsirpanis

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. - #103244
Merged
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Jun 20, 2024
Merged
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance.#103244
michaelgsharp merged 7 commits into
dotnet:mainfrom 
michaelgsharp:tensor-struct
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Changed TensorSpan and ReadOnlyTensorSpan layout for better performance. This included making a new span to hold all the shared data between them and allowing for allocation free behavior for dimensions less than 6.
@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.


1 similar comment

@ghost
ghost
commented
Jun 10, 2024

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@dotnet-policy-service
dotnet-policy-serviceBot
commented
Jun 10, 2024

Copy link


Copy Markdown

Contributor
Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.

This was referenced Jun 10, 2024

Closed


Closed


Closed

@michaelgsharpmichaelgsharp
mentioned this pull request
Jun 12, 2024

Closed

@ericstj
ericstj
commented
Jun 19, 2024

Copy link


Copy Markdown

Member
@tannergooding did you have any more feedback on this one?
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs

namespace System.Numerics.Tensors
{
 internal readonly struct TensorShape
@eiriktsarpaliseiriktsarpalisJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be a ref struct?
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to be able to use the same struct as part of Tensor<T> so we can avoid additional allocations for common cases there as well.
eiriktsarpalis
eiriktsarpalis
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs
 {
 internal readonly nint[]? _metadata; // 8 bytes

 internal readonly nint _memoryLength; // 8 bytes
@eiriktsarpaliseiriktsarpalisJun 19, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the layout and size concerns of this change, does the new type require an explicit StructLayoutAttribute annotation? Are we ok with using the default layout? Is there an expectation that the fields will be consumed by unmanaged code.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This struct contains a managed field (nint[]? _metadata) and as such is not blittable and will never have StructLayout respected, the runtime will treat it as Auto regardless (you can technically mark it as LayoutKind.Explicit and that would be respected, but that's a de-optimization and would cause many other issues).
Does Mono behave the same? I think it might respect Sequential for managed types.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
@MichalPetrykaMichalPetrykaJun 19, 2024

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Mono behave the same? I think it might respect Sequential for managed types.
That doesn't really matter. RyuJIT doesn't and so code cannot depend on the managed side being sequential, that is part of why we must use InlineArray (rather than declaring n-sequential fields).
This is likewise an internal struct, so it can't be used by developers in interop scenarios. Due to it being a struct containing a managed field, it likewise couldn't be used in interop without marshalling and so there is zero benefit to changing the layout from its default.
Yeah but I've meant that Auto could maybe still be beneficial for Mono.
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono should likely consider adjusting their layout algorithm to match the RyuJIT algorithm in that scenario.
There are thousands of structs across the BCL and we are not going to go and annotate every single one of them to be explicitly Auto because they contain some managed field.
tannergooding
tannergooding
reviewed
Jun 19, 2024
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan.cs

 _flattenedLength = linearLength;
 _memoryLength = array.Length;
 _shape = new TensorShape(array.Length - start, lengths, strides);
@tannergoodingtannergoodingJun 19, 2024

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if some of the validation logic could be in a helper on TensorShape
-- Not needed to fix in this PR, just a possible simplification opportunity
tannergooding
tannergooding
approved these changes
Jun 19, 2024
View reviewed changes
@tannergoodingtannergooding
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave some nits, but nothing that should block this being merged

This was referenced Jun 20, 2024

Open


Closed


Closed


Closed


Closed

@michaelgsharp
michaelgsharp
commented
Jun 20, 2024

Copy link


Copy Markdown

ContributorAuthor
/ba-g suppressing unrelated test failure to merge ahead of snap.
@michaelgsharp
michaelgsharp
merged commit 4cfc93b
into
dotnet:mainJun 20, 2024
rzikm pushed a commit
to rzikm/dotnet-runtime
that referenced
this pull request
Jun 24, 2024
@michaelgsharp@rzikm
Changed TensorSpan and ReadOnlyTensorSpan layout for better performan…
50458c6
…ce. (dotnet#103244)
* TensorShape
* fixed TensorShape issue
* removed extra inline buffer type. Fixed tests. Added large dimension testing
* removed typo
* adding unsaved files
* changes to use const for stack alloc comparison
* changes from pr comments
@github-actionsgithub-actionsBot
locked and limited conversation to collaborators
Jul 21, 2024
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@stephentoubstephentoubstephentoub left review comments
@eiriktsarpaliseiriktsarpaliseiriktsarpalis left review comments
@tannergoodingtannergoodingtannergooding approved these changes
@ericstjericstjAwaiting requested review from ericstj

+1 more reviewer

@MichalPetrykaMichalPetrykaMichalPetryka left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area-System.Numerics.Tensors

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

7 participants

@michaelgsharp@ericstj@stephentoub@eiriktsarpalis@tannergooding@MichalPetryka@teo-tsirpanis