Skip to content

6 more naive methods for Tensor Primitives. - #92142

Merged
stephentoub merged 2 commits into
dotnet:mainfrom
michaelgsharp:finish-naive-tensor
Sep 16, 2023
Merged

6 more naive methods for Tensor Primitives.#92142
stephentoub merged 2 commits into
dotnet:mainfrom
michaelgsharp:finish-naive-tensor

Conversation

@michaelgsharp

@michaelgsharpmichaelgsharp commented Sep 15, 2023

Copy link
Copy Markdown
Contributor

Added in naive implementations for

  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid

@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?

@ghost

ghost commented Sep 15, 2023

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost

ghost commented Sep 15, 2023

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors
See info in area-owners.md if you want to be subscribed.

Issue Details

Added in naive implementations for

  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid

@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?

Author:michaelgsharp
Assignees:michaelgsharp
Labels:

area-System.Numerics.Tensors, new-api-needs-documentation

Milestone:-

@stephentoubstephentoub mentioned this pull request Sep 15, 2023
Merged
gfoidl
gfoidl reviewed Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Outdated
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Outdated
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Outdated
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Outdated
stephentoub
stephentoub reviewed Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Outdated
stephentoub
stephentoub reviewed Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Outdated
stephentoub
stephentoub reviewed Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub reviewed Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub reviewed Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Outdated
ericstj
ericstj approved these changes Sep 16, 2023
View reviewed changes

@ericstjericstj left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

stephentoub
stephentoub approved these changes Sep 16, 2023
View reviewed changes

@stephentoubstephentoub left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this and then fix up a few things after rebasing my PR on it.

@stephentoub
stephentoub merged commit 7be3913 into dotnet:mainSep 16, 2023
jeffhandley
jeffhandley reviewed Sep 16, 2023
View reviewed changes

@jeffhandleyjeffhandley left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Echoing the existing feedback about tests, we could add a lot more tests for all these methods to capture edge/corner cases and more exceptional data too.

I meant to say, but we can add more tests after the RC2 milestone/snap

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
tannergooding
tannergooding reviewed Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
{
ThrowHelper.ThrowArgument_SpansMustHaveSameLength();
}
if (x.Length == 0 || y.Length == 0)

@tannergoodingtannergoodingSep 18, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need one of the checks here since we've already determined the lengths are equal.

That being said, is it desirable to throw here rather than allowing it to just return NaN?

tannergooding
tannergooding reviewed Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
/// </summary>
/// <param name="x">The first tensor, represented as a span.</param>
/// <returns>The L2 norm.</returns>
public static float L2Normalize(ReadOnlySpan<float> x) // BLAS1: nrm2

@tannergoodingtannergoodingSep 18, 2023•
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this L2Normalize? The standard API computes the "euclidean norm", that is the length or magnitude of the vector.

The name should likely just be Norm (or Length which is what it's called for Vector2/3/4.Length())

tannergooding
tannergooding reviewed Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

for (int i = 0; i < x.Length; i++)
{
expSum += MathF.Pow((float)Math.E, x[i]);

@tannergoodingtannergoodingSep 18, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why MathF.Pow and not MathF.Exp?

Why (float)Math.E and not MathF.E?

@stephentoubstephentoubSep 18, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tannergooding, can you submit a cleanup PR with the changes you think should be made? Thanks.

michaelgsharp added a commit to michaelgsharp/runtime that referenced this pull request Sep 18, 2023
@michaelgsharp
6 more naive methods for Tensor Primitives. (dotnet#92142)
96ba8c1
* 6 more naive methods
* updates from pr comments
ericstj pushed a commit that referenced this pull request Sep 19, 2023
@michaelgsharp@tannergooding
@ViktorHofer@stephentoub@tmds
[release/8.0] Numerics and Tensors backport (#92245)
4b7c754
* added Bcl.Numerics
* Adding a naive implementation of various primitive tensor operations (#91228)
* Adding a naive implementation of various primitive tensor operations
* Adding tests covering the new tensor primitives APIs
* Adding tensor primitives APIs to the ref assembly
* Allow .NET Framework to build/run
* Sync TFMs between ref and src, csproj simplication and clean-up
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Don't use var
* Fix the S.N.Tensors readme and remove the file marking it as non-shipping
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
* Start vectorizing TensorPrimitives (#91596)
* Start vectorizing TensorPrimitives
Just does two functions to establish the files into which the rest of the implementations can be moved.
* 6 more naive methods for Tensor Primitives. (#92142)
* 6 more naive methods
* updates from pr comments
* Add remaining set of TensorPrimitives APIs for .NET 8 (#92154)
* Add remaining set of TensorPrimitives APIs for .NET 8
Adds non-vectorized implementations of:
- Max
- Min
- MaxMagnitude
- MinMagnitude
- IndexOfMax
- IndexOfMin
- IndexOfMaxMagnitude
- ConvertToHalf (only on .NET Core)
- ConvertToSingle (only on .NET Core)
- IndexOfMinMagnitude
Adds vectorized implementations of:
- Sum
- SumOfSquares
- SumOfMagnitudes
- Product
- ProductOfSums
- ProductOfDifferences
Also includes the helpers that'll make it trivial to vectorize Dot.
Beyond vectorizing the non-vectorized ones, the vectorized implementations should be improved further, including:
- Handling alignment better
- Vectorizing the remainder that doesn't fit in a vector rather than falling back to scalar
* Cleanup after previous PR, vectorize CosineSimilarity/Dot/L2Normalize/Distance, add tests
* Address PR feedback, and fix a few other issues
* Fix TensorPrimitives.CosineSimilarity to use vectorized implementations (#92204)
* Fixed duplicated code from merge.
* New Microsoft.BCL.Numerics package (#91074)
* bcl numberics library added
* bcl done
* added explicit 2.1 target
* Minor doc updates
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* fixes from PR comments
* minor csproj fixes
* fixed ref target frameworks
* minor ref csproj updates
* minor csproj updates
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource. (#91402)
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource.
* Use NetCoreAppCurrent.
* Try fix CI test failures.
---------
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
@ghostghost locked as resolved and limited conversation to collaborators Oct 18, 2023
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@jeffhandleyjeffhandleyjeffhandley left review comments
@stephentoubstephentoubstephentoub approved these changes
@tannergoodingtannergoodingtannergooding left review comments
@ericstjericstjericstj approved these changes
+1 more reviewer
@gfoidlgfoidlgfoidl left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@michaelgsharpmichaelgsharp

Labels

area-System.Numerics.Tensorsnew-api-needs-documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@michaelgsharp@jeffhandley@stephentoub@gfoidl@ericstj@tannergooding
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
6 more naive methods for Tensor Primitives. by michaelgsharp · Pull Request #92142 · dotnet/runtime · GitHub
Skip to content
6 more naive methods for Tensor Primitives. - #92142
Merged
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Sep 16, 2023
Merged
6 more naive methods for Tensor Primitives.#92142
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Sep 15, 2023

Copy link


Copy Markdown

Contributor
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.
Issue Details
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
Author:michaelgsharp
Assignees:michaelgsharp
Labels:
area-System.Numerics.Tensors, new-api-needs-documentation
Milestone:-
@stephentoubstephentoub
mentioned this pull request
Sep 15, 2023
 Merged

gfoidl
gfoidl
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

ericstj
ericstj
approved these changes
Sep 16, 2023
View reviewed changes
@ericstjericstj
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
stephentoub
stephentoub
approved these changes
Sep 16, 2023
View reviewed changes
@stephentoubstephentoub
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this and then fix up a few things after rebasing my PR on it.
@stephentoub
stephentoub
merged commit 7be3913
into
dotnet:mainSep 16, 2023
jeffhandley
jeffhandley
reviewed
Sep 16, 2023
View reviewed changes
@jeffhandleyjeffhandley
left a comment
•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Echoing the existing feedback about tests, we could add a lot more tests for all these methods to capture edge/corner cases and more exceptional data too.
I meant to say, but we can add more tests after the RC2 milestone/snap
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 {
 ThrowHelper.ThrowArgument_SpansMustHaveSameLength();
 }
 if (x.Length == 0 || y.Length == 0)
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need one of the checks here since we've already determined the lengths are equal.
That being said, is it desirable to throw here rather than allowing it to just return NaN?
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 /// </summary>
 /// <param name="x">The first tensor, represented as a span.</param>
 /// <returns>The L2 norm.</returns>
 public static float L2Normalize(ReadOnlySpan<float> x) // BLAS1: nrm2
@tannergoodingtannergoodingSep 18, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this L2Normalize? The standard API computes the "euclidean norm", that is the length or magnitude of the vector.
The name should likely just be Norm (or Length which is what it's called for Vector2/3/4.Length())
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

 for (int i = 0; i < x.Length; i++)
 {
 expSum += MathF.Pow((float)Math.E, x[i]);
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why MathF.Pow and not MathF.Exp?
Why (float)Math.E and not MathF.E?
@stephentoubstephentoubSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tannergooding, can you submit a cleanup PR with the changes you think should be made? Thanks.
michaelgsharp added a commit
to michaelgsharp/runtime
that referenced
this pull request
Sep 18, 2023
@michaelgsharp
6 more naive methods for Tensor Primitives. (dotnet#92142)
96ba8c1
* 6 more naive methods
* updates from pr comments
ericstj pushed a commit
that referenced
this pull request
Sep 19, 2023
@michaelgsharp@tannergooding
@ViktorHofer@stephentoub@tmds
[release/8.0] Numerics and Tensors backport (#92245)
4b7c754
* added Bcl.Numerics
* Adding a naive implementation of various primitive tensor operations (#91228)
* Adding a naive implementation of various primitive tensor operations
* Adding tests covering the new tensor primitives APIs
* Adding tensor primitives APIs to the ref assembly
* Allow .NET Framework to build/run
* Sync TFMs between ref and src, csproj simplication and clean-up
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Don't use var
* Fix the S.N.Tensors readme and remove the file marking it as non-shipping
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
* Start vectorizing TensorPrimitives (#91596)
* Start vectorizing TensorPrimitives
Just does two functions to establish the files into which the rest of the implementations can be moved.
* 6 more naive methods for Tensor Primitives. (#92142)
* 6 more naive methods
* updates from pr comments
* Add remaining set of TensorPrimitives APIs for .NET 8 (#92154)
* Add remaining set of TensorPrimitives APIs for .NET 8
Adds non-vectorized implementations of:
- Max
- Min
- MaxMagnitude
- MinMagnitude
- IndexOfMax
- IndexOfMin
- IndexOfMaxMagnitude
- ConvertToHalf (only on .NET Core)
- ConvertToSingle (only on .NET Core)
- IndexOfMinMagnitude
Adds vectorized implementations of:
- Sum
- SumOfSquares
- SumOfMagnitudes
- Product
- ProductOfSums
- ProductOfDifferences
Also includes the helpers that'll make it trivial to vectorize Dot.
Beyond vectorizing the non-vectorized ones, the vectorized implementations should be improved further, including:
- Handling alignment better
- Vectorizing the remainder that doesn't fit in a vector rather than falling back to scalar
* Cleanup after previous PR, vectorize CosineSimilarity/Dot/L2Normalize/Distance, add tests
* Address PR feedback, and fix a few other issues
* Fix TensorPrimitives.CosineSimilarity to use vectorized implementations (#92204)
* Fixed duplicated code from merge.
* New Microsoft.BCL.Numerics package (#91074)
* bcl numberics library added
* bcl done
* added explicit 2.1 target
* Minor doc updates
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* fixes from PR comments
* minor csproj fixes
* fixed ref target frameworks
* minor ref csproj updates
* minor csproj updates
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource. (#91402)
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource.
* Use NetCoreAppCurrent.
* Try fix CI test failures.
---------
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
Oct 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jeffhandleyjeffhandleyjeffhandley left review comments
@stephentoubstephentoubstephentoub approved these changes
@tannergoodingtannergoodingtannergooding left review comments
@ericstjericstjericstj approved these changes

+1 more reviewer

@gfoidlgfoidlgfoidl left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@michaelgsharpmichaelgsharp

Labels

area-System.Numerics.Tensorsnew-api-needs-documentation

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@michaelgsharp@jeffhandley@stephentoub@gfoidl@ericstj@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
6 more naive methods for Tensor Primitives. by michaelgsharp · Pull Request #92142 · dotnet/runtime · GitHub
Skip to content
6 more naive methods for Tensor Primitives. - #92142
Merged
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Sep 16, 2023
Merged
6 more naive methods for Tensor Primitives.#92142
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Sep 15, 2023

Copy link


Copy Markdown

Contributor
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.
Issue Details
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
Author:michaelgsharp
Assignees:michaelgsharp
Labels:
area-System.Numerics.Tensors, new-api-needs-documentation
Milestone:-
@stephentoubstephentoub
mentioned this pull request
Sep 15, 2023
 Merged

gfoidl
gfoidl
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

ericstj
ericstj
approved these changes
Sep 16, 2023
View reviewed changes
@ericstjericstj
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
stephentoub
stephentoub
approved these changes
Sep 16, 2023
View reviewed changes
@stephentoubstephentoub
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this and then fix up a few things after rebasing my PR on it.
@stephentoub
stephentoub
merged commit 7be3913
into
dotnet:mainSep 16, 2023
jeffhandley
jeffhandley
reviewed
Sep 16, 2023
View reviewed changes
@jeffhandleyjeffhandley
left a comment
•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Echoing the existing feedback about tests, we could add a lot more tests for all these methods to capture edge/corner cases and more exceptional data too.
I meant to say, but we can add more tests after the RC2 milestone/snap
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 {
 ThrowHelper.ThrowArgument_SpansMustHaveSameLength();
 }
 if (x.Length == 0 || y.Length == 0)
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need one of the checks here since we've already determined the lengths are equal.
That being said, is it desirable to throw here rather than allowing it to just return NaN?
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 /// </summary>
 /// <param name="x">The first tensor, represented as a span.</param>
 /// <returns>The L2 norm.</returns>
 public static float L2Normalize(ReadOnlySpan<float> x) // BLAS1: nrm2
@tannergoodingtannergoodingSep 18, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this L2Normalize? The standard API computes the "euclidean norm", that is the length or magnitude of the vector.
The name should likely just be Norm (or Length which is what it's called for Vector2/3/4.Length())
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

 for (int i = 0; i < x.Length; i++)
 {
 expSum += MathF.Pow((float)Math.E, x[i]);
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why MathF.Pow and not MathF.Exp?
Why (float)Math.E and not MathF.E?
@stephentoubstephentoubSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tannergooding, can you submit a cleanup PR with the changes you think should be made? Thanks.
michaelgsharp added a commit
to michaelgsharp/runtime
that referenced
this pull request
Sep 18, 2023
@michaelgsharp
6 more naive methods for Tensor Primitives. (dotnet#92142)
96ba8c1
* 6 more naive methods
* updates from pr comments
ericstj pushed a commit
that referenced
this pull request
Sep 19, 2023
@michaelgsharp@tannergooding
@ViktorHofer@stephentoub@tmds
[release/8.0] Numerics and Tensors backport (#92245)
4b7c754
* added Bcl.Numerics
* Adding a naive implementation of various primitive tensor operations (#91228)
* Adding a naive implementation of various primitive tensor operations
* Adding tests covering the new tensor primitives APIs
* Adding tensor primitives APIs to the ref assembly
* Allow .NET Framework to build/run
* Sync TFMs between ref and src, csproj simplication and clean-up
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Don't use var
* Fix the S.N.Tensors readme and remove the file marking it as non-shipping
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
* Start vectorizing TensorPrimitives (#91596)
* Start vectorizing TensorPrimitives
Just does two functions to establish the files into which the rest of the implementations can be moved.
* 6 more naive methods for Tensor Primitives. (#92142)
* 6 more naive methods
* updates from pr comments
* Add remaining set of TensorPrimitives APIs for .NET 8 (#92154)
* Add remaining set of TensorPrimitives APIs for .NET 8
Adds non-vectorized implementations of:
- Max
- Min
- MaxMagnitude
- MinMagnitude
- IndexOfMax
- IndexOfMin
- IndexOfMaxMagnitude
- ConvertToHalf (only on .NET Core)
- ConvertToSingle (only on .NET Core)
- IndexOfMinMagnitude
Adds vectorized implementations of:
- Sum
- SumOfSquares
- SumOfMagnitudes
- Product
- ProductOfSums
- ProductOfDifferences
Also includes the helpers that'll make it trivial to vectorize Dot.
Beyond vectorizing the non-vectorized ones, the vectorized implementations should be improved further, including:
- Handling alignment better
- Vectorizing the remainder that doesn't fit in a vector rather than falling back to scalar
* Cleanup after previous PR, vectorize CosineSimilarity/Dot/L2Normalize/Distance, add tests
* Address PR feedback, and fix a few other issues
* Fix TensorPrimitives.CosineSimilarity to use vectorized implementations (#92204)
* Fixed duplicated code from merge.
* New Microsoft.BCL.Numerics package (#91074)
* bcl numberics library added
* bcl done
* added explicit 2.1 target
* Minor doc updates
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* fixes from PR comments
* minor csproj fixes
* fixed ref target frameworks
* minor ref csproj updates
* minor csproj updates
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource. (#91402)
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource.
* Use NetCoreAppCurrent.
* Try fix CI test failures.
---------
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
Oct 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jeffhandleyjeffhandleyjeffhandley left review comments
@stephentoubstephentoubstephentoub approved these changes
@tannergoodingtannergoodingtannergooding left review comments
@ericstjericstjericstj approved these changes

+1 more reviewer

@gfoidlgfoidlgfoidl left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@michaelgsharpmichaelgsharp

Labels

area-System.Numerics.Tensorsnew-api-needs-documentation

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@michaelgsharp@jeffhandley@stephentoub@gfoidl@ericstj@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
6 more naive methods for Tensor Primitives. by michaelgsharp · Pull Request #92142 · dotnet/runtime · GitHub
Skip to content
6 more naive methods for Tensor Primitives. - #92142
Merged
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Sep 16, 2023
Merged
6 more naive methods for Tensor Primitives.#92142
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Sep 15, 2023

Copy link


Copy Markdown

Contributor
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.
Issue Details
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
Author:michaelgsharp
Assignees:michaelgsharp
Labels:
area-System.Numerics.Tensors, new-api-needs-documentation
Milestone:-
@stephentoubstephentoub
mentioned this pull request
Sep 15, 2023
 Merged

gfoidl
gfoidl
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

ericstj
ericstj
approved these changes
Sep 16, 2023
View reviewed changes
@ericstjericstj
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
stephentoub
stephentoub
approved these changes
Sep 16, 2023
View reviewed changes
@stephentoubstephentoub
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this and then fix up a few things after rebasing my PR on it.
@stephentoub
stephentoub
merged commit 7be3913
into
dotnet:mainSep 16, 2023
jeffhandley
jeffhandley
reviewed
Sep 16, 2023
View reviewed changes
@jeffhandleyjeffhandley
left a comment
•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Echoing the existing feedback about tests, we could add a lot more tests for all these methods to capture edge/corner cases and more exceptional data too.
I meant to say, but we can add more tests after the RC2 milestone/snap
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 {
 ThrowHelper.ThrowArgument_SpansMustHaveSameLength();
 }
 if (x.Length == 0 || y.Length == 0)
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need one of the checks here since we've already determined the lengths are equal.
That being said, is it desirable to throw here rather than allowing it to just return NaN?
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 /// </summary>
 /// <param name="x">The first tensor, represented as a span.</param>
 /// <returns>The L2 norm.</returns>
 public static float L2Normalize(ReadOnlySpan<float> x) // BLAS1: nrm2
@tannergoodingtannergoodingSep 18, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this L2Normalize? The standard API computes the "euclidean norm", that is the length or magnitude of the vector.
The name should likely just be Norm (or Length which is what it's called for Vector2/3/4.Length())
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

 for (int i = 0; i < x.Length; i++)
 {
 expSum += MathF.Pow((float)Math.E, x[i]);
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why MathF.Pow and not MathF.Exp?
Why (float)Math.E and not MathF.E?
@stephentoubstephentoubSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tannergooding, can you submit a cleanup PR with the changes you think should be made? Thanks.
michaelgsharp added a commit
to michaelgsharp/runtime
that referenced
this pull request
Sep 18, 2023
@michaelgsharp
6 more naive methods for Tensor Primitives. (dotnet#92142)
96ba8c1
* 6 more naive methods
* updates from pr comments
ericstj pushed a commit
that referenced
this pull request
Sep 19, 2023
@michaelgsharp@tannergooding
@ViktorHofer@stephentoub@tmds
[release/8.0] Numerics and Tensors backport (#92245)
4b7c754
* added Bcl.Numerics
* Adding a naive implementation of various primitive tensor operations (#91228)
* Adding a naive implementation of various primitive tensor operations
* Adding tests covering the new tensor primitives APIs
* Adding tensor primitives APIs to the ref assembly
* Allow .NET Framework to build/run
* Sync TFMs between ref and src, csproj simplication and clean-up
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Don't use var
* Fix the S.N.Tensors readme and remove the file marking it as non-shipping
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
* Start vectorizing TensorPrimitives (#91596)
* Start vectorizing TensorPrimitives
Just does two functions to establish the files into which the rest of the implementations can be moved.
* 6 more naive methods for Tensor Primitives. (#92142)
* 6 more naive methods
* updates from pr comments
* Add remaining set of TensorPrimitives APIs for .NET 8 (#92154)
* Add remaining set of TensorPrimitives APIs for .NET 8
Adds non-vectorized implementations of:
- Max
- Min
- MaxMagnitude
- MinMagnitude
- IndexOfMax
- IndexOfMin
- IndexOfMaxMagnitude
- ConvertToHalf (only on .NET Core)
- ConvertToSingle (only on .NET Core)
- IndexOfMinMagnitude
Adds vectorized implementations of:
- Sum
- SumOfSquares
- SumOfMagnitudes
- Product
- ProductOfSums
- ProductOfDifferences
Also includes the helpers that'll make it trivial to vectorize Dot.
Beyond vectorizing the non-vectorized ones, the vectorized implementations should be improved further, including:
- Handling alignment better
- Vectorizing the remainder that doesn't fit in a vector rather than falling back to scalar
* Cleanup after previous PR, vectorize CosineSimilarity/Dot/L2Normalize/Distance, add tests
* Address PR feedback, and fix a few other issues
* Fix TensorPrimitives.CosineSimilarity to use vectorized implementations (#92204)
* Fixed duplicated code from merge.
* New Microsoft.BCL.Numerics package (#91074)
* bcl numberics library added
* bcl done
* added explicit 2.1 target
* Minor doc updates
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* fixes from PR comments
* minor csproj fixes
* fixed ref target frameworks
* minor ref csproj updates
* minor csproj updates
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource. (#91402)
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource.
* Use NetCoreAppCurrent.
* Try fix CI test failures.
---------
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
Oct 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jeffhandleyjeffhandleyjeffhandley left review comments
@stephentoubstephentoubstephentoub approved these changes
@tannergoodingtannergoodingtannergooding left review comments
@ericstjericstjericstj approved these changes

+1 more reviewer

@gfoidlgfoidlgfoidl left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@michaelgsharpmichaelgsharp

Labels

area-System.Numerics.Tensorsnew-api-needs-documentation

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@michaelgsharp@jeffhandley@stephentoub@gfoidl@ericstj@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
6 more naive methods for Tensor Primitives. by michaelgsharp · Pull Request #92142 · dotnet/runtime · GitHub
Skip to content
6 more naive methods for Tensor Primitives. - #92142
Merged
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Sep 16, 2023
Merged
6 more naive methods for Tensor Primitives.#92142
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Sep 15, 2023

Copy link


Copy Markdown

Contributor
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.
Issue Details
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
Author:michaelgsharp
Assignees:michaelgsharp
Labels:
area-System.Numerics.Tensors, new-api-needs-documentation
Milestone:-
@stephentoubstephentoub
mentioned this pull request
Sep 15, 2023
 Merged

gfoidl
gfoidl
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

ericstj
ericstj
approved these changes
Sep 16, 2023
View reviewed changes
@ericstjericstj
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
stephentoub
stephentoub
approved these changes
Sep 16, 2023
View reviewed changes
@stephentoubstephentoub
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this and then fix up a few things after rebasing my PR on it.
@stephentoub
stephentoub
merged commit 7be3913
into
dotnet:mainSep 16, 2023
jeffhandley
jeffhandley
reviewed
Sep 16, 2023
View reviewed changes
@jeffhandleyjeffhandley
left a comment
•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Echoing the existing feedback about tests, we could add a lot more tests for all these methods to capture edge/corner cases and more exceptional data too.
I meant to say, but we can add more tests after the RC2 milestone/snap
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 {
 ThrowHelper.ThrowArgument_SpansMustHaveSameLength();
 }
 if (x.Length == 0 || y.Length == 0)
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need one of the checks here since we've already determined the lengths are equal.
That being said, is it desirable to throw here rather than allowing it to just return NaN?
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 /// </summary>
 /// <param name="x">The first tensor, represented as a span.</param>
 /// <returns>The L2 norm.</returns>
 public static float L2Normalize(ReadOnlySpan<float> x) // BLAS1: nrm2
@tannergoodingtannergoodingSep 18, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this L2Normalize? The standard API computes the "euclidean norm", that is the length or magnitude of the vector.
The name should likely just be Norm (or Length which is what it's called for Vector2/3/4.Length())
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

 for (int i = 0; i < x.Length; i++)
 {
 expSum += MathF.Pow((float)Math.E, x[i]);
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why MathF.Pow and not MathF.Exp?
Why (float)Math.E and not MathF.E?
@stephentoubstephentoubSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tannergooding, can you submit a cleanup PR with the changes you think should be made? Thanks.
michaelgsharp added a commit
to michaelgsharp/runtime
that referenced
this pull request
Sep 18, 2023
@michaelgsharp
6 more naive methods for Tensor Primitives. (dotnet#92142)
96ba8c1
* 6 more naive methods
* updates from pr comments
ericstj pushed a commit
that referenced
this pull request
Sep 19, 2023
@michaelgsharp@tannergooding
@ViktorHofer@stephentoub@tmds
[release/8.0] Numerics and Tensors backport (#92245)
4b7c754
* added Bcl.Numerics
* Adding a naive implementation of various primitive tensor operations (#91228)
* Adding a naive implementation of various primitive tensor operations
* Adding tests covering the new tensor primitives APIs
* Adding tensor primitives APIs to the ref assembly
* Allow .NET Framework to build/run
* Sync TFMs between ref and src, csproj simplication and clean-up
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Don't use var
* Fix the S.N.Tensors readme and remove the file marking it as non-shipping
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
* Start vectorizing TensorPrimitives (#91596)
* Start vectorizing TensorPrimitives
Just does two functions to establish the files into which the rest of the implementations can be moved.
* 6 more naive methods for Tensor Primitives. (#92142)
* 6 more naive methods
* updates from pr comments
* Add remaining set of TensorPrimitives APIs for .NET 8 (#92154)
* Add remaining set of TensorPrimitives APIs for .NET 8
Adds non-vectorized implementations of:
- Max
- Min
- MaxMagnitude
- MinMagnitude
- IndexOfMax
- IndexOfMin
- IndexOfMaxMagnitude
- ConvertToHalf (only on .NET Core)
- ConvertToSingle (only on .NET Core)
- IndexOfMinMagnitude
Adds vectorized implementations of:
- Sum
- SumOfSquares
- SumOfMagnitudes
- Product
- ProductOfSums
- ProductOfDifferences
Also includes the helpers that'll make it trivial to vectorize Dot.
Beyond vectorizing the non-vectorized ones, the vectorized implementations should be improved further, including:
- Handling alignment better
- Vectorizing the remainder that doesn't fit in a vector rather than falling back to scalar
* Cleanup after previous PR, vectorize CosineSimilarity/Dot/L2Normalize/Distance, add tests
* Address PR feedback, and fix a few other issues
* Fix TensorPrimitives.CosineSimilarity to use vectorized implementations (#92204)
* Fixed duplicated code from merge.
* New Microsoft.BCL.Numerics package (#91074)
* bcl numberics library added
* bcl done
* added explicit 2.1 target
* Minor doc updates
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* fixes from PR comments
* minor csproj fixes
* fixed ref target frameworks
* minor ref csproj updates
* minor csproj updates
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource. (#91402)
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource.
* Use NetCoreAppCurrent.
* Try fix CI test failures.
---------
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
Oct 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jeffhandleyjeffhandleyjeffhandley left review comments
@stephentoubstephentoubstephentoub approved these changes
@tannergoodingtannergoodingtannergooding left review comments
@ericstjericstjericstj approved these changes

+1 more reviewer

@gfoidlgfoidlgfoidl left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@michaelgsharpmichaelgsharp

Labels

area-System.Numerics.Tensorsnew-api-needs-documentation

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@michaelgsharp@jeffhandley@stephentoub@gfoidl@ericstj@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
6 more naive methods for Tensor Primitives. by michaelgsharp · Pull Request #92142 · dotnet/runtime · GitHub
Skip to content
6 more naive methods for Tensor Primitives. - #92142
Merged
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Sep 16, 2023
Merged
6 more naive methods for Tensor Primitives.#92142
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Sep 15, 2023

Copy link


Copy Markdown

Contributor
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.
Issue Details
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
Author:michaelgsharp
Assignees:michaelgsharp
Labels:
area-System.Numerics.Tensors, new-api-needs-documentation
Milestone:-
@stephentoubstephentoub
mentioned this pull request
Sep 15, 2023
 Merged

gfoidl
gfoidl
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

ericstj
ericstj
approved these changes
Sep 16, 2023
View reviewed changes
@ericstjericstj
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
stephentoub
stephentoub
approved these changes
Sep 16, 2023
View reviewed changes
@stephentoubstephentoub
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this and then fix up a few things after rebasing my PR on it.
@stephentoub
stephentoub
merged commit 7be3913
into
dotnet:mainSep 16, 2023
jeffhandley
jeffhandley
reviewed
Sep 16, 2023
View reviewed changes
@jeffhandleyjeffhandley
left a comment
•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Echoing the existing feedback about tests, we could add a lot more tests for all these methods to capture edge/corner cases and more exceptional data too.
I meant to say, but we can add more tests after the RC2 milestone/snap
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 {
 ThrowHelper.ThrowArgument_SpansMustHaveSameLength();
 }
 if (x.Length == 0 || y.Length == 0)
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need one of the checks here since we've already determined the lengths are equal.
That being said, is it desirable to throw here rather than allowing it to just return NaN?
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 /// </summary>
 /// <param name="x">The first tensor, represented as a span.</param>
 /// <returns>The L2 norm.</returns>
 public static float L2Normalize(ReadOnlySpan<float> x) // BLAS1: nrm2
@tannergoodingtannergoodingSep 18, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this L2Normalize? The standard API computes the "euclidean norm", that is the length or magnitude of the vector.
The name should likely just be Norm (or Length which is what it's called for Vector2/3/4.Length())
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

 for (int i = 0; i < x.Length; i++)
 {
 expSum += MathF.Pow((float)Math.E, x[i]);
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why MathF.Pow and not MathF.Exp?
Why (float)Math.E and not MathF.E?
@stephentoubstephentoubSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tannergooding, can you submit a cleanup PR with the changes you think should be made? Thanks.
michaelgsharp added a commit
to michaelgsharp/runtime
that referenced
this pull request
Sep 18, 2023
@michaelgsharp
6 more naive methods for Tensor Primitives. (dotnet#92142)
96ba8c1
* 6 more naive methods
* updates from pr comments
ericstj pushed a commit
that referenced
this pull request
Sep 19, 2023
@michaelgsharp@tannergooding
@ViktorHofer@stephentoub@tmds
[release/8.0] Numerics and Tensors backport (#92245)
4b7c754
* added Bcl.Numerics
* Adding a naive implementation of various primitive tensor operations (#91228)
* Adding a naive implementation of various primitive tensor operations
* Adding tests covering the new tensor primitives APIs
* Adding tensor primitives APIs to the ref assembly
* Allow .NET Framework to build/run
* Sync TFMs between ref and src, csproj simplication and clean-up
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Don't use var
* Fix the S.N.Tensors readme and remove the file marking it as non-shipping
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
* Start vectorizing TensorPrimitives (#91596)
* Start vectorizing TensorPrimitives
Just does two functions to establish the files into which the rest of the implementations can be moved.
* 6 more naive methods for Tensor Primitives. (#92142)
* 6 more naive methods
* updates from pr comments
* Add remaining set of TensorPrimitives APIs for .NET 8 (#92154)
* Add remaining set of TensorPrimitives APIs for .NET 8
Adds non-vectorized implementations of:
- Max
- Min
- MaxMagnitude
- MinMagnitude
- IndexOfMax
- IndexOfMin
- IndexOfMaxMagnitude
- ConvertToHalf (only on .NET Core)
- ConvertToSingle (only on .NET Core)
- IndexOfMinMagnitude
Adds vectorized implementations of:
- Sum
- SumOfSquares
- SumOfMagnitudes
- Product
- ProductOfSums
- ProductOfDifferences
Also includes the helpers that'll make it trivial to vectorize Dot.
Beyond vectorizing the non-vectorized ones, the vectorized implementations should be improved further, including:
- Handling alignment better
- Vectorizing the remainder that doesn't fit in a vector rather than falling back to scalar
* Cleanup after previous PR, vectorize CosineSimilarity/Dot/L2Normalize/Distance, add tests
* Address PR feedback, and fix a few other issues
* Fix TensorPrimitives.CosineSimilarity to use vectorized implementations (#92204)
* Fixed duplicated code from merge.
* New Microsoft.BCL.Numerics package (#91074)
* bcl numberics library added
* bcl done
* added explicit 2.1 target
* Minor doc updates
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* fixes from PR comments
* minor csproj fixes
* fixed ref target frameworks
* minor ref csproj updates
* minor csproj updates
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource. (#91402)
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource.
* Use NetCoreAppCurrent.
* Try fix CI test failures.
---------
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
Oct 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jeffhandleyjeffhandleyjeffhandley left review comments
@stephentoubstephentoubstephentoub approved these changes
@tannergoodingtannergoodingtannergooding left review comments
@ericstjericstjericstj approved these changes

+1 more reviewer

@gfoidlgfoidlgfoidl left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@michaelgsharpmichaelgsharp

Labels

area-System.Numerics.Tensorsnew-api-needs-documentation

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@michaelgsharp@jeffhandley@stephentoub@gfoidl@ericstj@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
6 more naive methods for Tensor Primitives. by michaelgsharp · Pull Request #92142 · dotnet/runtime · GitHub
Skip to content
6 more naive methods for Tensor Primitives. - #92142
Merged
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Sep 16, 2023
Merged
6 more naive methods for Tensor Primitives.#92142
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Sep 15, 2023

Copy link


Copy Markdown

Contributor
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.
Issue Details
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
Author:michaelgsharp
Assignees:michaelgsharp
Labels:
area-System.Numerics.Tensors, new-api-needs-documentation
Milestone:-
@stephentoubstephentoub
mentioned this pull request
Sep 15, 2023
 Merged

gfoidl
gfoidl
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

ericstj
ericstj
approved these changes
Sep 16, 2023
View reviewed changes
@ericstjericstj
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
stephentoub
stephentoub
approved these changes
Sep 16, 2023
View reviewed changes
@stephentoubstephentoub
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this and then fix up a few things after rebasing my PR on it.
@stephentoub
stephentoub
merged commit 7be3913
into
dotnet:mainSep 16, 2023
jeffhandley
jeffhandley
reviewed
Sep 16, 2023
View reviewed changes
@jeffhandleyjeffhandley
left a comment
•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Echoing the existing feedback about tests, we could add a lot more tests for all these methods to capture edge/corner cases and more exceptional data too.
I meant to say, but we can add more tests after the RC2 milestone/snap
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 {
 ThrowHelper.ThrowArgument_SpansMustHaveSameLength();
 }
 if (x.Length == 0 || y.Length == 0)
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need one of the checks here since we've already determined the lengths are equal.
That being said, is it desirable to throw here rather than allowing it to just return NaN?
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 /// </summary>
 /// <param name="x">The first tensor, represented as a span.</param>
 /// <returns>The L2 norm.</returns>
 public static float L2Normalize(ReadOnlySpan<float> x) // BLAS1: nrm2
@tannergoodingtannergoodingSep 18, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this L2Normalize? The standard API computes the "euclidean norm", that is the length or magnitude of the vector.
The name should likely just be Norm (or Length which is what it's called for Vector2/3/4.Length())
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

 for (int i = 0; i < x.Length; i++)
 {
 expSum += MathF.Pow((float)Math.E, x[i]);
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why MathF.Pow and not MathF.Exp?
Why (float)Math.E and not MathF.E?
@stephentoubstephentoubSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tannergooding, can you submit a cleanup PR with the changes you think should be made? Thanks.
michaelgsharp added a commit
to michaelgsharp/runtime
that referenced
this pull request
Sep 18, 2023
@michaelgsharp
6 more naive methods for Tensor Primitives. (dotnet#92142)
96ba8c1
* 6 more naive methods
* updates from pr comments
ericstj pushed a commit
that referenced
this pull request
Sep 19, 2023
@michaelgsharp@tannergooding
@ViktorHofer@stephentoub@tmds
[release/8.0] Numerics and Tensors backport (#92245)
4b7c754
* added Bcl.Numerics
* Adding a naive implementation of various primitive tensor operations (#91228)
* Adding a naive implementation of various primitive tensor operations
* Adding tests covering the new tensor primitives APIs
* Adding tensor primitives APIs to the ref assembly
* Allow .NET Framework to build/run
* Sync TFMs between ref and src, csproj simplication and clean-up
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Don't use var
* Fix the S.N.Tensors readme and remove the file marking it as non-shipping
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
* Start vectorizing TensorPrimitives (#91596)
* Start vectorizing TensorPrimitives
Just does two functions to establish the files into which the rest of the implementations can be moved.
* 6 more naive methods for Tensor Primitives. (#92142)
* 6 more naive methods
* updates from pr comments
* Add remaining set of TensorPrimitives APIs for .NET 8 (#92154)
* Add remaining set of TensorPrimitives APIs for .NET 8
Adds non-vectorized implementations of:
- Max
- Min
- MaxMagnitude
- MinMagnitude
- IndexOfMax
- IndexOfMin
- IndexOfMaxMagnitude
- ConvertToHalf (only on .NET Core)
- ConvertToSingle (only on .NET Core)
- IndexOfMinMagnitude
Adds vectorized implementations of:
- Sum
- SumOfSquares
- SumOfMagnitudes
- Product
- ProductOfSums
- ProductOfDifferences
Also includes the helpers that'll make it trivial to vectorize Dot.
Beyond vectorizing the non-vectorized ones, the vectorized implementations should be improved further, including:
- Handling alignment better
- Vectorizing the remainder that doesn't fit in a vector rather than falling back to scalar
* Cleanup after previous PR, vectorize CosineSimilarity/Dot/L2Normalize/Distance, add tests
* Address PR feedback, and fix a few other issues
* Fix TensorPrimitives.CosineSimilarity to use vectorized implementations (#92204)
* Fixed duplicated code from merge.
* New Microsoft.BCL.Numerics package (#91074)
* bcl numberics library added
* bcl done
* added explicit 2.1 target
* Minor doc updates
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* fixes from PR comments
* minor csproj fixes
* fixed ref target frameworks
* minor ref csproj updates
* minor csproj updates
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource. (#91402)
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource.
* Use NetCoreAppCurrent.
* Try fix CI test failures.
---------
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
Oct 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jeffhandleyjeffhandleyjeffhandley left review comments
@stephentoubstephentoubstephentoub approved these changes
@tannergoodingtannergoodingtannergooding left review comments
@ericstjericstjericstj approved these changes

+1 more reviewer

@gfoidlgfoidlgfoidl left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@michaelgsharpmichaelgsharp

Labels

area-System.Numerics.Tensorsnew-api-needs-documentation

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@michaelgsharp@jeffhandley@stephentoub@gfoidl@ericstj@tannergooding

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
6 more naive methods for Tensor Primitives. by michaelgsharp · Pull Request #92142 · dotnet/runtime · GitHub
Skip to content
6 more naive methods for Tensor Primitives. - #92142
Merged
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Sep 16, 2023
Merged
6 more naive methods for Tensor Primitives.#92142
stephentoub merged 2 commits into
dotnet:mainfrom 
michaelgsharp:finish-naive-tensor
Conversation
@michaelgsharp
@michaelgsharpmichaelgsharp
commented
Sep 15, 2023

Copy link


Copy Markdown

Contributor
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Note regarding the new-api-needs-documentation label:
This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.
@ghost
ghost
commented
Sep 15, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-numerics-tensors

See info in area-owners.md if you want to be subscribed.
Issue Details
Added in naive implementations for
  • CosineSimilarty
  • Distance
  • Dot
  • Normalize
  • SoftMax
  • Sigmoid
@tannergooding@stephentoub@jeffhandley should we rename normalize as discussed since its not really the operation we are doing?
Author:michaelgsharp
Assignees:michaelgsharp
Labels:
area-System.Numerics.Tensors, new-api-needs-documentation
Milestone:-
@stephentoubstephentoub
mentioned this pull request
Sep 15, 2023
 Merged

gfoidl
gfoidl
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs
stephentoub
stephentoub
reviewed
Sep 15, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
Outdated

ericstj
ericstj
approved these changes
Sep 16, 2023
View reviewed changes
@ericstjericstj
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
stephentoub
stephentoub
approved these changes
Sep 16, 2023
View reviewed changes
@stephentoubstephentoub
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this and then fix up a few things after rebasing my PR on it.
@stephentoub
stephentoub
merged commit 7be3913
into
dotnet:mainSep 16, 2023
jeffhandley
jeffhandley
reviewed
Sep 16, 2023
View reviewed changes
@jeffhandleyjeffhandley
left a comment
•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Echoing the existing feedback about tests, we could add a lot more tests for all these methods to capture edge/corner cases and more exceptional data too.
I meant to say, but we can add more tests after the RC2 milestone/snap
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 {
 ThrowHelper.ThrowArgument_SpansMustHaveSameLength();
 }
 if (x.Length == 0 || y.Length == 0)
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need one of the checks here since we've already determined the lengths are equal.
That being said, is it desirable to throw here rather than allowing it to just return NaN?
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs
 /// </summary>
 /// <param name="x">The first tensor, represented as a span.</param>
 /// <returns>The L2 norm.</returns>
 public static float L2Normalize(ReadOnlySpan<float> x) // BLAS1: nrm2
@tannergoodingtannergoodingSep 18, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this L2Normalize? The standard API computes the "euclidean norm", that is the length or magnitude of the vector.
The name should likely just be Norm (or Length which is what it's called for Vector2/3/4.Length())
tannergooding
tannergooding
reviewed
Sep 18, 2023
View reviewed changes
Comment threadsrc/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs

 for (int i = 0; i < x.Length; i++)
 {
 expSum += MathF.Pow((float)Math.E, x[i]);
@tannergoodingtannergoodingSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why MathF.Pow and not MathF.Exp?
Why (float)Math.E and not MathF.E?
@stephentoubstephentoubSep 18, 2023

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tannergooding, can you submit a cleanup PR with the changes you think should be made? Thanks.
michaelgsharp added a commit
to michaelgsharp/runtime
that referenced
this pull request
Sep 18, 2023
@michaelgsharp
6 more naive methods for Tensor Primitives. (dotnet#92142)
96ba8c1
* 6 more naive methods
* updates from pr comments
ericstj pushed a commit
that referenced
this pull request
Sep 19, 2023
@michaelgsharp@tannergooding
@ViktorHofer@stephentoub@tmds
[release/8.0] Numerics and Tensors backport (#92245)
4b7c754
* added Bcl.Numerics
* Adding a naive implementation of various primitive tensor operations (#91228)
* Adding a naive implementation of various primitive tensor operations
* Adding tests covering the new tensor primitives APIs
* Adding tensor primitives APIs to the ref assembly
* Allow .NET Framework to build/run
* Sync TFMs between ref and src, csproj simplication and clean-up
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Don't use var
* Fix the S.N.Tensors readme and remove the file marking it as non-shipping
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
* Start vectorizing TensorPrimitives (#91596)
* Start vectorizing TensorPrimitives
Just does two functions to establish the files into which the rest of the implementations can be moved.
* 6 more naive methods for Tensor Primitives. (#92142)
* 6 more naive methods
* updates from pr comments
* Add remaining set of TensorPrimitives APIs for .NET 8 (#92154)
* Add remaining set of TensorPrimitives APIs for .NET 8
Adds non-vectorized implementations of:
- Max
- Min
- MaxMagnitude
- MinMagnitude
- IndexOfMax
- IndexOfMin
- IndexOfMaxMagnitude
- ConvertToHalf (only on .NET Core)
- ConvertToSingle (only on .NET Core)
- IndexOfMinMagnitude
Adds vectorized implementations of:
- Sum
- SumOfSquares
- SumOfMagnitudes
- Product
- ProductOfSums
- ProductOfDifferences
Also includes the helpers that'll make it trivial to vectorize Dot.
Beyond vectorizing the non-vectorized ones, the vectorized implementations should be improved further, including:
- Handling alignment better
- Vectorizing the remainder that doesn't fit in a vector rather than falling back to scalar
* Cleanup after previous PR, vectorize CosineSimilarity/Dot/L2Normalize/Distance, add tests
* Address PR feedback, and fix a few other issues
* Fix TensorPrimitives.CosineSimilarity to use vectorized implementations (#92204)
* Fixed duplicated code from merge.
* New Microsoft.BCL.Numerics package (#91074)
* bcl numberics library added
* bcl done
* added explicit 2.1 target
* Minor doc updates
* Apply suggestions from code review
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* fixes from PR comments
* minor csproj fixes
* fixed ref target frameworks
* minor ref csproj updates
* minor csproj updates
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource. (#91402)
* Microsoft.Bcl.Numerics.Tests: fix restore failure when DotNetBuildFromSource.
* Use NetCoreAppCurrent.
* Try fix CI test failures.
---------
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
@ghostghost
locked as resolved and limited conversation to collaborators
Oct 18, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@jeffhandleyjeffhandleyjeffhandley left review comments
@stephentoubstephentoubstephentoub approved these changes
@tannergoodingtannergoodingtannergooding left review comments
@ericstjericstjericstj approved these changes

+1 more reviewer

@gfoidlgfoidlgfoidl left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@michaelgsharpmichaelgsharp

Labels

area-System.Numerics.Tensorsnew-api-needs-documentation

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@michaelgsharp@jeffhandley@stephentoub@gfoidl@ericstj@tannergooding