Fix creation of dataviews inferred with .NET types with sparse vectors - #587

Merged
TomFinley merged 15 commits into
dotnet:masterfrom
xadupre:master
Jul 28, 2018
Merged

Fix creation of dataviews inferred with .NET types with sparse vectors#587
TomFinley merged 15 commits into
dotnet:masterfrom
xadupre:master

Conversation

@xadupre

@xaduprexadupre commented Jul 27, 2018•
edited by TomFinley
Loading

Copy link
Copy Markdown
Contributor

Fixes#586. DataViewConstructionUtils's methods to create dataviews over .NET types will now have correctly inferred getters in the case of sparse vectors.

sdpythonand others added 14 commits May 18, 2018 19:31
@sdpython
Compile CpuMathNative and FastTreeNative with charset=utf-8, fix for …
edba472
…issue dotnet#78
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5df6fba
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ca76231
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ea730aa
@sdpython
add method AddSerialize to declare entrypoints outside ML.net
95e3646
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
96cae17
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
e55882d
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
719adc4
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
0255942
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5d55dfa
@sdpython
add a unit test to check the creation of a cursor on sparse vectors
d5a419e
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
11aea96
@sdpython
fixdotnet#586and cursor creation on dataview with sparse vectors
3409b9f
@sdpython
revert changes not related todotnet#586
63fa647
@dnfclas

dnfclas commented Jul 27, 2018•
edited
Loading

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

TomFinley
TomFinley reviewed Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs Outdated
Done();
}

void GenericDenseDataView<T>(T[] v1, T[] v2)

@TomFinleyTomFinleyJul 27, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void [](start = 8, length = 4)

Explicit access modifier please. Here and everywhere.

TomFinley
TomFinley reviewed Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs Outdated

namespace Microsoft.ML.Runtime.RunTests
{
public sealed partial class TestSparseDataView : TestDataViewBase

@TomFinleyTomFinleyJul 27, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial [](start = 18, length = 7)

Why partial?

@xaduprexadupreJul 27, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste without thinking :)

@TomFinley

TomFinley commented Jul 27, 2018

Copy link
Copy Markdown
Contributor

Hi @xadupre thanks for adding this fix plus the extensive tests. Right now the PR description is just the default description. Perhaps something real could be written there. If we were to merge right now it would be very hard to come up with an adequate commit message. (Complicated also by the fact that this PR appears to be based on merges rather than rebases.)

Also if you write something like "fixes #586" in your description, then it will associate the issue and the PR, and close the issue when the PR is merged. You put the thing in the title, but if you check the issue page it was not actually associated. (Well, it will be now since I just mentioned it, but of course as the PR author it should be in the head description, rather than some random reviewer mentioning it in a comment far down the page.

TomFinley
TomFinley reviewed Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs Outdated
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Command;

@TomFinleyTomFinleyJul 27, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain not all of these are used.

Ivanidzo4ka
Ivanidzo4ka reviewed Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs Outdated
new SparseExample<T>() { X = new VBuffer<T> (5, 3, v1, new int[] { 0, 2, 4 }) },
new SparseExample<T>() { X = new VBuffer<T> (5, 3, v2, new int[] { 0, 1, 3 }) }
};
var host = new TlcEnvironment();

@Ivanidzo4kaIvanidzo4kaJul 27, 2018•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var host = new TlcEnvironment(); [](start = 12, length = 32)

can you wrap it in using? #Resolved

Ivanidzo4ka
Ivanidzo4ka reviewed Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs Outdated
GenericSparseDataView(new double[] { 1, 2, 3 }, new double[] { 1, 10, 100 });
GenericSparseDataView(new DvText[] { new DvText("a"), new DvText("b"), new DvText("c") },
new DvText[] { new DvText("aa"), new DvText("bb"), new DvText("cc") });
Done();

@Ivanidzo4kaIvanidzo4kaJul 27, 2018•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done(); [](start = 12, length = 7)

You don't have to call Done if you don't do any basefiles comparison #Resolved

Ivanidzo4ka
Ivanidzo4ka reviewed Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Assert.True(n == 2);
}

[Fact]

@Ivanidzo4kaIvanidzo4kaJul 27, 2018•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fact [](start = 9, length = 4)

Thank you for fixing my mess!
Your code definitely do right thing, and it's nice to have tests, but I want to point one thing. Your tests never hit TypedCursor changes.
There is two conversions: one from some collection to IDataview through code in DataViewConstructionUltis, and from IDataVIew to Collection via TypedCursor and your tests cover only first case.

In order to invoke second case you need to do something like:
dataView.AsEnumerable(env, false).GetEnumerator() and enumerate over it.
#Resolved

@xaduprexadupreJul 27, 2018•
edited by Ivanidzo4ka
Loading

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do it. #Resolved

@xaduprexadupre changed the title Fix the creation of getters on dataviews having sparse vectors (#586)Fixes #586, the creation of getters on dataviews having sparse vectorsJul 27, 2018
Ivanidzo4ka
Ivanidzo4ka approved these changes Jul 27, 2018
View reviewed changes

@Ivanidzo4kaIvanidzo4ka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@TomFinley

TomFinley commented Jul 28, 2018

Copy link
Copy Markdown
Contributor

Hi @xadupre thanks for addressing the comment! Clarifying note for the future, when I asked to modify the description, I meant the PR's description, not the PR's title. The description is still the little "we are excited to review your PR" built in text that we intend for people to write. I hope you don't mind, just going to change it so we can get this in since otherwise it looks good.

@TomFinleyTomFinley changed the title Fixes #586, the creation of getters on dataviews having sparse vectorsFix creation of dataviews inferred with .NET types with sparse vectorsJul 28, 2018
TomFinley
TomFinley approved these changes Jul 28, 2018
View reviewed changes

@TomFinleyTomFinley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xadupre

@TomFinley
TomFinley merged commit 2107b82 into dotnet:masterJul 28, 2018
@xadupre

xadupre commented Jul 30, 2018 via email

Copy link
Copy Markdown
ContributorAuthor

codemzs pushed a commit to codemzs/machinelearning that referenced this pull request Aug 1, 2018
@xadupre@codemzs
Fix creation of dataviews inferred with .NET types with sparse vectors (
760cef2
dotnet#587)
`DataViewConstructionUtils`'s methods to create dataviews over .NET types will now have correctly inferred "getters" in the case of sparse vectors.
@ghostghost locked as resolved and limited conversation to collaborators Mar 29, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers
@Ivanidzo4kaIvanidzo4kaIvanidzo4ka approved these changes
@TomFinleyTomFinleyTomFinley approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@xadupre@dnfclas@TomFinley@Ivanidzo4ka@sdpython
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Fix creation of dataviews inferred with .NET types with sparse vectors - #587
Merged
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Jul 28, 2018
Merged
Fix creation of dataviews inferred with .NET types with sparse vectors#587
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Conversation
@xadupre
@xaduprexadupre
commented
Jul 27, 2018•

edited by TomFinley

Loading

Copy link


Copy Markdown

Contributor
Fixes#586. DataViewConstructionUtils's methods to create dataviews over .NET types will now have correctly inferred getters in the case of sparse vectors.
sdpythonand others
added 14 commits
May 18, 2018 19:31
@sdpython
Compile CpuMathNative and FastTreeNative with charset=utf-8, fix for …
edba472
…issue dotnet#78
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5df6fba
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ca76231
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ea730aa
@sdpython
add method AddSerialize to declare entrypoints outside ML.net
95e3646
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
96cae17
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
e55882d
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
719adc4
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
0255942
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5d55dfa
@sdpython
add a unit test to check the creation of a cursor on sparse vectors
d5a419e
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
11aea96
@sdpython
fixdotnet#586and cursor creation on dataview with sparse vectors
3409b9f
@sdpython
revert changes not related todotnet#586
63fa647
@dnfclas
dnfclas
commented
Jul 27, 2018•

edited

Loading

Copy link


Copy Markdown

CLA assistant check
All CLA requirements met.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 Done();
 }

 void GenericDenseDataView<T>(T[] v1, T[] v2)
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void [](start = 8, length = 4)
Explicit access modifier please. Here and everywhere.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated


namespace Microsoft.ML.Runtime.RunTests
{
 public sealed partial class TestSparseDataView : TestDataViewBase
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial [](start = 18, length = 7)
Why partial?
@xaduprexadupreJul 27, 2018

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste without thinking :)
@TomFinley
TomFinley
commented
Jul 27, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for adding this fix plus the extensive tests. Right now the PR description is just the default description. Perhaps something real could be written there. If we were to merge right now it would be very hard to come up with an adequate commit message. (Complicated also by the fact that this PR appears to be based on merges rather than rebases.)
Also if you write something like "fixes #586" in your description, then it will associate the issue and the PR, and close the issue when the PR is merged. You put the thing in the title, but if you check the issue page it was not actually associated. (Well, it will be now since I just mentioned it, but of course as the PR author it should be in the head description, rather than some random reviewer mentioning it in a comment far down the page.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Command;
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain not all of these are used.
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v1, new int[] { 0, 2, 4 }) },
 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v2, new int[] { 0, 1, 3 }) }
 };
 var host = new TlcEnvironment();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var host = new TlcEnvironment(); [](start = 12, length = 32)
can you wrap it in using? #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 GenericSparseDataView(new double[] { 1, 2, 3 }, new double[] { 1, 10, 100 });
 GenericSparseDataView(new DvText[] { new DvText("a"), new DvText("b"), new DvText("c") },
 new DvText[] { new DvText("aa"), new DvText("bb"), new DvText("cc") });
 Done();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done(); [](start = 12, length = 7)
You don't have to call Done if you don't do any basefiles comparison #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
 Assert.True(n == 2);
 }

 [Fact]
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fact [](start = 9, length = 4)
Thank you for fixing my mess!

Your code definitely do right thing, and it's nice to have tests, but I want to point one thing. Your tests never hit TypedCursor changes.

There is two conversions: one from some collection to IDataview through code in DataViewConstructionUltis, and from IDataVIew to Collection via TypedCursor and your tests cover only first case.
In order to invoke second case you need to do something like:

dataView.AsEnumerable(env, false).GetEnumerator() and enumerate over it.

#Resolved
@xaduprexadupreJul 27, 2018•

edited by Ivanidzo4ka

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do it. #Resolved
@xaduprexadupre
changed the title
Fix the creation of getters on dataviews having sparse vectors (#586)Fixes #586, the creation of getters on dataviews having sparse vectorsJul 27, 2018
Ivanidzo4ka
Ivanidzo4ka
approved these changes
Jul 27, 2018
View reviewed changes
@Ivanidzo4kaIvanidzo4ka
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@TomFinley
TomFinley
commented
Jul 28, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for addressing the comment! Clarifying note for the future, when I asked to modify the description, I meant the PR's description, not the PR's title. The description is still the little "we are excited to review your PR" built in text that we intend for people to write. I hope you don't mind, just going to change it so we can get this in since otherwise it looks good.
@TomFinleyTomFinley
changed the title
Fixes #586, the creation of getters on dataviews having sparse vectorsFix creation of dataviews inferred with .NET types with sparse vectorsJul 28, 2018
TomFinley
TomFinley
approved these changes
Jul 28, 2018
View reviewed changes
@TomFinleyTomFinley
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xadupre
@TomFinley
TomFinley
merged commit 2107b82
into
dotnet:masterJul 28, 2018
@xadupre
xadupre
commented
Jul 30, 2018
via email 

Copy link


Copy Markdown

ContributorAuthor
codemzs pushed a commit
to codemzs/machinelearning
that referenced
this pull request
Aug 1, 2018
@xadupre@codemzs
Fix creation of dataviews inferred with .NET types with sparse vectors (
760cef2
dotnet#587)
`DataViewConstructionUtils`'s methods to create dataviews over .NET types will now have correctly inferred "getters" in the case of sparse vectors.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 29, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@Ivanidzo4kaIvanidzo4kaIvanidzo4ka approved these changes
@TomFinleyTomFinleyTomFinley approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@xadupre@dnfclas@TomFinley@Ivanidzo4ka@sdpython

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Fix creation of dataviews inferred with .NET types with sparse vectors - #587
Merged
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Jul 28, 2018
Merged
Fix creation of dataviews inferred with .NET types with sparse vectors#587
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Conversation
@xadupre
@xaduprexadupre
commented
Jul 27, 2018•

edited by TomFinley

Loading

Copy link


Copy Markdown

Contributor
Fixes#586. DataViewConstructionUtils's methods to create dataviews over .NET types will now have correctly inferred getters in the case of sparse vectors.
sdpythonand others
added 14 commits
May 18, 2018 19:31
@sdpython
Compile CpuMathNative and FastTreeNative with charset=utf-8, fix for …
edba472
…issue dotnet#78
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5df6fba
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ca76231
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ea730aa
@sdpython
add method AddSerialize to declare entrypoints outside ML.net
95e3646
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
96cae17
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
e55882d
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
719adc4
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
0255942
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5d55dfa
@sdpython
add a unit test to check the creation of a cursor on sparse vectors
d5a419e
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
11aea96
@sdpython
fixdotnet#586and cursor creation on dataview with sparse vectors
3409b9f
@sdpython
revert changes not related todotnet#586
63fa647
@dnfclas
dnfclas
commented
Jul 27, 2018•

edited

Loading

Copy link


Copy Markdown

CLA assistant check
All CLA requirements met.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 Done();
 }

 void GenericDenseDataView<T>(T[] v1, T[] v2)
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void [](start = 8, length = 4)
Explicit access modifier please. Here and everywhere.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated


namespace Microsoft.ML.Runtime.RunTests
{
 public sealed partial class TestSparseDataView : TestDataViewBase
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial [](start = 18, length = 7)
Why partial?
@xaduprexadupreJul 27, 2018

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste without thinking :)
@TomFinley
TomFinley
commented
Jul 27, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for adding this fix plus the extensive tests. Right now the PR description is just the default description. Perhaps something real could be written there. If we were to merge right now it would be very hard to come up with an adequate commit message. (Complicated also by the fact that this PR appears to be based on merges rather than rebases.)
Also if you write something like "fixes #586" in your description, then it will associate the issue and the PR, and close the issue when the PR is merged. You put the thing in the title, but if you check the issue page it was not actually associated. (Well, it will be now since I just mentioned it, but of course as the PR author it should be in the head description, rather than some random reviewer mentioning it in a comment far down the page.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Command;
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain not all of these are used.
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v1, new int[] { 0, 2, 4 }) },
 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v2, new int[] { 0, 1, 3 }) }
 };
 var host = new TlcEnvironment();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var host = new TlcEnvironment(); [](start = 12, length = 32)
can you wrap it in using? #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 GenericSparseDataView(new double[] { 1, 2, 3 }, new double[] { 1, 10, 100 });
 GenericSparseDataView(new DvText[] { new DvText("a"), new DvText("b"), new DvText("c") },
 new DvText[] { new DvText("aa"), new DvText("bb"), new DvText("cc") });
 Done();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done(); [](start = 12, length = 7)
You don't have to call Done if you don't do any basefiles comparison #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
 Assert.True(n == 2);
 }

 [Fact]
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fact [](start = 9, length = 4)
Thank you for fixing my mess!

Your code definitely do right thing, and it's nice to have tests, but I want to point one thing. Your tests never hit TypedCursor changes.

There is two conversions: one from some collection to IDataview through code in DataViewConstructionUltis, and from IDataVIew to Collection via TypedCursor and your tests cover only first case.
In order to invoke second case you need to do something like:

dataView.AsEnumerable(env, false).GetEnumerator() and enumerate over it.

#Resolved
@xaduprexadupreJul 27, 2018•

edited by Ivanidzo4ka

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do it. #Resolved
@xaduprexadupre
changed the title
Fix the creation of getters on dataviews having sparse vectors (#586)Fixes #586, the creation of getters on dataviews having sparse vectorsJul 27, 2018
Ivanidzo4ka
Ivanidzo4ka
approved these changes
Jul 27, 2018
View reviewed changes
@Ivanidzo4kaIvanidzo4ka
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@TomFinley
TomFinley
commented
Jul 28, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for addressing the comment! Clarifying note for the future, when I asked to modify the description, I meant the PR's description, not the PR's title. The description is still the little "we are excited to review your PR" built in text that we intend for people to write. I hope you don't mind, just going to change it so we can get this in since otherwise it looks good.
@TomFinleyTomFinley
changed the title
Fixes #586, the creation of getters on dataviews having sparse vectorsFix creation of dataviews inferred with .NET types with sparse vectorsJul 28, 2018
TomFinley
TomFinley
approved these changes
Jul 28, 2018
View reviewed changes
@TomFinleyTomFinley
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xadupre
@TomFinley
TomFinley
merged commit 2107b82
into
dotnet:masterJul 28, 2018
@xadupre
xadupre
commented
Jul 30, 2018
via email 

Copy link


Copy Markdown

ContributorAuthor
codemzs pushed a commit
to codemzs/machinelearning
that referenced
this pull request
Aug 1, 2018
@xadupre@codemzs
Fix creation of dataviews inferred with .NET types with sparse vectors (
760cef2
dotnet#587)
`DataViewConstructionUtils`'s methods to create dataviews over .NET types will now have correctly inferred "getters" in the case of sparse vectors.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 29, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@Ivanidzo4kaIvanidzo4kaIvanidzo4ka approved these changes
@TomFinleyTomFinleyTomFinley approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@xadupre@dnfclas@TomFinley@Ivanidzo4ka@sdpython

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Fix creation of dataviews inferred with .NET types with sparse vectors - #587
Merged
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Jul 28, 2018
Merged
Fix creation of dataviews inferred with .NET types with sparse vectors#587
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Conversation
@xadupre
@xaduprexadupre
commented
Jul 27, 2018•

edited by TomFinley

Loading

Copy link


Copy Markdown

Contributor
Fixes#586. DataViewConstructionUtils's methods to create dataviews over .NET types will now have correctly inferred getters in the case of sparse vectors.
sdpythonand others
added 14 commits
May 18, 2018 19:31
@sdpython
Compile CpuMathNative and FastTreeNative with charset=utf-8, fix for …
edba472
…issue dotnet#78
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5df6fba
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ca76231
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ea730aa
@sdpython
add method AddSerialize to declare entrypoints outside ML.net
95e3646
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
96cae17
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
e55882d
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
719adc4
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
0255942
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5d55dfa
@sdpython
add a unit test to check the creation of a cursor on sparse vectors
d5a419e
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
11aea96
@sdpython
fixdotnet#586and cursor creation on dataview with sparse vectors
3409b9f
@sdpython
revert changes not related todotnet#586
63fa647
@dnfclas
dnfclas
commented
Jul 27, 2018•

edited

Loading

Copy link


Copy Markdown

CLA assistant check
All CLA requirements met.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 Done();
 }

 void GenericDenseDataView<T>(T[] v1, T[] v2)
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void [](start = 8, length = 4)
Explicit access modifier please. Here and everywhere.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated


namespace Microsoft.ML.Runtime.RunTests
{
 public sealed partial class TestSparseDataView : TestDataViewBase
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial [](start = 18, length = 7)
Why partial?
@xaduprexadupreJul 27, 2018

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste without thinking :)
@TomFinley
TomFinley
commented
Jul 27, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for adding this fix plus the extensive tests. Right now the PR description is just the default description. Perhaps something real could be written there. If we were to merge right now it would be very hard to come up with an adequate commit message. (Complicated also by the fact that this PR appears to be based on merges rather than rebases.)
Also if you write something like "fixes #586" in your description, then it will associate the issue and the PR, and close the issue when the PR is merged. You put the thing in the title, but if you check the issue page it was not actually associated. (Well, it will be now since I just mentioned it, but of course as the PR author it should be in the head description, rather than some random reviewer mentioning it in a comment far down the page.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Command;
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain not all of these are used.
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v1, new int[] { 0, 2, 4 }) },
 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v2, new int[] { 0, 1, 3 }) }
 };
 var host = new TlcEnvironment();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var host = new TlcEnvironment(); [](start = 12, length = 32)
can you wrap it in using? #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 GenericSparseDataView(new double[] { 1, 2, 3 }, new double[] { 1, 10, 100 });
 GenericSparseDataView(new DvText[] { new DvText("a"), new DvText("b"), new DvText("c") },
 new DvText[] { new DvText("aa"), new DvText("bb"), new DvText("cc") });
 Done();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done(); [](start = 12, length = 7)
You don't have to call Done if you don't do any basefiles comparison #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
 Assert.True(n == 2);
 }

 [Fact]
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fact [](start = 9, length = 4)
Thank you for fixing my mess!

Your code definitely do right thing, and it's nice to have tests, but I want to point one thing. Your tests never hit TypedCursor changes.

There is two conversions: one from some collection to IDataview through code in DataViewConstructionUltis, and from IDataVIew to Collection via TypedCursor and your tests cover only first case.
In order to invoke second case you need to do something like:

dataView.AsEnumerable(env, false).GetEnumerator() and enumerate over it.

#Resolved
@xaduprexadupreJul 27, 2018•

edited by Ivanidzo4ka

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do it. #Resolved
@xaduprexadupre
changed the title
Fix the creation of getters on dataviews having sparse vectors (#586)Fixes #586, the creation of getters on dataviews having sparse vectorsJul 27, 2018
Ivanidzo4ka
Ivanidzo4ka
approved these changes
Jul 27, 2018
View reviewed changes
@Ivanidzo4kaIvanidzo4ka
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@TomFinley
TomFinley
commented
Jul 28, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for addressing the comment! Clarifying note for the future, when I asked to modify the description, I meant the PR's description, not the PR's title. The description is still the little "we are excited to review your PR" built in text that we intend for people to write. I hope you don't mind, just going to change it so we can get this in since otherwise it looks good.
@TomFinleyTomFinley
changed the title
Fixes #586, the creation of getters on dataviews having sparse vectorsFix creation of dataviews inferred with .NET types with sparse vectorsJul 28, 2018
TomFinley
TomFinley
approved these changes
Jul 28, 2018
View reviewed changes
@TomFinleyTomFinley
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xadupre
@TomFinley
TomFinley
merged commit 2107b82
into
dotnet:masterJul 28, 2018
@xadupre
xadupre
commented
Jul 30, 2018
via email 

Copy link


Copy Markdown

ContributorAuthor
codemzs pushed a commit
to codemzs/machinelearning
that referenced
this pull request
Aug 1, 2018
@xadupre@codemzs
Fix creation of dataviews inferred with .NET types with sparse vectors (
760cef2
dotnet#587)
`DataViewConstructionUtils`'s methods to create dataviews over .NET types will now have correctly inferred "getters" in the case of sparse vectors.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 29, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@Ivanidzo4kaIvanidzo4kaIvanidzo4ka approved these changes
@TomFinleyTomFinleyTomFinley approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@xadupre@dnfclas@TomFinley@Ivanidzo4ka@sdpython

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Fix creation of dataviews inferred with .NET types with sparse vectors - #587
Merged
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Jul 28, 2018
Merged
Fix creation of dataviews inferred with .NET types with sparse vectors#587
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Conversation
@xadupre
@xaduprexadupre
commented
Jul 27, 2018•

edited by TomFinley

Loading

Copy link


Copy Markdown

Contributor
Fixes#586. DataViewConstructionUtils's methods to create dataviews over .NET types will now have correctly inferred getters in the case of sparse vectors.
sdpythonand others
added 14 commits
May 18, 2018 19:31
@sdpython
Compile CpuMathNative and FastTreeNative with charset=utf-8, fix for …
edba472
…issue dotnet#78
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5df6fba
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ca76231
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ea730aa
@sdpython
add method AddSerialize to declare entrypoints outside ML.net
95e3646
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
96cae17
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
e55882d
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
719adc4
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
0255942
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5d55dfa
@sdpython
add a unit test to check the creation of a cursor on sparse vectors
d5a419e
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
11aea96
@sdpython
fixdotnet#586and cursor creation on dataview with sparse vectors
3409b9f
@sdpython
revert changes not related todotnet#586
63fa647
@dnfclas
dnfclas
commented
Jul 27, 2018•

edited

Loading

Copy link


Copy Markdown

CLA assistant check
All CLA requirements met.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 Done();
 }

 void GenericDenseDataView<T>(T[] v1, T[] v2)
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void [](start = 8, length = 4)
Explicit access modifier please. Here and everywhere.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated


namespace Microsoft.ML.Runtime.RunTests
{
 public sealed partial class TestSparseDataView : TestDataViewBase
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial [](start = 18, length = 7)
Why partial?
@xaduprexadupreJul 27, 2018

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste without thinking :)
@TomFinley
TomFinley
commented
Jul 27, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for adding this fix plus the extensive tests. Right now the PR description is just the default description. Perhaps something real could be written there. If we were to merge right now it would be very hard to come up with an adequate commit message. (Complicated also by the fact that this PR appears to be based on merges rather than rebases.)
Also if you write something like "fixes #586" in your description, then it will associate the issue and the PR, and close the issue when the PR is merged. You put the thing in the title, but if you check the issue page it was not actually associated. (Well, it will be now since I just mentioned it, but of course as the PR author it should be in the head description, rather than some random reviewer mentioning it in a comment far down the page.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Command;
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain not all of these are used.
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v1, new int[] { 0, 2, 4 }) },
 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v2, new int[] { 0, 1, 3 }) }
 };
 var host = new TlcEnvironment();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var host = new TlcEnvironment(); [](start = 12, length = 32)
can you wrap it in using? #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 GenericSparseDataView(new double[] { 1, 2, 3 }, new double[] { 1, 10, 100 });
 GenericSparseDataView(new DvText[] { new DvText("a"), new DvText("b"), new DvText("c") },
 new DvText[] { new DvText("aa"), new DvText("bb"), new DvText("cc") });
 Done();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done(); [](start = 12, length = 7)
You don't have to call Done if you don't do any basefiles comparison #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
 Assert.True(n == 2);
 }

 [Fact]
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fact [](start = 9, length = 4)
Thank you for fixing my mess!

Your code definitely do right thing, and it's nice to have tests, but I want to point one thing. Your tests never hit TypedCursor changes.

There is two conversions: one from some collection to IDataview through code in DataViewConstructionUltis, and from IDataVIew to Collection via TypedCursor and your tests cover only first case.
In order to invoke second case you need to do something like:

dataView.AsEnumerable(env, false).GetEnumerator() and enumerate over it.

#Resolved
@xaduprexadupreJul 27, 2018•

edited by Ivanidzo4ka

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do it. #Resolved
@xaduprexadupre
changed the title
Fix the creation of getters on dataviews having sparse vectors (#586)Fixes #586, the creation of getters on dataviews having sparse vectorsJul 27, 2018
Ivanidzo4ka
Ivanidzo4ka
approved these changes
Jul 27, 2018
View reviewed changes
@Ivanidzo4kaIvanidzo4ka
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@TomFinley
TomFinley
commented
Jul 28, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for addressing the comment! Clarifying note for the future, when I asked to modify the description, I meant the PR's description, not the PR's title. The description is still the little "we are excited to review your PR" built in text that we intend for people to write. I hope you don't mind, just going to change it so we can get this in since otherwise it looks good.
@TomFinleyTomFinley
changed the title
Fixes #586, the creation of getters on dataviews having sparse vectorsFix creation of dataviews inferred with .NET types with sparse vectorsJul 28, 2018
TomFinley
TomFinley
approved these changes
Jul 28, 2018
View reviewed changes
@TomFinleyTomFinley
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xadupre
@TomFinley
TomFinley
merged commit 2107b82
into
dotnet:masterJul 28, 2018
@xadupre
xadupre
commented
Jul 30, 2018
via email 

Copy link


Copy Markdown

ContributorAuthor
codemzs pushed a commit
to codemzs/machinelearning
that referenced
this pull request
Aug 1, 2018
@xadupre@codemzs
Fix creation of dataviews inferred with .NET types with sparse vectors (
760cef2
dotnet#587)
`DataViewConstructionUtils`'s methods to create dataviews over .NET types will now have correctly inferred "getters" in the case of sparse vectors.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 29, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@Ivanidzo4kaIvanidzo4kaIvanidzo4ka approved these changes
@TomFinleyTomFinleyTomFinley approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@xadupre@dnfclas@TomFinley@Ivanidzo4ka@sdpython

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Fix creation of dataviews inferred with .NET types with sparse vectors - #587
Merged
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Jul 28, 2018
Merged
Fix creation of dataviews inferred with .NET types with sparse vectors#587
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Conversation
@xadupre
@xaduprexadupre
commented
Jul 27, 2018•

edited by TomFinley

Loading

Copy link


Copy Markdown

Contributor
Fixes#586. DataViewConstructionUtils's methods to create dataviews over .NET types will now have correctly inferred getters in the case of sparse vectors.
sdpythonand others
added 14 commits
May 18, 2018 19:31
@sdpython
Compile CpuMathNative and FastTreeNative with charset=utf-8, fix for …
edba472
…issue dotnet#78
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5df6fba
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ca76231
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ea730aa
@sdpython
add method AddSerialize to declare entrypoints outside ML.net
95e3646
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
96cae17
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
e55882d
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
719adc4
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
0255942
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5d55dfa
@sdpython
add a unit test to check the creation of a cursor on sparse vectors
d5a419e
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
11aea96
@sdpython
fixdotnet#586and cursor creation on dataview with sparse vectors
3409b9f
@sdpython
revert changes not related todotnet#586
63fa647
@dnfclas
dnfclas
commented
Jul 27, 2018•

edited

Loading

Copy link


Copy Markdown

CLA assistant check
All CLA requirements met.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 Done();
 }

 void GenericDenseDataView<T>(T[] v1, T[] v2)
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void [](start = 8, length = 4)
Explicit access modifier please. Here and everywhere.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated


namespace Microsoft.ML.Runtime.RunTests
{
 public sealed partial class TestSparseDataView : TestDataViewBase
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial [](start = 18, length = 7)
Why partial?
@xaduprexadupreJul 27, 2018

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste without thinking :)
@TomFinley
TomFinley
commented
Jul 27, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for adding this fix plus the extensive tests. Right now the PR description is just the default description. Perhaps something real could be written there. If we were to merge right now it would be very hard to come up with an adequate commit message. (Complicated also by the fact that this PR appears to be based on merges rather than rebases.)
Also if you write something like "fixes #586" in your description, then it will associate the issue and the PR, and close the issue when the PR is merged. You put the thing in the title, but if you check the issue page it was not actually associated. (Well, it will be now since I just mentioned it, but of course as the PR author it should be in the head description, rather than some random reviewer mentioning it in a comment far down the page.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Command;
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain not all of these are used.
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v1, new int[] { 0, 2, 4 }) },
 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v2, new int[] { 0, 1, 3 }) }
 };
 var host = new TlcEnvironment();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var host = new TlcEnvironment(); [](start = 12, length = 32)
can you wrap it in using? #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 GenericSparseDataView(new double[] { 1, 2, 3 }, new double[] { 1, 10, 100 });
 GenericSparseDataView(new DvText[] { new DvText("a"), new DvText("b"), new DvText("c") },
 new DvText[] { new DvText("aa"), new DvText("bb"), new DvText("cc") });
 Done();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done(); [](start = 12, length = 7)
You don't have to call Done if you don't do any basefiles comparison #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
 Assert.True(n == 2);
 }

 [Fact]
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fact [](start = 9, length = 4)
Thank you for fixing my mess!

Your code definitely do right thing, and it's nice to have tests, but I want to point one thing. Your tests never hit TypedCursor changes.

There is two conversions: one from some collection to IDataview through code in DataViewConstructionUltis, and from IDataVIew to Collection via TypedCursor and your tests cover only first case.
In order to invoke second case you need to do something like:

dataView.AsEnumerable(env, false).GetEnumerator() and enumerate over it.

#Resolved
@xaduprexadupreJul 27, 2018•

edited by Ivanidzo4ka

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do it. #Resolved
@xaduprexadupre
changed the title
Fix the creation of getters on dataviews having sparse vectors (#586)Fixes #586, the creation of getters on dataviews having sparse vectorsJul 27, 2018
Ivanidzo4ka
Ivanidzo4ka
approved these changes
Jul 27, 2018
View reviewed changes
@Ivanidzo4kaIvanidzo4ka
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@TomFinley
TomFinley
commented
Jul 28, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for addressing the comment! Clarifying note for the future, when I asked to modify the description, I meant the PR's description, not the PR's title. The description is still the little "we are excited to review your PR" built in text that we intend for people to write. I hope you don't mind, just going to change it so we can get this in since otherwise it looks good.
@TomFinleyTomFinley
changed the title
Fixes #586, the creation of getters on dataviews having sparse vectorsFix creation of dataviews inferred with .NET types with sparse vectorsJul 28, 2018
TomFinley
TomFinley
approved these changes
Jul 28, 2018
View reviewed changes
@TomFinleyTomFinley
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xadupre
@TomFinley
TomFinley
merged commit 2107b82
into
dotnet:masterJul 28, 2018
@xadupre
xadupre
commented
Jul 30, 2018
via email 

Copy link


Copy Markdown

ContributorAuthor
codemzs pushed a commit
to codemzs/machinelearning
that referenced
this pull request
Aug 1, 2018
@xadupre@codemzs
Fix creation of dataviews inferred with .NET types with sparse vectors (
760cef2
dotnet#587)
`DataViewConstructionUtils`'s methods to create dataviews over .NET types will now have correctly inferred "getters" in the case of sparse vectors.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 29, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@Ivanidzo4kaIvanidzo4kaIvanidzo4ka approved these changes
@TomFinleyTomFinleyTomFinley approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@xadupre@dnfclas@TomFinley@Ivanidzo4ka@sdpython

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Fix creation of dataviews inferred with .NET types with sparse vectors - #587
Merged
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Jul 28, 2018
Merged
Fix creation of dataviews inferred with .NET types with sparse vectors#587
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Conversation
@xadupre
@xaduprexadupre
commented
Jul 27, 2018•

edited by TomFinley

Loading

Copy link


Copy Markdown

Contributor
Fixes#586. DataViewConstructionUtils's methods to create dataviews over .NET types will now have correctly inferred getters in the case of sparse vectors.
sdpythonand others
added 14 commits
May 18, 2018 19:31
@sdpython
Compile CpuMathNative and FastTreeNative with charset=utf-8, fix for …
edba472
…issue dotnet#78
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5df6fba
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ca76231
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ea730aa
@sdpython
add method AddSerialize to declare entrypoints outside ML.net
95e3646
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
96cae17
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
e55882d
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
719adc4
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
0255942
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5d55dfa
@sdpython
add a unit test to check the creation of a cursor on sparse vectors
d5a419e
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
11aea96
@sdpython
fixdotnet#586and cursor creation on dataview with sparse vectors
3409b9f
@sdpython
revert changes not related todotnet#586
63fa647
@dnfclas
dnfclas
commented
Jul 27, 2018•

edited

Loading

Copy link


Copy Markdown

CLA assistant check
All CLA requirements met.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 Done();
 }

 void GenericDenseDataView<T>(T[] v1, T[] v2)
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void [](start = 8, length = 4)
Explicit access modifier please. Here and everywhere.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated


namespace Microsoft.ML.Runtime.RunTests
{
 public sealed partial class TestSparseDataView : TestDataViewBase
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial [](start = 18, length = 7)
Why partial?
@xaduprexadupreJul 27, 2018

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste without thinking :)
@TomFinley
TomFinley
commented
Jul 27, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for adding this fix plus the extensive tests. Right now the PR description is just the default description. Perhaps something real could be written there. If we were to merge right now it would be very hard to come up with an adequate commit message. (Complicated also by the fact that this PR appears to be based on merges rather than rebases.)
Also if you write something like "fixes #586" in your description, then it will associate the issue and the PR, and close the issue when the PR is merged. You put the thing in the title, but if you check the issue page it was not actually associated. (Well, it will be now since I just mentioned it, but of course as the PR author it should be in the head description, rather than some random reviewer mentioning it in a comment far down the page.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Command;
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain not all of these are used.
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v1, new int[] { 0, 2, 4 }) },
 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v2, new int[] { 0, 1, 3 }) }
 };
 var host = new TlcEnvironment();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var host = new TlcEnvironment(); [](start = 12, length = 32)
can you wrap it in using? #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 GenericSparseDataView(new double[] { 1, 2, 3 }, new double[] { 1, 10, 100 });
 GenericSparseDataView(new DvText[] { new DvText("a"), new DvText("b"), new DvText("c") },
 new DvText[] { new DvText("aa"), new DvText("bb"), new DvText("cc") });
 Done();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done(); [](start = 12, length = 7)
You don't have to call Done if you don't do any basefiles comparison #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
 Assert.True(n == 2);
 }

 [Fact]
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fact [](start = 9, length = 4)
Thank you for fixing my mess!

Your code definitely do right thing, and it's nice to have tests, but I want to point one thing. Your tests never hit TypedCursor changes.

There is two conversions: one from some collection to IDataview through code in DataViewConstructionUltis, and from IDataVIew to Collection via TypedCursor and your tests cover only first case.
In order to invoke second case you need to do something like:

dataView.AsEnumerable(env, false).GetEnumerator() and enumerate over it.

#Resolved
@xaduprexadupreJul 27, 2018•

edited by Ivanidzo4ka

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do it. #Resolved
@xaduprexadupre
changed the title
Fix the creation of getters on dataviews having sparse vectors (#586)Fixes #586, the creation of getters on dataviews having sparse vectorsJul 27, 2018
Ivanidzo4ka
Ivanidzo4ka
approved these changes
Jul 27, 2018
View reviewed changes
@Ivanidzo4kaIvanidzo4ka
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@TomFinley
TomFinley
commented
Jul 28, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for addressing the comment! Clarifying note for the future, when I asked to modify the description, I meant the PR's description, not the PR's title. The description is still the little "we are excited to review your PR" built in text that we intend for people to write. I hope you don't mind, just going to change it so we can get this in since otherwise it looks good.
@TomFinleyTomFinley
changed the title
Fixes #586, the creation of getters on dataviews having sparse vectorsFix creation of dataviews inferred with .NET types with sparse vectorsJul 28, 2018
TomFinley
TomFinley
approved these changes
Jul 28, 2018
View reviewed changes
@TomFinleyTomFinley
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xadupre
@TomFinley
TomFinley
merged commit 2107b82
into
dotnet:masterJul 28, 2018
@xadupre
xadupre
commented
Jul 30, 2018
via email 

Copy link


Copy Markdown

ContributorAuthor
codemzs pushed a commit
to codemzs/machinelearning
that referenced
this pull request
Aug 1, 2018
@xadupre@codemzs
Fix creation of dataviews inferred with .NET types with sparse vectors (
760cef2
dotnet#587)
`DataViewConstructionUtils`'s methods to create dataviews over .NET types will now have correctly inferred "getters" in the case of sparse vectors.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 29, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@Ivanidzo4kaIvanidzo4kaIvanidzo4ka approved these changes
@TomFinleyTomFinleyTomFinley approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@xadupre@dnfclas@TomFinley@Ivanidzo4ka@sdpython

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Fix creation of dataviews inferred with .NET types with sparse vectors - #587
Merged
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Jul 28, 2018
Merged
Fix creation of dataviews inferred with .NET types with sparse vectors#587
TomFinley merged 15 commits into
dotnet:masterfrom 
xadupre:master
Conversation
@xadupre
@xaduprexadupre
commented
Jul 27, 2018•

edited by TomFinley

Loading

Copy link


Copy Markdown

Contributor
Fixes#586. DataViewConstructionUtils's methods to create dataviews over .NET types will now have correctly inferred getters in the case of sparse vectors.
sdpythonand others
added 14 commits
May 18, 2018 19:31
@sdpython
Compile CpuMathNative and FastTreeNative with charset=utf-8, fix for …
edba472
…issue dotnet#78
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5df6fba
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ca76231
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
ea730aa
@sdpython
add method AddSerialize to declare entrypoints outside ML.net
95e3646
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
96cae17
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
e55882d
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
719adc4
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
0255942
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
5d55dfa
@sdpython
add a unit test to check the creation of a cursor on sparse vectors
d5a419e
@sdpython
Merge branch 'master' ofhttps://github.com/dotnet/machinelearning
11aea96
@sdpython
fixdotnet#586and cursor creation on dataview with sparse vectors
3409b9f
@sdpython
revert changes not related todotnet#586
63fa647
@dnfclas
dnfclas
commented
Jul 27, 2018•

edited

Loading

Copy link


Copy Markdown

CLA assistant check
All CLA requirements met.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 Done();
 }

 void GenericDenseDataView<T>(T[] v1, T[] v2)
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void [](start = 8, length = 4)
Explicit access modifier please. Here and everywhere.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated


namespace Microsoft.ML.Runtime.RunTests
{
 public sealed partial class TestSparseDataView : TestDataViewBase
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial [](start = 18, length = 7)
Why partial?
@xaduprexadupreJul 27, 2018

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste without thinking :)
@TomFinley
TomFinley
commented
Jul 27, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for adding this fix plus the extensive tests. Right now the PR description is just the default description. Perhaps something real could be written there. If we were to merge right now it would be very hard to come up with an adequate commit message. (Complicated also by the fact that this PR appears to be based on merges rather than rebases.)
Also if you write something like "fixes #586" in your description, then it will associate the issue and the PR, and close the issue when the PR is merged. You put the thing in the title, but if you check the issue page it was not actually associated. (Well, it will be now since I just mentioned it, but of course as the PR author it should be in the head description, rather than some random reviewer mentioning it in a comment far down the page.
TomFinley
TomFinley
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.ML.Runtime.Command;
@TomFinleyTomFinleyJul 27, 2018

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly certain not all of these are used.
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v1, new int[] { 0, 2, 4 }) },
 new SparseExample<T>() { X = new VBuffer<T> (5, 3, v2, new int[] { 0, 1, 3 }) }
 };
 var host = new TlcEnvironment();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var host = new TlcEnvironment(); [](start = 12, length = 32)
can you wrap it in using? #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
Outdated

 GenericSparseDataView(new double[] { 1, 2, 3 }, new double[] { 1, 10, 100 });
 GenericSparseDataView(new DvText[] { new DvText("a"), new DvText("b"), new DvText("c") },
 new DvText[] { new DvText("aa"), new DvText("bb"), new DvText("cc") });
 Done();
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done(); [](start = 12, length = 7)
You don't have to call Done if you don't do any basefiles comparison #Resolved
Ivanidzo4ka
Ivanidzo4ka
reviewed
Jul 27, 2018
View reviewed changes
Comment threadtest/Microsoft.ML.TestFramework/TestSparseDataView.cs
 Assert.True(n == 2);
 }

 [Fact]
@Ivanidzo4kaIvanidzo4kaJul 27, 2018•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fact [](start = 9, length = 4)
Thank you for fixing my mess!

Your code definitely do right thing, and it's nice to have tests, but I want to point one thing. Your tests never hit TypedCursor changes.

There is two conversions: one from some collection to IDataview through code in DataViewConstructionUltis, and from IDataVIew to Collection via TypedCursor and your tests cover only first case.
In order to invoke second case you need to do something like:

dataView.AsEnumerable(env, false).GetEnumerator() and enumerate over it.

#Resolved
@xaduprexadupreJul 27, 2018•

edited by Ivanidzo4ka

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do it. #Resolved
@xaduprexadupre
changed the title
Fix the creation of getters on dataviews having sparse vectors (#586)Fixes #586, the creation of getters on dataviews having sparse vectorsJul 27, 2018
Ivanidzo4ka
Ivanidzo4ka
approved these changes
Jul 27, 2018
View reviewed changes
@Ivanidzo4kaIvanidzo4ka
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@TomFinley
TomFinley
commented
Jul 28, 2018

Copy link


Copy Markdown

Contributor
Hi @xadupre thanks for addressing the comment! Clarifying note for the future, when I asked to modify the description, I meant the PR's description, not the PR's title. The description is still the little "we are excited to review your PR" built in text that we intend for people to write. I hope you don't mind, just going to change it so we can get this in since otherwise it looks good.
@TomFinleyTomFinley
changed the title
Fixes #586, the creation of getters on dataviews having sparse vectorsFix creation of dataviews inferred with .NET types with sparse vectorsJul 28, 2018
TomFinley
TomFinley
approved these changes
Jul 28, 2018
View reviewed changes
@TomFinleyTomFinley
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xadupre
@TomFinley
TomFinley
merged commit 2107b82
into
dotnet:masterJul 28, 2018
@xadupre
xadupre
commented
Jul 30, 2018
via email 

Copy link


Copy Markdown

ContributorAuthor
codemzs pushed a commit
to codemzs/machinelearning
that referenced
this pull request
Aug 1, 2018
@xadupre@codemzs
Fix creation of dataviews inferred with .NET types with sparse vectors (
760cef2
dotnet#587)
`DataViewConstructionUtils`'s methods to create dataviews over .NET types will now have correctly inferred "getters" in the case of sparse vectors.
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 29, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers


2 more reviewers

@Ivanidzo4kaIvanidzo4kaIvanidzo4ka approved these changes
@TomFinleyTomFinleyTomFinley approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@xadupre@dnfclas@TomFinley@Ivanidzo4ka@sdpython