Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues - #3216

Merged
artidoro merged 8 commits into
dotnet:masterfrom
artidoro:samples1
Apr 10, 2019
Merged

Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues#3216
artidoro merged 8 commits into
dotnet:masterfrom
artidoro:samples1

Conversation

@artidoro

@artidoroartidoro commented Apr 5, 2019

Copy link
Copy Markdown
Contributor

Related to #1209
Fixes#3117

Made samples for the multi-column setting of:

  • ReplaceMissingValues
  • IndicateMissingValues

I also made a sample to save and load the CustomMapping estimator.

@artidoroartidoro self-assigned this Apr 5, 2019
@artidoroartidoro added the documentation Related to documentation of ML.NET label Apr 5, 2019
abgoswam
abgoswam reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs Outdated
@shmoradims

shmoradims commented Apr 5, 2019•
edited by artidoro
Loading

Copy link
Copy Markdown

using System;

please remove Sample from filenames: CustomMapingSample -> CustomMapping

Same with the other file #Resolved

Refers to: docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs:1 in d32f027. [](commit_id = d32f027, deletion_comment = False)

shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSampleSaveAndLoad.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
// ReplaceMissingValues is used to create a column where missing values are replaced according to the ReplacementMode.
var defaultPipeline = mlContext.Transforms.ReplaceMissingValues(new[] {
new InputOutputColumnPair("MissingReplaced1", "Features1"),
new InputOutputColumnPair("MissingReplaced2", "Features2")

@shmoradimsshmoradimsApr 5, 2019•
edited by artidoro
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: we don't have an in-place transformation multicolumn sample. can we transform the data in-place and not have MissingReplaced1/MissingReplaced2?

shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs Outdated
shmoradims
shmoradims reviewed Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/ExtensionsCatalog.cs Outdated
wschin
wschin reviewed Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs Outdated
wschin
wschin reviewed Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs Outdated
wschin
wschin reviewed Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs Outdated
@artidoro

artidoro commented Apr 8, 2019•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Thank you for the review comments, I have update the code accordingly.

artidoro
artidoro commented Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Microsoft.ML.Samples.csproj Outdated
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
<OutputType>Exe</OutputType>
<AssemblyOriginatorKeyFile>$(ToolsDir)Test.snk</AssemblyOriginatorKeyFile>

@artidoroartidoroApr 9, 2019•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the assemblies are signed using Open.snk. However, if we want to register an assembly in the component catalog using mlContext.ComponentCatalog.RegisterAssembly(), the assembly needs to pass the following condition:

machinelearning/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

Lines 1025 to 1034 in b861b5d

privatestaticboolCanContainExtensions(Assemblyassembly)
{
if(assembly.FullName.StartsWith("Microsoft.ML.",StringComparison.Ordinal)
&&HasMLNetPublicKey(assembly))
{
returnfalse;
}
returntrue;
}

So it needs to have a different signature.

@artidoroartidoroApr 9, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with Eric, it could be even simpler not to sign this assembly. We don't need strong naming for the Samples assembly as it should not be referenced by external code anyways.

rogancarr
rogancarr reviewed Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs Outdated
rogancarr
rogancarr reviewed Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs Outdated
rogancarr
rogancarr reviewed Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs Outdated
rogancarr
rogancarr reviewed Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs Outdated
rogancarr
rogancarr reviewed Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs Outdated
rogancarr
rogancarr reviewed Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs Outdated
rogancarr
rogancarr reviewed Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs Outdated
rogancarr
rogancarr reviewed Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs Outdated
// Features: [-1, 2, -3] MissingReplaced: [-1, 2, -3]
// Features: [-1, NaN, -3] MissingReplaced: [-1, 0, -3]

// Mean ReplaceMode:

@rogancarrrogancarrApr 9, 2019•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Mean ReplaceMode: [](start = 12, length = 20)

Isn't there one more mode?

@artidoroartidoroApr 9, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also maximum and minimum, but I don't think it adds much to add them to this sample.

In reply to: 273727261 [](ancestors = 273727261)

@rogancarrrogancarrApr 9, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the documentation, if we have more than one, we should have all of them — right now, it looks inconsistent or incomplete.

In reply to: 273735371 [](ancestors = 273735371,273727261)

@artidoroartidoroApr 9, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather remove the mean replacement mode in this sample then.
We are not trying to exhaust all the possible settings in our samples. Would that be fine?

In reply to: 273736506 [](ancestors = 273736506,273735371,273727261)

wschin
wschin reviewed Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSaveAndLoad.cs Outdated
private class TransformedData : InputData
{
public bool IsUnderThirty { get; set; }

@wschinwschinApr 9, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.

singlis
singlis reviewed Apr 10, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
// Expected output:
// Features1: [1, 1, 0] MissingIndicator1: [False, False, False] Features2: [1, 1] MissingIndicator2: [False, False]
// Features1: [0, NaN, 1] MissingIndicator1: [False, True, False] Features2: [NaN, 1] MissingIndicator2: [True, False]
// Features1: [-1, NaN, -3] MissingIndicator1: [False, True, False] Features2: [1, ∞] MissingIndicator2: [False, False]

@singlissinglisApr 10, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT - spacing for MissingIndicator doesnt align with the above lines.

singlis
singlis approved these changes Apr 10, 2019
View reviewed changes

@singlissinglis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@codecov

codecovBot commented Apr 10, 2019•
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #3216 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@ Coverage Diff @@## master #3216 +/- ##
==========================================
- Coverage 72.62% 72.62% -0.01% 
==========================================
Files 807 807 Lines 145080 145080 Branches 16213 16213 ==========================================
- Hits 105369 105365 -4 - Misses 35294 35298 +4 
Partials 4417 4417
FlagCoverage Δ
#Debug72.62% <ø> (-0.01%)⬇️
#production68.17% <ø> (-0.01%)⬇️
#test88.92% <ø> (-0.01%)⬇️
Impacted FilesCoverage Δ
src/Microsoft.ML.Transforms/ExtensionsCatalog.cs57.14% <ø> (ø)⬆️
...rc/Microsoft.ML.Transforms/CustomMappingCatalog.cs100% <ø> (ø)⬆️
src/Microsoft.ML.Transforms/Text/LdaTransform.cs89.26% <0%> (-0.63%)⬇️
...soft.ML.TestFramework/DataPipe/TestDataPipeBase.cs73.7% <0%> (-0.34%)⬇️
...StandardTrainers/Standard/LinearModelParameters.cs60.05% <0%> (-0.27%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.26% <0%> (+0.15%)⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs85.31% <0%> (+0.6%)⬆️

@artidoro
artidoro merged commit 304170a into dotnet:masterApr 10, 2019
artidoro added a commit to artidoro/machinelearning that referenced this pull request Apr 10, 2019
@artidoro
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues (
ee2ffdb
dotnet#3216)
@artidoroartidoro mentioned this pull request Apr 10, 2019
Closed
@ghostghost locked as resolved and limited conversation to collaborators Mar 23, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@sfilipisfilipiAwaiting requested review from sfilipi
5 more reviewers
@abgoswamabgoswamabgoswam left review comments
@shmoradimsshmoradimsshmoradims left review comments
@rogancarrrogancarrrogancarr left review comments
@wschinwschinwschin approved these changes
@singlissinglissinglis approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@artidoroartidoro

Labels

documentationRelated to documentation of ML.NET

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@artidoro@shmoradims@wschin@abgoswam@singlis@rogancarr
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues - #3216
Merged
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Apr 10, 2019
Merged
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues#3216
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Conversation
@artidoro
@artidoroartidoro
commented
Apr 5, 2019

Copy link


Copy Markdown

Contributor
Related to #1209
Fixes#3117
Made samples for the multi-column setting of:
  • ReplaceMissingValues
  • IndicateMissingValues
I also made a sample to save and load the CustomMapping estimator.
@artidoroartidoro
self-assigned this
Apr 5, 2019
@artidoroartidoro
added
the 
documentation
Related to documentation of ML.NET
label
Apr 5, 2019
abgoswam
abgoswam
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@shmoradims
shmoradims
commented
Apr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

using System;
please remove Sample from filenames: CustomMapingSample -> CustomMapping
Same with the other file #Resolved
Refers to: docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs:1 in d32f027. [](commit_id = d32f027, deletion_comment = False)
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSampleSaveAndLoad.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
 // ReplaceMissingValues is used to create a column where missing values are replaced according to the ReplacementMode.
 var defaultPipeline = mlContext.Transforms.ReplaceMissingValues(new[] {
 new InputOutputColumnPair("MissingReplaced1", "Features1"),
 new InputOutputColumnPair("MissingReplaced2", "Features2")
@shmoradimsshmoradimsApr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: we don't have an in-place transformation multicolumn sample. can we transform the data in-place and not have MissingReplaced1/MissingReplaced2?
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/ExtensionsCatalog.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@artidoro
artidoro
commented
Apr 8, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Thank you for the review comments, I have update the code accordingly.
artidoro
artidoro
commented
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Microsoft.ML.Samples.csproj
Outdated

 <PropertyGroup>
 <TargetFramework>netcoreapp2.1</TargetFramework>
 <OutputType>Exe</OutputType>
 <AssemblyOriginatorKeyFile>$(ToolsDir)Test.snk</AssemblyOriginatorKeyFile>
@artidoroartidoroApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the assemblies are signed using Open.snk. However, if we want to register an assembly in the component catalog using mlContext.ComponentCatalog.RegisterAssembly(), the assembly needs to pass the following condition:
machinelearning/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

Lines 1025 to 1034
in
b861b5d
privatestaticboolCanContainExtensions(Assemblyassembly)
{
if(assembly.FullName.StartsWith("Microsoft.ML.",StringComparison.Ordinal)
&&HasMLNetPublicKey(assembly))
{
returnfalse;
}
returntrue;
}
So it needs to have a different signature.
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with Eric, it could be even simpler not to sign this assembly. We don't need strong naming for the Samples assembly as it should not be referenced by external code anyways.
rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

 // Features: [-1, 2, -3] MissingReplaced: [-1, 2, -3]
 // Features: [-1, NaN, -3] MissingReplaced: [-1, 0, -3]

 // Mean ReplaceMode:
@rogancarrrogancarrApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Mean ReplaceMode: [](start = 12, length = 20)
Isn't there one more mode?
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also maximum and minimum, but I don't think it adds much to add them to this sample.
In reply to: 273727261 [](ancestors = 273727261)
@rogancarrrogancarrApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the documentation, if we have more than one, we should have all of them — right now, it looks inconsistent or incomplete.
In reply to: 273735371 [](ancestors = 273735371,273727261)
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather remove the mean replacement mode in this sample then.

We are not trying to exhaust all the possible settings in our samples. Would that be fine?
In reply to: 273736506 [](ancestors = 273736506,273735371,273727261)
wschin
wschin
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSaveAndLoad.cs
Outdated

 private class TransformedData : InputData
 {
 public bool IsUnderThirty { get; set; }

@wschinwschinApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.
singlis
singlis
reviewed
Apr 10, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
 // Expected output:
 // Features1: [1, 1, 0] MissingIndicator1: [False, False, False] Features2: [1, 1] MissingIndicator2: [False, False]
 // Features1: [0, NaN, 1] MissingIndicator1: [False, True, False] Features2: [NaN, 1] MissingIndicator2: [True, False]
 // Features1: [-1, NaN, -3] MissingIndicator1: [False, True, False] Features2: [1, ∞] MissingIndicator2: [False, False]
@singlissinglisApr 10, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT - spacing for MissingIndicator doesnt align with the above lines.
singlis
singlis
approved these changes
Apr 10, 2019
View reviewed changes
@singlissinglis
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@codecov
codecovBot
commented
Apr 10, 2019•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #3216 into master will decrease coverage by <.01%.

The diff coverage is n/a.
@@ Coverage Diff @@## master #3216 +/- ##
==========================================
- Coverage 72.62% 72.62% -0.01% 
==========================================
Files 807 807 Lines 145080 145080 Branches 16213 16213 ==========================================
- Hits 105369 105365 -4 - Misses 35294 35298 +4 
Partials 4417 4417
FlagCoverage Δ
#Debug72.62% <ø> (-0.01%)⬇️
#production68.17% <ø> (-0.01%)⬇️
#test88.92% <ø> (-0.01%)⬇️
Impacted FilesCoverage Δ
src/Microsoft.ML.Transforms/ExtensionsCatalog.cs57.14% <ø> (ø)⬆️
...rc/Microsoft.ML.Transforms/CustomMappingCatalog.cs100% <ø> (ø)⬆️
src/Microsoft.ML.Transforms/Text/LdaTransform.cs89.26% <0%> (-0.63%)⬇️
...soft.ML.TestFramework/DataPipe/TestDataPipeBase.cs73.7% <0%> (-0.34%)⬇️
...StandardTrainers/Standard/LinearModelParameters.cs60.05% <0%> (-0.27%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.26% <0%> (+0.15%)⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs85.31% <0%> (+0.6%)⬆️
@artidoro
artidoro
merged commit 304170a
into
dotnet:masterApr 10, 2019
artidoro added a commit
to artidoro/machinelearning
that referenced
this pull request
Apr 10, 2019
@artidoro
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues (
ee2ffdb
dotnet#3216)
@artidoroartidoro
mentioned this pull request
Apr 10, 2019
 Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 23, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@sfilipisfilipiAwaiting requested review from sfilipi

5 more reviewers

@abgoswamabgoswamabgoswam left review comments
@shmoradimsshmoradimsshmoradims left review comments
@rogancarrrogancarrrogancarr left review comments
@wschinwschinwschin approved these changes
@singlissinglissinglis approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@artidoroartidoro

Labels

documentationRelated to documentation of ML.NET

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@artidoro@shmoradims@wschin@abgoswam@singlis@rogancarr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues - #3216
Merged
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Apr 10, 2019
Merged
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues#3216
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Conversation
@artidoro
@artidoroartidoro
commented
Apr 5, 2019

Copy link


Copy Markdown

Contributor
Related to #1209
Fixes#3117
Made samples for the multi-column setting of:
  • ReplaceMissingValues
  • IndicateMissingValues
I also made a sample to save and load the CustomMapping estimator.
@artidoroartidoro
self-assigned this
Apr 5, 2019
@artidoroartidoro
added
the 
documentation
Related to documentation of ML.NET
label
Apr 5, 2019
abgoswam
abgoswam
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@shmoradims
shmoradims
commented
Apr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

using System;
please remove Sample from filenames: CustomMapingSample -> CustomMapping
Same with the other file #Resolved
Refers to: docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs:1 in d32f027. [](commit_id = d32f027, deletion_comment = False)
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSampleSaveAndLoad.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
 // ReplaceMissingValues is used to create a column where missing values are replaced according to the ReplacementMode.
 var defaultPipeline = mlContext.Transforms.ReplaceMissingValues(new[] {
 new InputOutputColumnPair("MissingReplaced1", "Features1"),
 new InputOutputColumnPair("MissingReplaced2", "Features2")
@shmoradimsshmoradimsApr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: we don't have an in-place transformation multicolumn sample. can we transform the data in-place and not have MissingReplaced1/MissingReplaced2?
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/ExtensionsCatalog.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@artidoro
artidoro
commented
Apr 8, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Thank you for the review comments, I have update the code accordingly.
artidoro
artidoro
commented
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Microsoft.ML.Samples.csproj
Outdated

 <PropertyGroup>
 <TargetFramework>netcoreapp2.1</TargetFramework>
 <OutputType>Exe</OutputType>
 <AssemblyOriginatorKeyFile>$(ToolsDir)Test.snk</AssemblyOriginatorKeyFile>
@artidoroartidoroApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the assemblies are signed using Open.snk. However, if we want to register an assembly in the component catalog using mlContext.ComponentCatalog.RegisterAssembly(), the assembly needs to pass the following condition:
machinelearning/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

Lines 1025 to 1034
in
b861b5d
privatestaticboolCanContainExtensions(Assemblyassembly)
{
if(assembly.FullName.StartsWith("Microsoft.ML.",StringComparison.Ordinal)
&&HasMLNetPublicKey(assembly))
{
returnfalse;
}
returntrue;
}
So it needs to have a different signature.
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with Eric, it could be even simpler not to sign this assembly. We don't need strong naming for the Samples assembly as it should not be referenced by external code anyways.
rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

 // Features: [-1, 2, -3] MissingReplaced: [-1, 2, -3]
 // Features: [-1, NaN, -3] MissingReplaced: [-1, 0, -3]

 // Mean ReplaceMode:
@rogancarrrogancarrApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Mean ReplaceMode: [](start = 12, length = 20)
Isn't there one more mode?
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also maximum and minimum, but I don't think it adds much to add them to this sample.
In reply to: 273727261 [](ancestors = 273727261)
@rogancarrrogancarrApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the documentation, if we have more than one, we should have all of them — right now, it looks inconsistent or incomplete.
In reply to: 273735371 [](ancestors = 273735371,273727261)
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather remove the mean replacement mode in this sample then.

We are not trying to exhaust all the possible settings in our samples. Would that be fine?
In reply to: 273736506 [](ancestors = 273736506,273735371,273727261)
wschin
wschin
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSaveAndLoad.cs
Outdated

 private class TransformedData : InputData
 {
 public bool IsUnderThirty { get; set; }

@wschinwschinApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.
singlis
singlis
reviewed
Apr 10, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
 // Expected output:
 // Features1: [1, 1, 0] MissingIndicator1: [False, False, False] Features2: [1, 1] MissingIndicator2: [False, False]
 // Features1: [0, NaN, 1] MissingIndicator1: [False, True, False] Features2: [NaN, 1] MissingIndicator2: [True, False]
 // Features1: [-1, NaN, -3] MissingIndicator1: [False, True, False] Features2: [1, ∞] MissingIndicator2: [False, False]
@singlissinglisApr 10, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT - spacing for MissingIndicator doesnt align with the above lines.
singlis
singlis
approved these changes
Apr 10, 2019
View reviewed changes
@singlissinglis
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@codecov
codecovBot
commented
Apr 10, 2019•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #3216 into master will decrease coverage by <.01%.

The diff coverage is n/a.
@@ Coverage Diff @@## master #3216 +/- ##
==========================================
- Coverage 72.62% 72.62% -0.01% 
==========================================
Files 807 807 Lines 145080 145080 Branches 16213 16213 ==========================================
- Hits 105369 105365 -4 - Misses 35294 35298 +4 
Partials 4417 4417
FlagCoverage Δ
#Debug72.62% <ø> (-0.01%)⬇️
#production68.17% <ø> (-0.01%)⬇️
#test88.92% <ø> (-0.01%)⬇️
Impacted FilesCoverage Δ
src/Microsoft.ML.Transforms/ExtensionsCatalog.cs57.14% <ø> (ø)⬆️
...rc/Microsoft.ML.Transforms/CustomMappingCatalog.cs100% <ø> (ø)⬆️
src/Microsoft.ML.Transforms/Text/LdaTransform.cs89.26% <0%> (-0.63%)⬇️
...soft.ML.TestFramework/DataPipe/TestDataPipeBase.cs73.7% <0%> (-0.34%)⬇️
...StandardTrainers/Standard/LinearModelParameters.cs60.05% <0%> (-0.27%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.26% <0%> (+0.15%)⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs85.31% <0%> (+0.6%)⬆️
@artidoro
artidoro
merged commit 304170a
into
dotnet:masterApr 10, 2019
artidoro added a commit
to artidoro/machinelearning
that referenced
this pull request
Apr 10, 2019
@artidoro
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues (
ee2ffdb
dotnet#3216)
@artidoroartidoro
mentioned this pull request
Apr 10, 2019
 Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 23, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@sfilipisfilipiAwaiting requested review from sfilipi

5 more reviewers

@abgoswamabgoswamabgoswam left review comments
@shmoradimsshmoradimsshmoradims left review comments
@rogancarrrogancarrrogancarr left review comments
@wschinwschinwschin approved these changes
@singlissinglissinglis approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@artidoroartidoro

Labels

documentationRelated to documentation of ML.NET

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@artidoro@shmoradims@wschin@abgoswam@singlis@rogancarr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues - #3216
Merged
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Apr 10, 2019
Merged
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues#3216
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Conversation
@artidoro
@artidoroartidoro
commented
Apr 5, 2019

Copy link


Copy Markdown

Contributor
Related to #1209
Fixes#3117
Made samples for the multi-column setting of:
  • ReplaceMissingValues
  • IndicateMissingValues
I also made a sample to save and load the CustomMapping estimator.
@artidoroartidoro
self-assigned this
Apr 5, 2019
@artidoroartidoro
added
the 
documentation
Related to documentation of ML.NET
label
Apr 5, 2019
abgoswam
abgoswam
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@shmoradims
shmoradims
commented
Apr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

using System;
please remove Sample from filenames: CustomMapingSample -> CustomMapping
Same with the other file #Resolved
Refers to: docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs:1 in d32f027. [](commit_id = d32f027, deletion_comment = False)
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSampleSaveAndLoad.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
 // ReplaceMissingValues is used to create a column where missing values are replaced according to the ReplacementMode.
 var defaultPipeline = mlContext.Transforms.ReplaceMissingValues(new[] {
 new InputOutputColumnPair("MissingReplaced1", "Features1"),
 new InputOutputColumnPair("MissingReplaced2", "Features2")
@shmoradimsshmoradimsApr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: we don't have an in-place transformation multicolumn sample. can we transform the data in-place and not have MissingReplaced1/MissingReplaced2?
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/ExtensionsCatalog.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@artidoro
artidoro
commented
Apr 8, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Thank you for the review comments, I have update the code accordingly.
artidoro
artidoro
commented
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Microsoft.ML.Samples.csproj
Outdated

 <PropertyGroup>
 <TargetFramework>netcoreapp2.1</TargetFramework>
 <OutputType>Exe</OutputType>
 <AssemblyOriginatorKeyFile>$(ToolsDir)Test.snk</AssemblyOriginatorKeyFile>
@artidoroartidoroApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the assemblies are signed using Open.snk. However, if we want to register an assembly in the component catalog using mlContext.ComponentCatalog.RegisterAssembly(), the assembly needs to pass the following condition:
machinelearning/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

Lines 1025 to 1034
in
b861b5d
privatestaticboolCanContainExtensions(Assemblyassembly)
{
if(assembly.FullName.StartsWith("Microsoft.ML.",StringComparison.Ordinal)
&&HasMLNetPublicKey(assembly))
{
returnfalse;
}
returntrue;
}
So it needs to have a different signature.
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with Eric, it could be even simpler not to sign this assembly. We don't need strong naming for the Samples assembly as it should not be referenced by external code anyways.
rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

 // Features: [-1, 2, -3] MissingReplaced: [-1, 2, -3]
 // Features: [-1, NaN, -3] MissingReplaced: [-1, 0, -3]

 // Mean ReplaceMode:
@rogancarrrogancarrApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Mean ReplaceMode: [](start = 12, length = 20)
Isn't there one more mode?
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also maximum and minimum, but I don't think it adds much to add them to this sample.
In reply to: 273727261 [](ancestors = 273727261)
@rogancarrrogancarrApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the documentation, if we have more than one, we should have all of them — right now, it looks inconsistent or incomplete.
In reply to: 273735371 [](ancestors = 273735371,273727261)
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather remove the mean replacement mode in this sample then.

We are not trying to exhaust all the possible settings in our samples. Would that be fine?
In reply to: 273736506 [](ancestors = 273736506,273735371,273727261)
wschin
wschin
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSaveAndLoad.cs
Outdated

 private class TransformedData : InputData
 {
 public bool IsUnderThirty { get; set; }

@wschinwschinApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.
singlis
singlis
reviewed
Apr 10, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
 // Expected output:
 // Features1: [1, 1, 0] MissingIndicator1: [False, False, False] Features2: [1, 1] MissingIndicator2: [False, False]
 // Features1: [0, NaN, 1] MissingIndicator1: [False, True, False] Features2: [NaN, 1] MissingIndicator2: [True, False]
 // Features1: [-1, NaN, -3] MissingIndicator1: [False, True, False] Features2: [1, ∞] MissingIndicator2: [False, False]
@singlissinglisApr 10, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT - spacing for MissingIndicator doesnt align with the above lines.
singlis
singlis
approved these changes
Apr 10, 2019
View reviewed changes
@singlissinglis
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@codecov
codecovBot
commented
Apr 10, 2019•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #3216 into master will decrease coverage by <.01%.

The diff coverage is n/a.
@@ Coverage Diff @@## master #3216 +/- ##
==========================================
- Coverage 72.62% 72.62% -0.01% 
==========================================
Files 807 807 Lines 145080 145080 Branches 16213 16213 ==========================================
- Hits 105369 105365 -4 - Misses 35294 35298 +4 
Partials 4417 4417
FlagCoverage Δ
#Debug72.62% <ø> (-0.01%)⬇️
#production68.17% <ø> (-0.01%)⬇️
#test88.92% <ø> (-0.01%)⬇️
Impacted FilesCoverage Δ
src/Microsoft.ML.Transforms/ExtensionsCatalog.cs57.14% <ø> (ø)⬆️
...rc/Microsoft.ML.Transforms/CustomMappingCatalog.cs100% <ø> (ø)⬆️
src/Microsoft.ML.Transforms/Text/LdaTransform.cs89.26% <0%> (-0.63%)⬇️
...soft.ML.TestFramework/DataPipe/TestDataPipeBase.cs73.7% <0%> (-0.34%)⬇️
...StandardTrainers/Standard/LinearModelParameters.cs60.05% <0%> (-0.27%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.26% <0%> (+0.15%)⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs85.31% <0%> (+0.6%)⬆️
@artidoro
artidoro
merged commit 304170a
into
dotnet:masterApr 10, 2019
artidoro added a commit
to artidoro/machinelearning
that referenced
this pull request
Apr 10, 2019
@artidoro
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues (
ee2ffdb
dotnet#3216)
@artidoroartidoro
mentioned this pull request
Apr 10, 2019
 Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 23, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@sfilipisfilipiAwaiting requested review from sfilipi

5 more reviewers

@abgoswamabgoswamabgoswam left review comments
@shmoradimsshmoradimsshmoradims left review comments
@rogancarrrogancarrrogancarr left review comments
@wschinwschinwschin approved these changes
@singlissinglissinglis approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@artidoroartidoro

Labels

documentationRelated to documentation of ML.NET

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@artidoro@shmoradims@wschin@abgoswam@singlis@rogancarr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues - #3216
Merged
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Apr 10, 2019
Merged
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues#3216
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Conversation
@artidoro
@artidoroartidoro
commented
Apr 5, 2019

Copy link


Copy Markdown

Contributor
Related to #1209
Fixes#3117
Made samples for the multi-column setting of:
  • ReplaceMissingValues
  • IndicateMissingValues
I also made a sample to save and load the CustomMapping estimator.
@artidoroartidoro
self-assigned this
Apr 5, 2019
@artidoroartidoro
added
the 
documentation
Related to documentation of ML.NET
label
Apr 5, 2019
abgoswam
abgoswam
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@shmoradims
shmoradims
commented
Apr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

using System;
please remove Sample from filenames: CustomMapingSample -> CustomMapping
Same with the other file #Resolved
Refers to: docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs:1 in d32f027. [](commit_id = d32f027, deletion_comment = False)
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSampleSaveAndLoad.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
 // ReplaceMissingValues is used to create a column where missing values are replaced according to the ReplacementMode.
 var defaultPipeline = mlContext.Transforms.ReplaceMissingValues(new[] {
 new InputOutputColumnPair("MissingReplaced1", "Features1"),
 new InputOutputColumnPair("MissingReplaced2", "Features2")
@shmoradimsshmoradimsApr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: we don't have an in-place transformation multicolumn sample. can we transform the data in-place and not have MissingReplaced1/MissingReplaced2?
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/ExtensionsCatalog.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@artidoro
artidoro
commented
Apr 8, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Thank you for the review comments, I have update the code accordingly.
artidoro
artidoro
commented
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Microsoft.ML.Samples.csproj
Outdated

 <PropertyGroup>
 <TargetFramework>netcoreapp2.1</TargetFramework>
 <OutputType>Exe</OutputType>
 <AssemblyOriginatorKeyFile>$(ToolsDir)Test.snk</AssemblyOriginatorKeyFile>
@artidoroartidoroApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the assemblies are signed using Open.snk. However, if we want to register an assembly in the component catalog using mlContext.ComponentCatalog.RegisterAssembly(), the assembly needs to pass the following condition:
machinelearning/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

Lines 1025 to 1034
in
b861b5d
privatestaticboolCanContainExtensions(Assemblyassembly)
{
if(assembly.FullName.StartsWith("Microsoft.ML.",StringComparison.Ordinal)
&&HasMLNetPublicKey(assembly))
{
returnfalse;
}
returntrue;
}
So it needs to have a different signature.
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with Eric, it could be even simpler not to sign this assembly. We don't need strong naming for the Samples assembly as it should not be referenced by external code anyways.
rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

 // Features: [-1, 2, -3] MissingReplaced: [-1, 2, -3]
 // Features: [-1, NaN, -3] MissingReplaced: [-1, 0, -3]

 // Mean ReplaceMode:
@rogancarrrogancarrApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Mean ReplaceMode: [](start = 12, length = 20)
Isn't there one more mode?
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also maximum and minimum, but I don't think it adds much to add them to this sample.
In reply to: 273727261 [](ancestors = 273727261)
@rogancarrrogancarrApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the documentation, if we have more than one, we should have all of them — right now, it looks inconsistent or incomplete.
In reply to: 273735371 [](ancestors = 273735371,273727261)
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather remove the mean replacement mode in this sample then.

We are not trying to exhaust all the possible settings in our samples. Would that be fine?
In reply to: 273736506 [](ancestors = 273736506,273735371,273727261)
wschin
wschin
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSaveAndLoad.cs
Outdated

 private class TransformedData : InputData
 {
 public bool IsUnderThirty { get; set; }

@wschinwschinApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.
singlis
singlis
reviewed
Apr 10, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
 // Expected output:
 // Features1: [1, 1, 0] MissingIndicator1: [False, False, False] Features2: [1, 1] MissingIndicator2: [False, False]
 // Features1: [0, NaN, 1] MissingIndicator1: [False, True, False] Features2: [NaN, 1] MissingIndicator2: [True, False]
 // Features1: [-1, NaN, -3] MissingIndicator1: [False, True, False] Features2: [1, ∞] MissingIndicator2: [False, False]
@singlissinglisApr 10, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT - spacing for MissingIndicator doesnt align with the above lines.
singlis
singlis
approved these changes
Apr 10, 2019
View reviewed changes
@singlissinglis
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@codecov
codecovBot
commented
Apr 10, 2019•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #3216 into master will decrease coverage by <.01%.

The diff coverage is n/a.
@@ Coverage Diff @@## master #3216 +/- ##
==========================================
- Coverage 72.62% 72.62% -0.01% 
==========================================
Files 807 807 Lines 145080 145080 Branches 16213 16213 ==========================================
- Hits 105369 105365 -4 - Misses 35294 35298 +4 
Partials 4417 4417
FlagCoverage Δ
#Debug72.62% <ø> (-0.01%)⬇️
#production68.17% <ø> (-0.01%)⬇️
#test88.92% <ø> (-0.01%)⬇️
Impacted FilesCoverage Δ
src/Microsoft.ML.Transforms/ExtensionsCatalog.cs57.14% <ø> (ø)⬆️
...rc/Microsoft.ML.Transforms/CustomMappingCatalog.cs100% <ø> (ø)⬆️
src/Microsoft.ML.Transforms/Text/LdaTransform.cs89.26% <0%> (-0.63%)⬇️
...soft.ML.TestFramework/DataPipe/TestDataPipeBase.cs73.7% <0%> (-0.34%)⬇️
...StandardTrainers/Standard/LinearModelParameters.cs60.05% <0%> (-0.27%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.26% <0%> (+0.15%)⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs85.31% <0%> (+0.6%)⬆️
@artidoro
artidoro
merged commit 304170a
into
dotnet:masterApr 10, 2019
artidoro added a commit
to artidoro/machinelearning
that referenced
this pull request
Apr 10, 2019
@artidoro
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues (
ee2ffdb
dotnet#3216)
@artidoroartidoro
mentioned this pull request
Apr 10, 2019
 Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 23, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@sfilipisfilipiAwaiting requested review from sfilipi

5 more reviewers

@abgoswamabgoswamabgoswam left review comments
@shmoradimsshmoradimsshmoradims left review comments
@rogancarrrogancarrrogancarr left review comments
@wschinwschinwschin approved these changes
@singlissinglissinglis approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@artidoroartidoro

Labels

documentationRelated to documentation of ML.NET

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@artidoro@shmoradims@wschin@abgoswam@singlis@rogancarr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues - #3216
Merged
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Apr 10, 2019
Merged
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues#3216
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Conversation
@artidoro
@artidoroartidoro
commented
Apr 5, 2019

Copy link


Copy Markdown

Contributor
Related to #1209
Fixes#3117
Made samples for the multi-column setting of:
  • ReplaceMissingValues
  • IndicateMissingValues
I also made a sample to save and load the CustomMapping estimator.
@artidoroartidoro
self-assigned this
Apr 5, 2019
@artidoroartidoro
added
the 
documentation
Related to documentation of ML.NET
label
Apr 5, 2019
abgoswam
abgoswam
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@shmoradims
shmoradims
commented
Apr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

using System;
please remove Sample from filenames: CustomMapingSample -> CustomMapping
Same with the other file #Resolved
Refers to: docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs:1 in d32f027. [](commit_id = d32f027, deletion_comment = False)
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSampleSaveAndLoad.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
 // ReplaceMissingValues is used to create a column where missing values are replaced according to the ReplacementMode.
 var defaultPipeline = mlContext.Transforms.ReplaceMissingValues(new[] {
 new InputOutputColumnPair("MissingReplaced1", "Features1"),
 new InputOutputColumnPair("MissingReplaced2", "Features2")
@shmoradimsshmoradimsApr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: we don't have an in-place transformation multicolumn sample. can we transform the data in-place and not have MissingReplaced1/MissingReplaced2?
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/ExtensionsCatalog.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@artidoro
artidoro
commented
Apr 8, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Thank you for the review comments, I have update the code accordingly.
artidoro
artidoro
commented
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Microsoft.ML.Samples.csproj
Outdated

 <PropertyGroup>
 <TargetFramework>netcoreapp2.1</TargetFramework>
 <OutputType>Exe</OutputType>
 <AssemblyOriginatorKeyFile>$(ToolsDir)Test.snk</AssemblyOriginatorKeyFile>
@artidoroartidoroApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the assemblies are signed using Open.snk. However, if we want to register an assembly in the component catalog using mlContext.ComponentCatalog.RegisterAssembly(), the assembly needs to pass the following condition:
machinelearning/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

Lines 1025 to 1034
in
b861b5d
privatestaticboolCanContainExtensions(Assemblyassembly)
{
if(assembly.FullName.StartsWith("Microsoft.ML.",StringComparison.Ordinal)
&&HasMLNetPublicKey(assembly))
{
returnfalse;
}
returntrue;
}
So it needs to have a different signature.
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with Eric, it could be even simpler not to sign this assembly. We don't need strong naming for the Samples assembly as it should not be referenced by external code anyways.
rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

 // Features: [-1, 2, -3] MissingReplaced: [-1, 2, -3]
 // Features: [-1, NaN, -3] MissingReplaced: [-1, 0, -3]

 // Mean ReplaceMode:
@rogancarrrogancarrApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Mean ReplaceMode: [](start = 12, length = 20)
Isn't there one more mode?
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also maximum and minimum, but I don't think it adds much to add them to this sample.
In reply to: 273727261 [](ancestors = 273727261)
@rogancarrrogancarrApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the documentation, if we have more than one, we should have all of them — right now, it looks inconsistent or incomplete.
In reply to: 273735371 [](ancestors = 273735371,273727261)
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather remove the mean replacement mode in this sample then.

We are not trying to exhaust all the possible settings in our samples. Would that be fine?
In reply to: 273736506 [](ancestors = 273736506,273735371,273727261)
wschin
wschin
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSaveAndLoad.cs
Outdated

 private class TransformedData : InputData
 {
 public bool IsUnderThirty { get; set; }

@wschinwschinApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.
singlis
singlis
reviewed
Apr 10, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
 // Expected output:
 // Features1: [1, 1, 0] MissingIndicator1: [False, False, False] Features2: [1, 1] MissingIndicator2: [False, False]
 // Features1: [0, NaN, 1] MissingIndicator1: [False, True, False] Features2: [NaN, 1] MissingIndicator2: [True, False]
 // Features1: [-1, NaN, -3] MissingIndicator1: [False, True, False] Features2: [1, ∞] MissingIndicator2: [False, False]
@singlissinglisApr 10, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT - spacing for MissingIndicator doesnt align with the above lines.
singlis
singlis
approved these changes
Apr 10, 2019
View reviewed changes
@singlissinglis
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@codecov
codecovBot
commented
Apr 10, 2019•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #3216 into master will decrease coverage by <.01%.

The diff coverage is n/a.
@@ Coverage Diff @@## master #3216 +/- ##
==========================================
- Coverage 72.62% 72.62% -0.01% 
==========================================
Files 807 807 Lines 145080 145080 Branches 16213 16213 ==========================================
- Hits 105369 105365 -4 - Misses 35294 35298 +4 
Partials 4417 4417
FlagCoverage Δ
#Debug72.62% <ø> (-0.01%)⬇️
#production68.17% <ø> (-0.01%)⬇️
#test88.92% <ø> (-0.01%)⬇️
Impacted FilesCoverage Δ
src/Microsoft.ML.Transforms/ExtensionsCatalog.cs57.14% <ø> (ø)⬆️
...rc/Microsoft.ML.Transforms/CustomMappingCatalog.cs100% <ø> (ø)⬆️
src/Microsoft.ML.Transforms/Text/LdaTransform.cs89.26% <0%> (-0.63%)⬇️
...soft.ML.TestFramework/DataPipe/TestDataPipeBase.cs73.7% <0%> (-0.34%)⬇️
...StandardTrainers/Standard/LinearModelParameters.cs60.05% <0%> (-0.27%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.26% <0%> (+0.15%)⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs85.31% <0%> (+0.6%)⬆️
@artidoro
artidoro
merged commit 304170a
into
dotnet:masterApr 10, 2019
artidoro added a commit
to artidoro/machinelearning
that referenced
this pull request
Apr 10, 2019
@artidoro
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues (
ee2ffdb
dotnet#3216)
@artidoroartidoro
mentioned this pull request
Apr 10, 2019
 Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 23, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@sfilipisfilipiAwaiting requested review from sfilipi

5 more reviewers

@abgoswamabgoswamabgoswam left review comments
@shmoradimsshmoradimsshmoradims left review comments
@rogancarrrogancarrrogancarr left review comments
@wschinwschinwschin approved these changes
@singlissinglissinglis approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@artidoroartidoro

Labels

documentationRelated to documentation of ML.NET

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@artidoro@shmoradims@wschin@abgoswam@singlis@rogancarr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues - #3216
Merged
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Apr 10, 2019
Merged
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues#3216
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Conversation
@artidoro
@artidoroartidoro
commented
Apr 5, 2019

Copy link


Copy Markdown

Contributor
Related to #1209
Fixes#3117
Made samples for the multi-column setting of:
  • ReplaceMissingValues
  • IndicateMissingValues
I also made a sample to save and load the CustomMapping estimator.
@artidoroartidoro
self-assigned this
Apr 5, 2019
@artidoroartidoro
added
the 
documentation
Related to documentation of ML.NET
label
Apr 5, 2019
abgoswam
abgoswam
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@shmoradims
shmoradims
commented
Apr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

using System;
please remove Sample from filenames: CustomMapingSample -> CustomMapping
Same with the other file #Resolved
Refers to: docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs:1 in d32f027. [](commit_id = d32f027, deletion_comment = False)
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSampleSaveAndLoad.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
 // ReplaceMissingValues is used to create a column where missing values are replaced according to the ReplacementMode.
 var defaultPipeline = mlContext.Transforms.ReplaceMissingValues(new[] {
 new InputOutputColumnPair("MissingReplaced1", "Features1"),
 new InputOutputColumnPair("MissingReplaced2", "Features2")
@shmoradimsshmoradimsApr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: we don't have an in-place transformation multicolumn sample. can we transform the data in-place and not have MissingReplaced1/MissingReplaced2?
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/ExtensionsCatalog.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@artidoro
artidoro
commented
Apr 8, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Thank you for the review comments, I have update the code accordingly.
artidoro
artidoro
commented
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Microsoft.ML.Samples.csproj
Outdated

 <PropertyGroup>
 <TargetFramework>netcoreapp2.1</TargetFramework>
 <OutputType>Exe</OutputType>
 <AssemblyOriginatorKeyFile>$(ToolsDir)Test.snk</AssemblyOriginatorKeyFile>
@artidoroartidoroApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the assemblies are signed using Open.snk. However, if we want to register an assembly in the component catalog using mlContext.ComponentCatalog.RegisterAssembly(), the assembly needs to pass the following condition:
machinelearning/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

Lines 1025 to 1034
in
b861b5d
privatestaticboolCanContainExtensions(Assemblyassembly)
{
if(assembly.FullName.StartsWith("Microsoft.ML.",StringComparison.Ordinal)
&&HasMLNetPublicKey(assembly))
{
returnfalse;
}
returntrue;
}
So it needs to have a different signature.
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with Eric, it could be even simpler not to sign this assembly. We don't need strong naming for the Samples assembly as it should not be referenced by external code anyways.
rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

 // Features: [-1, 2, -3] MissingReplaced: [-1, 2, -3]
 // Features: [-1, NaN, -3] MissingReplaced: [-1, 0, -3]

 // Mean ReplaceMode:
@rogancarrrogancarrApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Mean ReplaceMode: [](start = 12, length = 20)
Isn't there one more mode?
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also maximum and minimum, but I don't think it adds much to add them to this sample.
In reply to: 273727261 [](ancestors = 273727261)
@rogancarrrogancarrApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the documentation, if we have more than one, we should have all of them — right now, it looks inconsistent or incomplete.
In reply to: 273735371 [](ancestors = 273735371,273727261)
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather remove the mean replacement mode in this sample then.

We are not trying to exhaust all the possible settings in our samples. Would that be fine?
In reply to: 273736506 [](ancestors = 273736506,273735371,273727261)
wschin
wschin
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSaveAndLoad.cs
Outdated

 private class TransformedData : InputData
 {
 public bool IsUnderThirty { get; set; }

@wschinwschinApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.
singlis
singlis
reviewed
Apr 10, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
 // Expected output:
 // Features1: [1, 1, 0] MissingIndicator1: [False, False, False] Features2: [1, 1] MissingIndicator2: [False, False]
 // Features1: [0, NaN, 1] MissingIndicator1: [False, True, False] Features2: [NaN, 1] MissingIndicator2: [True, False]
 // Features1: [-1, NaN, -3] MissingIndicator1: [False, True, False] Features2: [1, ∞] MissingIndicator2: [False, False]
@singlissinglisApr 10, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT - spacing for MissingIndicator doesnt align with the above lines.
singlis
singlis
approved these changes
Apr 10, 2019
View reviewed changes
@singlissinglis
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@codecov
codecovBot
commented
Apr 10, 2019•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #3216 into master will decrease coverage by <.01%.

The diff coverage is n/a.
@@ Coverage Diff @@## master #3216 +/- ##
==========================================
- Coverage 72.62% 72.62% -0.01% 
==========================================
Files 807 807 Lines 145080 145080 Branches 16213 16213 ==========================================
- Hits 105369 105365 -4 - Misses 35294 35298 +4 
Partials 4417 4417
FlagCoverage Δ
#Debug72.62% <ø> (-0.01%)⬇️
#production68.17% <ø> (-0.01%)⬇️
#test88.92% <ø> (-0.01%)⬇️
Impacted FilesCoverage Δ
src/Microsoft.ML.Transforms/ExtensionsCatalog.cs57.14% <ø> (ø)⬆️
...rc/Microsoft.ML.Transforms/CustomMappingCatalog.cs100% <ø> (ø)⬆️
src/Microsoft.ML.Transforms/Text/LdaTransform.cs89.26% <0%> (-0.63%)⬇️
...soft.ML.TestFramework/DataPipe/TestDataPipeBase.cs73.7% <0%> (-0.34%)⬇️
...StandardTrainers/Standard/LinearModelParameters.cs60.05% <0%> (-0.27%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.26% <0%> (+0.15%)⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs85.31% <0%> (+0.6%)⬆️
@artidoro
artidoro
merged commit 304170a
into
dotnet:masterApr 10, 2019
artidoro added a commit
to artidoro/machinelearning
that referenced
this pull request
Apr 10, 2019
@artidoro
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues (
ee2ffdb
dotnet#3216)
@artidoroartidoro
mentioned this pull request
Apr 10, 2019
 Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 23, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@sfilipisfilipiAwaiting requested review from sfilipi

5 more reviewers

@abgoswamabgoswamabgoswam left review comments
@shmoradimsshmoradimsshmoradims left review comments
@rogancarrrogancarrrogancarr left review comments
@wschinwschinwschin approved these changes
@singlissinglissinglis approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@artidoroartidoro

Labels

documentationRelated to documentation of ML.NET

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@artidoro@shmoradims@wschin@abgoswam@singlis@rogancarr

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues - #3216
Merged
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Apr 10, 2019
Merged
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues#3216
artidoro merged 8 commits into
dotnet:masterfrom 
artidoro:samples1
Conversation
@artidoro
@artidoroartidoro
commented
Apr 5, 2019

Copy link


Copy Markdown

Contributor
Related to #1209
Fixes#3117
Made samples for the multi-column setting of:
  • ReplaceMissingValues
  • IndicateMissingValues
I also made a sample to save and load the CustomMapping estimator.
@artidoroartidoro
self-assigned this
Apr 5, 2019
@artidoroartidoro
added
the 
documentation
Related to documentation of ML.NET
label
Apr 5, 2019
abgoswam
abgoswam
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@shmoradims
shmoradims
commented
Apr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

using System;
please remove Sample from filenames: CustomMapingSample -> CustomMapping
Same with the other file #Resolved
Refers to: docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs:1 in d32f027. [](commit_id = d32f027, deletion_comment = False)
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSampleSaveAndLoad.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
 // ReplaceMissingValues is used to create a column where missing values are replaced according to the ReplacementMode.
 var defaultPipeline = mlContext.Transforms.ReplaceMissingValues(new[] {
 new InputOutputColumnPair("MissingReplaced1", "Features1"),
 new InputOutputColumnPair("MissingReplaced2", "Features2")
@shmoradimsshmoradimsApr 5, 2019•

edited by artidoro

Loading

Copy link


Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: we don't have an in-place transformation multicolumn sample. can we transform the data in-place and not have MissingReplaced1/MissingReplaced2?
shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValuesMultiColumn.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/CustomMappingCatalog.cs
Outdated

shmoradims
shmoradims
reviewed
Apr 5, 2019
View reviewed changes
Comment threadsrc/Microsoft.ML.Transforms/ExtensionsCatalog.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

wschin
wschin
reviewed
Apr 6, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs
Outdated

@artidoro
artidoro
commented
Apr 8, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Thank you for the review comments, I have update the code accordingly.
artidoro
artidoro
commented
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Microsoft.ML.Samples.csproj
Outdated

 <PropertyGroup>
 <TargetFramework>netcoreapp2.1</TargetFramework>
 <OutputType>Exe</OutputType>
 <AssemblyOriginatorKeyFile>$(ToolsDir)Test.snk</AssemblyOriginatorKeyFile>
@artidoroartidoroApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default the assemblies are signed using Open.snk. However, if we want to register an assembly in the component catalog using mlContext.ComponentCatalog.RegisterAssembly(), the assembly needs to pass the following condition:
machinelearning/src/Microsoft.ML.Core/ComponentModel/ComponentCatalog.cs

Lines 1025 to 1034
in
b861b5d
privatestaticboolCanContainExtensions(Assemblyassembly)
{
if(assembly.FullName.StartsWith("Microsoft.ML.",StringComparison.Ordinal)
&&HasMLNetPublicKey(assembly))
{
returnfalse;
}
returntrue;
}
So it needs to have a different signature.
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with Eric, it could be even simpler not to sign this assembly. We don't need strong naming for the Samples assembly as it should not be referenced by external code anyways.
rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMapping.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

rogancarr
rogancarr
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/ReplaceMissingValues.cs
Outdated

 // Features: [-1, 2, -3] MissingReplaced: [-1, 2, -3]
 // Features: [-1, NaN, -3] MissingReplaced: [-1, 0, -3]

 // Mean ReplaceMode:
@rogancarrrogancarrApr 9, 2019•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Mean ReplaceMode: [](start = 12, length = 20)
Isn't there one more mode?
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also maximum and minimum, but I don't think it adds much to add them to this sample.
In reply to: 273727261 [](ancestors = 273727261)
@rogancarrrogancarrApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the documentation, if we have more than one, we should have all of them — right now, it looks inconsistent or incomplete.
In reply to: 273735371 [](ancestors = 273735371,273727261)
@artidoroartidoroApr 9, 2019

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather remove the mean replacement mode in this sample then.

We are not trying to exhaust all the possible settings in our samples. Would that be fine?
In reply to: 273736506 [](ancestors = 273736506,273735371,273727261)
wschin
wschin
reviewed
Apr 9, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSaveAndLoad.cs
Outdated

 private class TransformedData : InputData
 {
 public bool IsUnderThirty { get; set; }

@wschinwschinApr 9, 2019

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra empty line.
singlis
singlis
reviewed
Apr 10, 2019
View reviewed changes
Comment threaddocs/samples/Microsoft.ML.Samples/Dynamic/Transforms/IndicateMissingValuesMultiColumn.cs
 // Expected output:
 // Features1: [1, 1, 0] MissingIndicator1: [False, False, False] Features2: [1, 1] MissingIndicator2: [False, False]
 // Features1: [0, NaN, 1] MissingIndicator1: [False, True, False] Features2: [NaN, 1] MissingIndicator2: [True, False]
 // Features1: [-1, NaN, -3] MissingIndicator1: [False, True, False] Features2: [1, ∞] MissingIndicator2: [False, False]
@singlissinglisApr 10, 2019

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT - spacing for MissingIndicator doesnt align with the above lines.
singlis
singlis
approved these changes
Apr 10, 2019
View reviewed changes
@singlissinglis
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:
@codecov
codecovBot
commented
Apr 10, 2019•

edited

Loading

Copy link


Copy Markdown

Codecov Report
Merging #3216 into master will decrease coverage by <.01%.

The diff coverage is n/a.
@@ Coverage Diff @@## master #3216 +/- ##
==========================================
- Coverage 72.62% 72.62% -0.01% 
==========================================
Files 807 807 Lines 145080 145080 Branches 16213 16213 ==========================================
- Hits 105369 105365 -4 - Misses 35294 35298 +4 
Partials 4417 4417
FlagCoverage Δ
#Debug72.62% <ø> (-0.01%)⬇️
#production68.17% <ø> (-0.01%)⬇️
#test88.92% <ø> (-0.01%)⬇️
Impacted FilesCoverage Δ
src/Microsoft.ML.Transforms/ExtensionsCatalog.cs57.14% <ø> (ø)⬆️
...rc/Microsoft.ML.Transforms/CustomMappingCatalog.cs100% <ø> (ø)⬆️
src/Microsoft.ML.Transforms/Text/LdaTransform.cs89.26% <0%> (-0.63%)⬇️
...soft.ML.TestFramework/DataPipe/TestDataPipeBase.cs73.7% <0%> (-0.34%)⬇️
...StandardTrainers/Standard/LinearModelParameters.cs60.05% <0%> (-0.27%)⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs86.26% <0%> (+0.15%)⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs85.31% <0%> (+0.6%)⬆️
@artidoro
artidoro
merged commit 304170a
into
dotnet:masterApr 10, 2019
artidoro added a commit
to artidoro/machinelearning
that referenced
this pull request
Apr 10, 2019
@artidoro
Samples for CustomMapping, IndicateMissingValues, ReplaceMissingValues (
ee2ffdb
dotnet#3216)
@artidoroartidoro
mentioned this pull request
Apr 10, 2019
 Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 23, 2022
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@sfilipisfilipiAwaiting requested review from sfilipi

5 more reviewers

@abgoswamabgoswamabgoswam left review comments
@shmoradimsshmoradimsshmoradims left review comments
@rogancarrrogancarrrogancarr left review comments
@wschinwschinwschin approved these changes
@singlissinglissinglis approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@artidoroartidoro

Labels

documentationRelated to documentation of ML.NET

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

6 participants

@artidoro@shmoradims@wschin@abgoswam@singlis@rogancarr